AU data center integrations not completing
Incident Report for Solver
Postmortem

Service Fabric, the orchastration service used to manage integrations, moved a service from one node to another. This resulted in a cascading failure of integrations in process. Noted in this doc, "_Service Fabric might be responsible for terminating your code package for a variety of reasons. For example, ... load-balancing purposes."

We are reviewing the possibility of adding a flag to prevent core servics from moving nodes while jobs are running.

docs.microsoft.com

Posted May 12, 2021 - 09:17 PDT

Resolved
Several integrations that started at 15:00UTC, did not complete processing. The integrations have been cancelled and will run at the next scheduled time.
Posted May 09, 2021 - 08:00 PDT