Azure clusters not reconciling

Incident Report for CASTAI

Resolved

All impacted CAST AI managed AKS clusters recovered. Azure team identified the change that caused an issue and rolled it back.

Posted Jan 25, 2023 - 09:38 UTC

Update

Azure team, continues the investigation in to Networking issue affecting multiple regions. https://status.azure.com/en-gb/status

Posted Jan 25, 2023 - 09:00 UTC

Identified

Azure clusters are failing to reconcile with Azure API related errors. Our investigation points to Azure API that is currently experiencing outage (multiple regions are affected). Currently in the console you might see a number of clusters in "Warning" or "Failed" sates.

Posted Jan 25, 2023 - 08:44 UTC

This incident affected: Uptime of public API: api.cast.ai and Autoscaler.