Summary: Microsoft applied patches in Azure on Tuesday evening, resulting in estate wide VM restarts. Some services did not correctly start up after the restarts. Monitoring system is setup to alert on this, but none were sent to the out of hours team. Users discovered and reported the issue instead.

Root cause: Our monitoring system’s schedule was not correctly set due to human error, resulting in alerts not sending to our out of hours team. They were unable to respond to the issue as they were not aware of it.

Further Actions: A review of our monitoring system will be undertaken to investigate if there are alternative methods we can employ to ensure there is no recurrence.

Details of this incident have been added after the incident occured.

Impacted Services: Inbound device data is affected, resulting in backlogs and delayed data. This manifested to users as vehicles in the platform were running behind.