Resolved -
This afternoon a deploy introduced a mistake that caused database connections of a specific route not to be returned to the pool. This resulted in connection starvation for new requests, resulting in timeouts and 5XX responses starting at 13:59Z. After the rollback of 14:09Z our services were operational again. The bug has been fixed in a new deploy around 15:03Z. We have been monitoring the services and conclude that the incident can now be closed.
Jul 4, 15:21 UTC
Monitoring -
The API became unavailable at 13:59Z. We have rolled back and the API became available again at 14:09Z. We're monitoring the situation.
Jul 4, 14:15 UTC
Investigating -
We are currently investigating this issue.
Jul 4, 13:59 UTC