We've made some additional small adjustments to mitigate the impact of future scans and builds, and we will continue working to optimize / improve as we move forward. At this time, we are currently stable and not expecting any additional slowdowns, queue backups, or blocked jobs and are hereby declaring this incident as resolved.
Posted Oct 24, 2024 - 13:31 PDT
Update
Backlogs have been largely resolved, we know the root cause and will be working to implement several permanent fixes.
Posted Oct 24, 2024 - 11:37 PDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Oct 24, 2024 - 11:34 PDT
Update
We are continuing to investigate the issue, part of the backlog has reduced, however, we are still seeing some elevated build times at this time.
Posted Oct 24, 2024 - 10:03 PDT
Investigating
We are presently experiencing a high backlog load of builds. As a result, database performance is degraded and build scanning is slowed. We are actively investigating the cause and will provide updates.
Posted Oct 24, 2024 - 08:22 PDT
This incident affected: Component analysis (Licensing, Vulnerabilities, Quality) and Web application (Issue scanning).