UCB Team : build queues/delays 20 November 2018

Due to an internal system issue Cloud Build has been unable to start new builds for the past hour (6AM Central US time). The issue is being worked on now, and any queued builds should be able to start once it's resolved - more updates to follow.

 

Update - the issue is identified and being worked on (a new database index is needed in a system that Cloud Build depends on, due to a tipping point in data volume). The service should be back to normal capacity within the hour, and queued builds will start running at that point. Thanks for everyone's patience.

 

The underlying issue was resolved in the past hour, allowing Cloud Build to start catching up on a whole lot of builds that had queued. At this point it appears to be in general about an hour's backlog, but falling - although due to concurrency or sequencing, some builds may show longer queue times until everything clears.

 

As of 8:05pm UTC last night the build queue had returned to normal status and you should no longer be seeing builds queued or delayed due to this issue.

 

Yesterday afternoon (US time) saw a few unrelated delays; a subset of build attempts got stuck in a bug/error case. That in turn provided a good illustration of that case and it’s being worked on now, and at least one support ticket on it was addressed. That scenario hopefully should continue to be a rare case until it’s fully fixed - but the status page at https://status.cloud.unity3d.com will reflect if this or other queue issues are happening.