[infra] A change to Zuul's queuing behavior

Matt Riedemann mriedemos at gmail.com
Fri Dec 7 21:01:55 UTC 2018


On 12/3/2018 3:30 PM, James E. Blair wrote:
> Since some larger projects consume the bulk of cloud resources in our
> system, this can be especially frustrating for smaller projects.  To be
> sure, it impacts everyone, but while larger projects receive a
> continuous stream of results (even if delayed) smaller projects may wait
> hours before seeing results on a single change.
> 
> In order to help all projects maintain a minimal velocity, we've begun
> dynamically prioritizing node requests based on the number of changes a
> project has in a given pipeline.

FWIW, and maybe this is happening across the board right now, but it's 
taking probably ~16 hours to get results on nova changes right now, 
which becomes increasingly frustrating when they finally get a node, 
tests run and then the job times out or something because the node is 
slow (or some other known race test failure).

Is there any way to determine or somehow track how long a change has 
been queued up before and take that into consideration when it's 
re-enqueued? Like take this change:

https://review.openstack.org/#/c/620154/

That took about 3 days to merge with constant rechecks from the time it 
was approved. It would be cool if there was a way to say, from within 50 
queued nova changes (using the example in the original email), let's say 
zuul knew that 10 of those 50 have already gone through one or more 
times and weigh those differently so when they do get queued up, they 
are higher in the queue than maybe something that is just going through 
it's first time.

-- 

Thanks,

Matt



More information about the openstack-discuss mailing list