[infra] A change to Zuul's queuing behavior
Matt Riedemann
mriedemos at gmail.com
Fri Dec 7 21:01:55 UTC 2018
On 12/3/2018 3:30 PM, James E. Blair wrote:
> Since some larger projects consume the bulk of cloud resources in our
> system, this can be especially frustrating for smaller projects. To be
> sure, it impacts everyone, but while larger projects receive a
> continuous stream of results (even if delayed) smaller projects may wait
> hours before seeing results on a single change.
>
> In order to help all projects maintain a minimal velocity, we've begun
> dynamically prioritizing node requests based on the number of changes a
> project has in a given pipeline.
FWIW, and maybe this is happening across the board right now, but it's
taking probably ~16 hours to get results on nova changes right now,
which becomes increasingly frustrating when they finally get a node,
tests run and then the job times out or something because the node is
slow (or some other known race test failure).
Is there any way to determine or somehow track how long a change has
been queued up before and take that into consideration when it's
re-enqueued? Like take this change:
https://review.openstack.org/#/c/620154/
That took about 3 days to merge with constant rechecks from the time it
was approved. It would be cool if there was a way to say, from within 50
queued nova changes (using the example in the original email), let's say
zuul knew that 10 of those 50 have already gone through one or more
times and weigh those differently so when they do get queued up, they
are higher in the queue than maybe something that is just going through
it's first time.
--
Thanks,
Matt
More information about the openstack-discuss
mailing list