Open Stack

On Mon, 2018-12-03 at 13:30 -0800, James E. Blair wrote:
> Hi,
> 
> We recently made a change to how Zuul and Nodepool prioritize node
> requests.  Cloud resources are the major constraint in how long it takes
> Zuul to run test jobs on proposed changes.  Because we're using more
> resources than ever before (but not necessarily because we're doing more
> work -- Clark has been helping to identify inefficiencies in other
> mailing list threads), the amount of time it takes to receive results on
> a change has been increasing.
> 
> Since some larger projects consume the bulk of cloud resources in our
> system, this can be especially frustrating for smaller projects.  To be
> sure, it impacts everyone, but while larger projects receive a
> continuous stream of results (even if delayed) smaller projects may wait
> hours before seeing results on a single change.
> 
> In order to help all projects maintain a minimal velocity, we've begun
> dynamically prioritizing node requests based on the number of changes a
> project has in a given pipeline.
> 
> This means that the first change for every project in the check pipeline
> has the same priority.  The same is true for the second change of each
> project in the pipeline.  The result is that if a project has 50 changes
> in check, and another project has a single change in check, the second
> project won't have to wait for all 50 changes ahead before it gets nodes
> allocated.
i could be imagineing this but is this not how zuul v2 used to work or rather
how the gate was configured a few cycles ago.
i remember in the past when i was working on smaller projects it
was often quicker to submit patches to those instead fo nova for example.

in partacal i remember working on both os-vif and nova in the past and finding
my os-vif jobs would often get started much quicker then nova.
anyway i think this is hopefully a good change for the majority of projects but
it triggered of feeling of deja vu, was this how the gates used to run?
>  As conditions change (requests are fulfilled, changes are
> added and removed) the priorities of any unfulfilled requests are
> adjusted accordingly.
> 
> In the gate pipeline, the grouping is by shared change queue.  But the
> gate pipeline still has a higher overall precedence than check.
> 
> We hope that this will make for a significant improvement in the
> experience for smaller projects without causing undue hardship for
> larger ones.
>  We will be closely observing the new behavior and make any
> necessary tuning adjustments over the next few weeks.  Please let us
> know if you see any adverse impacts, but don't be surprised if you
> notice node requests being filled "out of order".
> 
> -Jim
>