Open Stack

Hi,

If you've had difficulty overriding jobs in project-templates, please
read and provide feedback on this proposed change.

We tried to make the Zuul v3 configuration language as intuitive as
possible, and incorporated a lot that we learned from our years running
Zuul v2.  One thing that we didn't anticipate was how folks would end up
wanting to use a job in both project-templates *and* local project
stanzas.

Essentially, we had assumed that if you wanted to control how a job was
run, you would add it to a project stanza directly rather that use a
project-template.  It's easy to do so if you use one or the other.
However, it turns out there are lots of good reasons to use both.  For
example, in a project-template we may want to establish a recommended
way to run a job, or that a job should always be run with a set of
related jobs.  Yet a project may still want to indicate that job should
only run on certain changes in that specific repo.

To be very specific -- a very commonly expressed frustration is that a
project can't specify a "files" or "irrelevant-files" matcher to
override a job that appears in a project-template.

Reconciling those is difficult, largely because once Zuul decides to run
a job (for example, by a declaration in a project-template) it is
impossible to dissuade it from running that job by adding any extra
configuration to a project.  We need to tread carefully when fixing
this, because quite a number of related concepts could be affected.  For
instance, we need to preserve branch independence (a change to stop
running a job in one branch shouldn't affect others).  And we need to
preserve the ability for job variants to layer on to each other (a
project-local variant should still be able to alter a variant in a
project-template).

I propose that we remedy this by making a small change to how Zuul
determines that a job should run:

When a job appears multiple times on a project (for instance if it
appears in a project-template and also on the project itself), all of
the project-local variants which match the item's branch must also match
the item in order for the job to run.  In other words, if a job appears
in a project-template used by a project and on the project, then both
must match.

This effectively causes the "files" and "irrelevant-files" attributes on
all of the project-local job definitions matching a given branch to be
combined.  The combination of multiple files matchers behaves as a
union, and irrelevant-files matchers as an intersection.

================  ========  =======  =======
Matcher           Template  Project  Result
================  ========  =======  =======
files             AB        BC       ABC
irrelevant-files  AB        BC       B
================  ========  =======  =======

I believe this will address the shortcoming identified above, but before
we get too far in implementing it, I'd like to ask folks to take a
moment and evaluate whether it will address the issues you've seen, or
if you foresee any problems which I haven't anticipated.

Thanks,

Jim