Open Stack

Devananda van der Veen <devananda.vdv at gmail.com> writes:

> Hi all!
>
> This is a follow-up to several summit discussions on
> how-do-we-deprecate-baremetal, a summary of the plan forward, a call to
> raise awareness of the project's status, and hopefully gain some interest
> from folks on nova-core to help with spec and code reviews.
>
> The nova.virt.ironic driver lives in Ironic's git tree today [1]. We're
> cleaning it up and submitting it to Nova again this cycle. I've posted
> specs [2] outlining the design and planned upgrade process. Earlier today,
> we enabled voting in Ironic's check and gate queues for the
> tempest-dsvm-virtual-ironic job. This runs a tempest scenario test [3]
> against devstack, exercising Nova with the Ironic driver to PXE boot a
> virtual machine. It has been running for a few months on Ironic, and has
> been stable for more than a month. However, because Ironic is not
> integrated, we also can't vote in check/gate queues on integrated projects
> (like Nova). We can - and do - report the test result in a non-voting way,
> though that's easy to miss, since it looks like every other non-voting test.
>
> At the summit [4], it was suggested that we make this job report as though
> it were a third-party CI test for a Nova driver. This would be removed at
> the time that Ironic graduates and the job is allowed to vote in the gate.
> Until that time, I'm happy to have the nova.virt.ironic driver reporting as
> a third-party driver (even though it's not) simply to help raise awareness
> (third-party CI jobs are watched more closely than non-voting jobs) and
> decrease the likelihood that Nova developers will inadvertently break
> Ironic's gate.
>
> Given that there's a concrete plan forward, why am I sending this email to
> all three teams? A few reasons:
> - document the plan that we discussed
> - many people from infra and nova were not present during the discussion
> and may not be aware of the details
> - I may have gotten something wrong (it was a long week)
> - and mostly because I don't technically know how to make an upstream job
> report as though it's a third-party job, and am hoping someone wants to
> volunteer to help figure that out

I think it's a reasonable plan.  To elaborate a bit, I think we
identified three categories of jobs that we run:

a) jobs that are voting
b) jobs that are non-voting because they are advisory
c) jobs that are non-voting for policy reasons but we feel fairly
   strongly about

There's a pretty subtle distinction between b and c.  Ideally, there
shouldn't be any.  We've tried to minimize the number of non-voting jobs
to make sure that people don't ignore them.  Nonetheless, it seems that
a large enough number of people still do that non-voting jobs are
considered ineffective in Nova.  I think it's worth noting the potential
danger of de-emphasizing the actual results.  It may make other
non-voting jobs even less effective than they already are.

The intent is to make the jobs described by (c) into voting jobs, but in
a way that they can still be overridden if need be.  The aim is to help
new (eg, incubated) projects join the integrated gate in a way that lets
them prove they are sufficiently mature to do so without impacting the
currently integrated projects.  I believe we're currently thinking that
point is after their integration approval.  If we are comfortable with
incubated projects being able to block the integrated gate earlier, we
could simply make the non-voting jobs voting instead.

Back to the proposal at hand.  I think we should call the kinds of jobs
described in (c) as "non-binding".

The best way to do that is to register a second user with Gerrit for
Zuul to use, and have it report non-binding jobs with a +/- 1 vote in
the check queue that is separate from the normal "Jenkins" vote.  In
order to do that, we will have to modify Zuul to be able to support a
second user, and associate that user with a pipeline.  Then configure a
new "non-binding" pipeline to use that user and run the desired jobs.

Note that a similar problem of curation may occur with the non-binding
jobs.  If we run jobs for the incubated projects Foo and Bar, they will
share a vote in Gerrit, and Nova developers will have to examine the
results of -1 votes; if Bar consistently fails tests, it may need to be
made non-voting or removed to avoid obscuring Foo's results.

I expect the Zuul modification to take an experienced Zuul developer
about 2-3 days to write, or an inexperienced one about a week.  If no
one else has started it by then, I will probably have some time around
the middle of the cycle to hack on it.  In the mean time, we may want to
make sure that the number of non-voting jobs is at a minimum (and
further reduce them if possible), and emphasize to reviewers the
importance of checking posted results.

-Jim