[openstack-dev] [all] Zuul v3 Status - and Rollback Information

Jean-Philippe Evrard jean-philippe at evrard.me
Tue Oct 3 18:38:57 UTC 2017

On Tue, Oct 3, 2017 at 5:40 PM, Monty Taylor <mordred at inaugust.com> wrote:
> Hey everybody!
> Thank you all for your patience over the last week as we've worked through
> the v3 rollout.  While we anticipated some problems, we've run into a number
> of unexpected issues, some of which will take some time to resolve.  It has
> become unreasonable to continue in our current state while they are being
> worked.
> With that in mind, we are going to perform a partial rollback to Zuul v2
> while we work on it so that teams can get work done. The details of that are
> as follows:
> The project-config repo remains frozen.  Generally we don't want to make
> changes to v2 jobs.  If a change must be made, it will need to be made to
> both v2 and v3 versions.  We will not run the migration script again.
> Zuul v3 will continue to run check and periodic jobs on all repos.  It will
> leave review messages, including +1/-1 votes.
> Our nodepool quota will be allocated 80% to Zuul v2, and 20% to ZUul v3.
> This will slow v2 down slightly, but allow us to continue to exercise v3
> enough to find problems.
> Zuul v2 and v3 can not both gate a project or set of projects.  In general,
> Zuul v2 will be gating all projects, except the few projects that are
> specifically v3-only: zuul-jobs, openstack-zuul-jobs, project-config, and
> zuul itself.
> We appreciate that some projects would prefer to use v3 exclusively, even
> while we continue to work to stabilize it.  However, in order to complete
> our work as quickly as possible, we may need to restart frequently or take
> extended v3 downtimes.  Because of this, and the reduced capacity that v3
> will be running with, we will keep the set of projects gating under v3
> limited to only those necessary.  But keep in mind, v3 will still be running
> check jobs on all projects, so you can continue to iterate on v3 job content
> in check.
> If you modified a script in your repo that is called from a job to work in
> v3, you may need to modify it to be compatible with both.  If you need to
> determine whether you are running under Zuul v2 or under v3 with legacy
> compatibility shims, check for the LOG_PATH environment variable.  It will
> only be present when running under Zuul v2 (and it is the variable that we
> are least likely to add to the v3 compatibility shim).
> Again - thank you all for your patience, and for all of the great work
> people have done working through the issues we've uncovered. As soon as
> we've got a handle on the most critical issues, we'll plan another
> roll-forward ... hopefully in a week or two, but we'll send out status
> updates as we have them.
> Thanks!
> Monty
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I'd like to first thank everyone involved, doing this hard work.

As an extra comment, please remember that Newton EOL is next week, so
it's maybe worth waiting after
that for the next full rollout/roll-forward (whatever the term is!) to
avoid the overlap of critical events in the same timeframe.

Best regards,
Jean-Philippe Evrard (evrardjp)

More information about the OpenStack-dev mailing list