Open Stack

It is an even year again which means there is a new Ubuntu LTS release
out (Xenial). We currently have Xenial  images building and uploaded and
the Openstack Ansible and Openstack Puppet groups are both taking
advantage of them. Our "base" set of devstack jobs also seem happy on
Xenial as well (see the experimental results on
https://review.openstack.org/#/c/330835/1). All this to say I think we
are ready to do the big switch and get everyone running on Xenial by
default.

The tough part of this switch is that we will want to continue running
stable/liberty and stable/mitaka jobs on Trusty and restrict Xenial to
master (Newton) and future branches. Now if we want
gate-tempest-dsvm-full to run on Trusty nodes for stable branches and
Xenial for master we need some way of signaling that between gearman
client and workers. I think we have two options for doing this (but
really hope I am overlooking something because I am not super happy with
these options).

The first is we can do what we did for the Precise/Trusty split. We used
zuul parameter functions to set the node type based on the branch the
build was for. This meant by default everything on old stable went to
precise and everything on newer branches went to Trusty. If you wanted
to do anything else like run on CentOS or Fedora then you had to have an
explicit override in that parameter function. You can see what that
looks like in the change that removed it,
https://review.openstack.org/#/c/260214/.

The big downside to this option is it created confusion for people not
realizing they need explicit override to break out of the default
Precise/Trusty or Trusty/Xenial split. The upside to this is we can
define each job once and for many jobs/projects/individuals they never
have to think about where the job will run, it is handled for them
properly.

The other option is we can explicitly tie every job to a specific node
type. This means gate-tempest-dsvm-full becomes
gate-tempest-dsvm-full-trusty and gate-tempest-dsvm-full-xenial (or
similar). Then in the zuul layout we default to running old stable
against any job ending in -trusty and run master against any job ending
in -xenial. I have mocked this up in
https://review.openstack.org/335166.

The downside here is we double our total number of jobs (though we do
not double the number of gearman job registrations since gearman will
register a job per node type regardless of option used). It is also much
more explicit and will likely require a greater understanding of our job
configs to edit them (this isn't all bad, where things used to mostly
work before by magic they will now work by explicit design).

I would like to start transitioning jobs to Xenial soon so feedback on
this is appreciated. Also if you can come up with better options I would
love to hear about them as I am not entirely happy about the options
above.

Thank you,
Clark