Open Stack

On Mon, Nov 20, 2017 at 3:31 PM, David Moreau Simard <dms at redhat.com> wrote:
> Hi,
>
> As the migration of review.rdoproject.org to Zuul v3 draws closer, I'd like
> to open up the discussion around how we want to approach an eventual
> migration to Zuul v3 outside the gate.
> I'd like to take this opportunity to allow ourselves to think outside the
> box, think about how we would like to shape the CI of TripleO from upstream
> to the product and then iterate to reach that goal.
>
> The reason why I mention "outside the gate" is because one of the features
> of Zuul v3 is to dynamically construct its configuration by including
> different repositories.
> For example, the Zuul v3 from review.rdoproject.org can selectively include
> parts of git.openstack.org/openstack-infra/tripleo-ci [1] and it will load
> the configuration found there for jobs, nodesets, projects, etc.
>
> This opens a great deal of opportunities for sharing content or centralizing
> the different playbooks, roles and job parameters in one single repository
> rather than spread across different repositories across the production
> chain.
> If we do things right, this could give us the ability to run the same jobs
> (which can be customized with parameters depending on the environment,
> release, scenario, etc.) from the upstream gate down to
> review.rdoproject.org and the later productization steps.
>
> There's pros and cons to the idea, but this is just an example of what we
> can do with Zuul v3.
>
> Another example of an interesting thought from Sagi is to boot virtual
> machines directly with pre-built images instead of installing the
> undercloud/overcloud every time.
> Something else to think about is how can we leverage all the Ansible things
> from TripleO Quickstart in Zuul v3 natively.
>
> There's of course constraints about what we can and can't do in the upstream
> gate... but let's avoid prematurely blocking ourselves and try to think
> about what we want to do ideally and figure out if, and how, we can do it.
> Whether it's about the things that we would like to do, can't do, or the
> things that don't work, I'm sure the feedback and outcome of this could
> prove useful to improve Zuul.
>
> How would everyone like to proceed ? Should we start an etherpad ? Do some
> "design dession" meetings ?
> I'm willing to help get the ball rolling and spearhead the effort but this
> is a community effort :)
>

So we had a meeting today around this topic and we chatted about two
distinct efforts on this front.  The first one is that we need to
figure out how/where to migrate the review.rdoproject jobs.

Some notes can be found at
https://etherpad.openstack.org/p/rdosf_zuulv3_planning

It was agreed that we should use the openstack-infra/tripleo-ci for
the job configuration for review.rdoproject as this is where we keep
the current upstream openstack Zuul v3 job definitions for tripleo.
The action items for this migration would be:

1) Compile a list of the jobs in review.rdo
  https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml
  https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/jobs/tripleo-upstream.yml
2) Compare this list of jobs to already defined list of jobs in
openstack-infra/tripleoci
  https://github.com/openstack-infra/tripleo-ci/tree/master/zuul.d
3) Determine the ability to reuse existing jobs and convert any
missing jobs as necessary
4) Define new missing jobs in tripleo-ci
5) Import the project/jobs into a zuul v3 for review.rdoproject
6) Test
7) Switch over


The other future actions that need to be discussed around being able
to use Zuul v3 natively require investigating how Zuul should be
executing code from quickstart.  It was mentioned that there might
need to be improvements in Zuul depending on what the execution of
quickstart needs to look like (multiple playbooks, where do the
variables come from, etc). It was also mentioned that we need to
understand/document the expectations that we around what does the
invocation of quickstart actually mean to both a developer and CI and
we shouldn't necessarily just adapt it for primarily CI use case.
Since quickstart in essentially executing ansible, should be be
exposing that or should the v3 be running the exact same interaction
as a developer would.  This sounded like a longer discussion outside
of the scope of getting review.rdoproject switched over to leveraging
Zuul v3.


Thanks,
-Alex



> Thanks !
>
> [1]: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/zuul.d
>
> David Moreau Simard
> Senior Software Engineer | OpenStack RDO
>
> dmsimard = [irc, github, twitter]