Open Stack

Hi,

Something we need to be aware of for the Zuul v3 migration is the use of
the zuul-cloner tool.  In many of the more "standardized" jobs, this
will be taken care of either by the fact that we have reworked the jobs
into native v3 format (eg unittests), or the migration script will map
existing JJB builders (zuul-git-prep) into v3 equivalents.  However,
there are many jobs which perform their own zuul-cloner actions, some
defined in project-config, and others scattered throughout project
repos.

In those cases, it would take considerable time to track down all the
uses and put a migration plan in place.  Instead, I propose a
compatibility shim.

We can create a new script which takes the same arguments as
zuul-cloner.  It will ignore most of the arguments except the repos and
the destination which it will use to copy the specified repos from
~zuul/src/git.openstack.org/... to the destination.

When jobs run this script, they will get the Zuul v3 prepared repos, and
they will be placed in the locations that the job expects.  If it can
support the clonemap file, that's even better.

If jobs provide arguments to zuul-cloner which alter its behavior about
which branches are checked out, they may fail with this plan.  It will
be easy to correct this by altering the Zuul v3 job configuration to
specify override branches, however, this is likely a situation which
will require human intervention to fix.  Ideally, there won't be very
many of these cases.

Additionally, any of the repos used by zuul-cloner in a job will need to
be specified as required-projects in Zuul.  Determining this
automatically is hard, but we can handle many cases with heuristics.
For example, a substantial number of these jobs are for neutron
plugins.  We can easily tell the migration script to add 'neutron' as a
required-project for all such jobs.

Zuul-cloner is baked into our images.  In order to allow us to easily
move between Zuul v2 and v3, I propose we have the Zuul v3 base job
automatically replace the zuul-cloner script with the one proposed here,
so we can continue to use the same images for Zuul v2 and v3 during the
transition.

Ultimately, it would be useful to have this in Zuul v3 for other users.
It probably makes the most sense to add this to the feature/zuulv3
branch as the zuul-cloner binary.  We can release it as a deprecated
migration tool.  The base job can just wget it from git.o.o for
simplicity.  To facilitate that, we may want to ensure the script is
self-contained.  After we fully migrate to v3, we can bake it into the
image again.

-Jim