[openstack-dev] [python3] building tools for the transition

Doug Hellmann doug at doughellmann.com
Wed Jun 20 15:34:10 UTC 2018


I want to thank Nguyễn Trí Hải, Ma Lei, and Huang Zhiping for
agreeing to be a part of the goal champion team for the python3
goal for Stein.

The next step for us is to build some tools to make the process a
little easier.

One of the aspects of this goal that makes it difficult is that we
need to change so many repositories. There are more than 480
repositories associated with official project teams. I do not think
we want to edit their zuul configurations by hand. :-)

It would be ideal if we had a script that could read the
openstack-infra/project-config/zuul.d/projects.yaml file to find
the project settings for a repository and copy those settings into
the right settings file within the repository. We could then review
the patch locally before proposing the change to gerrit.  A second
script to remove the settings from the project-config file, and
then submit that second change as a patch that has a Depends-On
listed for the first patch would also be useful.

Another aspect that makes it difficult is that zuul is very flexible
with how it reads its configuration files. The configuration can
be in .zuul.yaml, zuul.yaml, .zuul.d/*.yaml, or zuul.d/*.yaml.
Projects have not been consistent with the way they have named their
files, and that will make writing a script to automate editing them
more difficult. For example, I found automaton/.zuul.yaml,
rally/zuul.yaml, oslo.config/.zuul.d, and python-ironicclient/zuul.d.

When I was working on adding the lower-constraints jobs, I created some
tools in https://github.com/dhellmann/openstack-requirements-stop-sync
to help create the patches, and we may be able to reuse some of that
code to make similar changes for this goal.
https://github.com/dhellmann/openstack-requirements-stop-sync/blob/master/make_patches.sh
is the script that makes the patches, and
https://github.com/dhellmann/openstack-requirements-stop-sync/blob/master/add_job.py
is the python script that edits the YAML file.

The task for this goal is a little more complicated, since we are
not just adding 1 template to the existing project settings.  We
may have to add several templates and jobs to the existing settings,
merging the two sets together, and removing branch specifiers at
the same time. And we may need to do that in several branches.

Would a couple of you have time to work on the script to prepare
the patches? We can work in the openstack/goal-tools repository so
that we can collaborate on the code in an official OpenStack
repository (instead of using my GitHub project).

Doug



More information about the OpenStack-dev mailing list