Open Stack

Excerpts from Doug Hellmann's message of 2018-06-20 11:34:10 -0400:
> I want to thank Nguyễn Trí Hải, Ma Lei, and Huang Zhiping for
> agreeing to be a part of the goal champion team for the python3
> goal for Stein.
> 
> The next step for us is to build some tools to make the process a
> little easier.
> 
> One of the aspects of this goal that makes it difficult is that we
> need to change so many repositories. There are more than 480
> repositories associated with official project teams. I do not think
> we want to edit their zuul configurations by hand. :-)
> 
> It would be ideal if we had a script that could read the
> openstack-infra/project-config/zuul.d/projects.yaml file to find
> the project settings for a repository and copy those settings into
> the right settings file within the repository. We could then review
> the patch locally before proposing the change to gerrit.  A second
> script to remove the settings from the project-config file, and
> then submit that second change as a patch that has a Depends-On
> listed for the first patch would also be useful.
> 
> Another aspect that makes it difficult is that zuul is very flexible
> with how it reads its configuration files. The configuration can
> be in .zuul.yaml, zuul.yaml, .zuul.d/*.yaml, or zuul.d/*.yaml.
> Projects have not been consistent with the way they have named their
> files, and that will make writing a script to automate editing them
> more difficult. For example, I found automaton/.zuul.yaml,
> rally/zuul.yaml, oslo.config/.zuul.d, and python-ironicclient/zuul.d.
> 
> When I was working on adding the lower-constraints jobs, I created some
> tools in https://github.com/dhellmann/openstack-requirements-stop-sync
> to help create the patches, and we may be able to reuse some of that
> code to make similar changes for this goal.
> https://github.com/dhellmann/openstack-requirements-stop-sync/blob/master/make_patches.sh
> is the script that makes the patches, and
> https://github.com/dhellmann/openstack-requirements-stop-sync/blob/master/add_job.py
> is the python script that edits the YAML file.
> 
> The task for this goal is a little more complicated, since we are
> not just adding 1 template to the existing project settings.  We
> may have to add several templates and jobs to the existing settings,
> merging the two sets together, and removing branch specifiers at
> the same time. And we may need to do that in several branches.
> 
> Would a couple of you have time to work on the script to prepare
> the patches? We can work in the openstack/goal-tools repository so
> that we can collaborate on the code in an official OpenStack
> repository (instead of using my GitHub project).
> 
> Doug

I started working on these tools today. Given the complexity, for now
the two commands only print the expected settings. The 'jobs extract'
command shows which job settings should be copied from project-config to
the zuul settings in a given branch of a repository, and 'jobs retain'
shows which jobs settings should remain in place.

Please look over the changeset
(https://review.openstack.org/#/c/578194/) and leave review comments if
you have them.

Doug