Open Stack

We're making good progress. Some of the important parts of the
global job changes are in place. There are still a lot of open
patches to add the lower-constraints jobs to repos, however.

Excerpts from Doug Hellmann's message of 2018-03-15 07:03:11 -0400:

[...]
> What I Want to Do
> -----------------
> 
> 1. Update the requirements-check test job to change the check for
>    an exact match to be a check for compatibility with the
>    upper-constraints.txt value.

This change has merged: https://review.openstack.org/#/c/555402/

There are some additional changes to that job still in the queue.
In particular, the change in https://review.openstack.org/#/c/557034/3
will start enforcing some rules to ensure the lower-constraints.txt
settings stay at the bottom of the requirements files.

Because we had some communication issues and did a few steps out
of order, when this patch lands projects that have approved
bot-proposed requirements updates may find that their requirements
and lower-constraints files no longer match, which may lead to job
failures. It should be easy enough to fix the problems by making
the values in the constraints files match the values in the
requirements files (by editing either set of files, depending on
what is appropriate). I apologize for any inconvenience this causes.

> 2. We should stop syncing dependencies by turning off the
>    propose-update-requirements job entirely.

This is also done: https://review.openstack.org/#/c/555426/

> 3. Remove the minimum specifications from the global requirements
>    list to make clear that the global list is no longer expressing
>    minimums.
> 
>    This clean-up step has been a bit more controversial among the
>    requirements team, but I think it is a key piece. As the minimum
>    versions of dependencies diverge within projects, there will no
>    longer *be* a real global set of minimum values. Tracking a list of
>    "highest minimums", would either require rebuilding the list from the
>    settings in all projects, or requiring two patches to change the
>    minimum version of a dependency within a project.
> 
>    Maintaining a global list of minimums also implies that we
>    consider it OK to run OpenStack as a whole with that list. This
>    message conflicts with the message we've been sending about the
>    upper constraints list since that was established, which is that
>    we have a known good list of versions and deploying all of
>    OpenStack with different versions of those dependencies is
>    untested.

We've decided not to do this step, because some of the other
requirements team members want to use those lower bound values.
Projects are no longer required to be consistent with the lower
bounds in that global file, however.

> Testing Lower Bounds of Dependencies
> ------------------------------------
[...]
> 
> The results of those steps can be combined into a single patch and
> proposed to the project. To avoid overwhelming zuul's job configuration
> resolver, we need to propose the patches in separate batches of
> about 10 repos at a time. This is all mostly scriptable, so I will
> write a script and propose the patches (unless someone else wants to do
> it all -- we need a single person to keep up with how many patches we're
> proposing at one time).
> 
> The point of creating the initial lower-constraints.txt file is not
> necessarily to be "accurate" with the constraints immediately, but
> to have something to work from. After the patches are proposed,
> please either plan to land them or vote -2 indicating that you don't
> want a job like that on that repo. If you want to change the
> constraints significantly, please do that in a separate patch. With
> ~325 of them, I'm not going to be able to keep up with everyone's
> separate needs and this is all meant to just establish the initial
> version of the job anyway.

I ended up needing fewer patches than expected because many of the
projects receiving requirements syncs didn't have unit test jobs
(ansible roles, and some other packaging-related things, that are tested
other ways).

Approvals have been making good progress. As I say above, if you
have minor issues with the patch, either propose a fix on top of
it or take it over and fix it directly. Even though there are fewer
patches than I expected, I'm still not going to be able to be able
to keep up with lots of individual differences or merge conflicts
in projects. Help wanted.

> For projects that currently only support python 2 we can modify the
> proposed patches to not set base-python to use python3.
> 
> You will have noticed that this will only apply to unit test jobs.
> Projects are free to use the results to add their own functional
> test jobs using the same lower-constraints.txt files, but that's
> up to them to do.

I'm not aware of anyone trying to do this, yet. If you are, please let
us know how it's going.

Doug