<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Hi,</p>
<p>Thanks for the detailed summary gibi!</p>
<p>Let me share my view:</p>
<p>Option 1: Since the time I am reviewing stable patches I remember
that we usually avoided dependency's version bumps (except the
case when the new pip dependency resolver started to work properly
and revealed that lot of the lower constraints settings were
broken) on stable branches. The written rule is "Following the <span class="reference internal">Review guidelines</span>.
Specifically, not allowing backports of
new features, *new* *dependencies*, or backward incompatible
changes" [1].</p>
<p>Option 2: In OpenStack projects (who follows stable policy) we
use global requirements, and upper constraints for stable
branches. Why not use the same for the tools as well? It caused
problem in the past as well (changes in virtualenv, setuptools,
etc). In my opinion this would be better or future proof solution.
Though it's true, we need to find the proper place and way to add
the new constraints when installing tox in ensure-tox zuul role.</p>
<p>Option 3: it is really just a temporary solution (maybe worse
than Option 1), though it is probably acceptable to unblock gate
on stable branches until the appropriate solution will be chosen.
Also worth to emphasize that this problem doesn't only affect
lower-constraints tests, but also could affect branches where we
have old packages (versions) in upper-constraints.txt!</p>
<p>Option 4: I'm not familiar with pyproject.toml, but I'm curious
about the opinions of those who already used/uses it.<br>
</p>
<p><br>
</p>
<p>Long story short: I'd recommend option 2, as I think that is the
best solution and fits the best to OpenStack stable way of
working... If that is possible to add in our zuul job roles.<br>
</p>
<p>Nevertheless, I am curious about further opinions from the
community. Thanks,<br>
</p>
<p>Előd</p>
<p>[1]
<a class="moz-txt-link-freetext" href="https://docs.openstack.org/project-team-guide/stable-branches.html#active-maintenance">https://docs.openstack.org/project-team-guide/stable-branches.html#active-maintenance</a><br>
</p>
<p><br>
</p>
<p>On 2021. 09. 22. 14:11, Balazs Gibizer wrote:</p>
<blockquote type="cite" cite="mid:6J4UZQ.VOBD0LVDTPUX1@est.tech">Hi,
<br>
<br>
<br>
The latest setuptools (58.0) removed support for "use_2to3" [1]
(deprecated since 46.2). Many OpenStack modules defines
decorator==3.4.0 as lower constraints[2]. However, decorator 3.4.0
cannot be installed anymore with the latest setuptool as it
depends on "use_2to3".
<br>
<br>
On master, this can be solved easily by bumping the dependency to
decorator 4.0.0. On stable/xena we can still solve it the same way
with a new RC. But on older stable branches such solution might be
against the stable policy as it would require a major bump on our
dependencies.
<br>
<br>
This issue is not limited to lower-constraints testing it just hit
us there first. A similar break could happen in our upper
constraints as well on old stable branches.
<br>
<br>
The root of the problem is that we always use the latest
setuptools in our CI testing even on old stable branches. Zuul's
ensure-tox task[4] installs tox which installs the virtualenv
package which bundles the latest setuptools package. This happens
without applying any constraints. Then tox is used to install the
project's dependencies under lower or upper constraints with the
unconstrained setuptools available.
<br>
<br>
During and after yesterday's nova meeting [3] we discussed
possible solutions.
<br>
<br>
Option 1: Bump the major version of the decorator dependency on
stable.
<br>
Pros:
<br>
* easy to implement
<br>
Cons:
<br>
* might against the policy / breaks downstream packagers
<br>
* needs to be done in each affected project[3] separately
<br>
* not future proof. After a future setuptools release we can see a
similar break with another of our dependencies.
<br>
<br>
@Stable Cores: how do you feel about such bump?
<br>
<br>
<br>
Option 2: Pin the setuptools version during tox installation
<br>
Pros:
<br>
* single solution for all affected projects
<br>
* future proof against breaks introduced by future setuptools
releases
<br>
Cons:
<br>
* complex change as it probably requires to extend the base task
in zuul itself
<br>
<br>
<br>
Option 3: turn off lower-constraints testing
<br>
Pros:
<br>
* easy to implement
<br>
* some of us want to get rid of lower constraints anyhow
<br>
Cons:
<br>
* needs per project changes
<br>
* not future proof against similar break affecting our upper
constraints on old stable branches.
<br>
<br>
<br>
Option 4: utilize pyproject.toml[6] to specify build-time
requirements
<br>
Unfortunately, I'm not sure if it is possible to restrict the
maximum setuptools version with it
<br>
<br>
<br>
As a side note I tried applying [tox]requires in tox.ini to
restrict setuptools version[7] but It does not seem to take effect
in our case[8].
<br>
<br>
<br>
@Stable Cores: what do you think what could be the way forward?
<br>
<br>
Cheers,
<br>
gibi
<br>
<br>
<br>
[1] <a class="moz-txt-link-freetext" href="https://setuptools.pypa.io/en/latest/history.html#v58-0-2">https://setuptools.pypa.io/en/latest/history.html#v58-0-2</a>
<br>
[2]
<a class="moz-txt-link-freetext" href="https://codesearch.openstack.org/?q=decorator%3D%3D3.4.0&i=nope&literal=nope&files=&excludeFiles=&repos=">https://codesearch.openstack.org/?q=decorator%3D%3D3.4.0&i=nope&literal=nope&files=&excludeFiles=&repos=</a><br>
[3]
<a class="moz-txt-link-freetext" href="https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2021-09-21.log.html#t2021-09-21T16:31:49">https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2021-09-21.log.html#t2021-09-21T16:31:49</a><br>
[4]
<a class="moz-txt-link-freetext" href="https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks/main.yaml#L28">https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks/main.yaml#L28</a><br>
[5] <a class="moz-txt-link-freetext" href="https://tox.readthedocs.io/en/latest/config.html#conf-requires">https://tox.readthedocs.io/en/latest/config.html#conf-requires</a>
<br>
[6] <a class="moz-txt-link-freetext" href="https://www.python.org/dev/peps/pep-0518/">https://www.python.org/dev/peps/pep-0518/</a>
<br>
[7]
<a class="moz-txt-link-freetext" href="https://review.opendev.org/c/openstack/placement/+/810293/1/tox.ini">https://review.opendev.org/c/openstack/placement/+/810293/1/tox.ini</a>
<br>
[8]
<a class="moz-txt-link-freetext" href="https://zuul.opendev.org/t/openstack/build/f78a0300c8734e7c8475309af1d2e1a4/log/tox/lower-constraints-0.log#7">https://zuul.opendev.org/t/openstack/build/f78a0300c8734e7c8475309af1d2e1a4/log/tox/lower-constraints-0.log#7</a><br>
<br>
<br>
<br>
<br>
</blockquote>
</body>
</html>