On Wed, Sep 22, 2021, at 5:11 AM, Balazs Gibizer wrote:
Hi,
Snip
The root of the problem is that we always use the latest setuptools in our CI testing even on old stable branches. Zuul's ensure-tox task[4] installs tox which installs the virtualenv package which bundles the latest setuptools package. This happens without applying any constraints. Then tox is used to install the project's dependencies under lower or upper constraints with the unconstrained setuptools available.
During and after yesterday's nova meeting [3] we discussed possible solutions.
Option 1: Bump the major version of the decorator dependency on stable. Pros: * easy to implement Cons: * might against the policy / breaks downstream packagers * needs to be done in each affected project[3] separately * not future proof. After a future setuptools release we can see a similar break with another of our dependencies.
@Stable Cores: how do you feel about such bump?
Option 2: Pin the setuptools version during tox installation Pros: * single solution for all affected projects * future proof against breaks introduced by future setuptools releases Cons: * complex change as it probably requires to extend the base task in zuul itself
Note as mentioned during the meeting yesterday I believe you actually need to pin virtualenv during the tox installation. If you want to pin setuptools it needs to happen when tox creates its virtualenvs. One major con to this approach is now you've stopped testing that your software can deploy with newer versions of setuptools. It doesn't work right this moment, but as circumstances change you'll be without up to date information.
Option 3: turn off lower-constraints testing Pros: * easy to implement * some of us want to get rid of lower constraints anyhow Cons: * needs per project changes * not future proof against similar break affecting our upper constraints on old stable branches.
Option 4: utilize pyproject.toml[6] to specify build-time requirements Unfortunately, I'm not sure if it is possible to restrict the maximum setuptools version with it
As a side note I tried applying [tox]requires in tox.ini to restrict setuptools version[7] but It does not seem to take effect in our case[8].
This didn't work because the requires specification seems to only affect the meta venv that tox uses to bootstrap itself if necessary [9]. Basically this pinned setuptools in the wrong virtualenv. This might work if you pinned virtualenv here as suggested above.
@Stable Cores: what do you think what could be the way forward?
Cheers, gibi
[1] https://setuptools.pypa.io/en/latest/history.html#v58-0-2 [2] https://codesearch.openstack.org/?q=decorator%3D%3D3.4.0&i=nope&literal=nope&files=&excludeFiles=&repos= [3] https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.202... [4] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks/... [5] https://tox.readthedocs.io/en/latest/config.html#conf-requires [6] https://www.python.org/dev/peps/pep-0518/ [7] https://review.opendev.org/c/openstack/placement/+/810293/1/tox.ini [8] https://zuul.opendev.org/t/openstack/build/f78a0300c8734e7c8475309af1d2e1a4/...
[9] https://tox.readthedocs.io/en/latest/example/basic.html#tox-auto-provisionin...