On Wed, Sep 22, 2021, at 5:11 AM, Balazs Gibizer wrote: > Hi, > Snip > > The root of the problem is that we always use the latest setuptools in > our CI testing even on old stable branches. Zuul's ensure-tox task[4] > installs tox which installs the virtualenv package which bundles the > latest setuptools package. This happens without applying any > constraints. Then tox is used to install the project's dependencies > under lower or upper constraints with the unconstrained setuptools > available. > > During and after yesterday's nova meeting [3] we discussed possible > solutions. > > Option 1: Bump the major version of the decorator dependency on stable. > Pros: > * easy to implement > Cons: > * might against the policy / breaks downstream packagers > * needs to be done in each affected project[3] separately > * not future proof. After a future setuptools release we can see a > similar break with another of our dependencies. > > @Stable Cores: how do you feel about such bump? > > > Option 2: Pin the setuptools version during tox installation > Pros: > * single solution for all affected projects > * future proof against breaks introduced by future setuptools releases > Cons: > * complex change as it probably requires to extend the base task in > zuul itself Note as mentioned during the meeting yesterday I believe you actually need to pin virtualenv during the tox installation. If you want to pin setuptools it needs to happen when tox creates its virtualenvs. One major con to this approach is now you've stopped testing that your software can deploy with newer versions of setuptools. It doesn't work right this moment, but as circumstances change you'll be without up to date information. > > > Option 3: turn off lower-constraints testing > Pros: > * easy to implement > * some of us want to get rid of lower constraints anyhow > Cons: > * needs per project changes > * not future proof against similar break affecting our upper > constraints on old stable branches. > > > Option 4: utilize pyproject.toml[6] to specify build-time requirements > Unfortunately, I'm not sure if it is possible to restrict the maximum > setuptools version with it > > > As a side note I tried applying [tox]requires in tox.ini to restrict > setuptools version[7] but It does not seem to take effect in our > case[8]. This didn't work because the requires specification seems to only affect the meta venv that tox uses to bootstrap itself if necessary [9]. Basically this pinned setuptools in the wrong virtualenv. This might work if you pinned virtualenv here as suggested above. > > > @Stable Cores: what do you think what could be the way forward? > > Cheers, > gibi > > > [1] https://setuptools.pypa.io/en/latest/history.html#v58-0-2 > [2] > https://codesearch.openstack.org/?q=decorator%3D%3D3.4.0&i=nope&literal=nope&files=&excludeFiles=&repos= > [3] > https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2021-09-21.log.html#t2021-09-21T16:31:49 > [4] > https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks/main.yaml#L28 > [5] https://tox.readthedocs.io/en/latest/config.html#conf-requires > [6] https://www.python.org/dev/peps/pep-0518/ > [7] https://review.opendev.org/c/openstack/placement/+/810293/1/tox.ini > [8] > https://zuul.opendev.org/t/openstack/build/f78a0300c8734e7c8475309af1d2e1a4/log/tox/lower-constraints-0.log#7 [9] https://tox.readthedocs.io/en/latest/example/basic.html#tox-auto-provisioning