On Wed, Sep 22, 2021 at 6:13 AM Balazs Gibizer <balazs.gibizer at est.tech> wrote: > Hi, > > > The latest setuptools (58.0) removed support for "use_2to3" [1] > (deprecated since 46.2). Many OpenStack modules defines > decorator==3.4.0 as lower constraints[2]. However, decorator 3.4.0 > cannot be installed anymore with the latest setuptool as it depends on > "use_2to3". > > JFYI as I was looking into some other requirements issues yesterday, I hit this error with anyjson[0] 0.3.3 as well. It's used in a handful of projects[1] and there has not been a release since 2012[2] so this might be a problem in xena. I haven't checked the projects respective gates, but just want to highlight we'll probably have additional fallout from the setuptools change. [0] https://github.com/pypa/setuptools/issues/1120#issuecomment-919239967 [1] https://codesearch.opendev.org/?q=anyjson&i=nope&literal=nope&files=&excludeFiles=&repos= [2] https://pypi.org/project/anyjson/#history > On master, this can be solved easily by bumping the dependency to > decorator 4.0.0. On stable/xena we can still solve it the same way with > a new RC. But on older stable branches such solution might be against > the stable policy as it would require a major bump on our dependencies. > > This issue is not limited to lower-constraints testing it just hit us > there first. A similar break could happen in our upper constraints as > well on old stable branches. > > The root of the problem is that we always use the latest setuptools in > our CI testing even on old stable branches. Zuul's ensure-tox task[4] > installs tox which installs the virtualenv package which bundles the > latest setuptools package. This happens without applying any > constraints. Then tox is used to install the project's dependencies > under lower or upper constraints with the unconstrained setuptools > available. > > During and after yesterday's nova meeting [3] we discussed possible > solutions. > > Option 1: Bump the major version of the decorator dependency on stable. > Pros: > * easy to implement > Cons: > * might against the policy / breaks downstream packagers > * needs to be done in each affected project[3] separately > * not future proof. After a future setuptools release we can see a > similar break with another of our dependencies. > > @Stable Cores: how do you feel about such bump? > > > Option 2: Pin the setuptools version during tox installation > Pros: > * single solution for all affected projects > * future proof against breaks introduced by future setuptools releases > Cons: > * complex change as it probably requires to extend the base task in > zuul itself > > > Option 3: turn off lower-constraints testing > Pros: > * easy to implement > * some of us want to get rid of lower constraints anyhow > Cons: > * needs per project changes > * not future proof against similar break affecting our upper > constraints on old stable branches. > > > Option 4: utilize pyproject.toml[6] to specify build-time requirements > Unfortunately, I'm not sure if it is possible to restrict the maximum > setuptools version with it > > > As a side note I tried applying [tox]requires in tox.ini to restrict > setuptools version[7] but It does not seem to take effect in our > case[8]. > > > @Stable Cores: what do you think what could be the way forward? > > Cheers, > gibi > > > [1] https://setuptools.pypa.io/en/latest/history.html#v58-0-2 > [2] > > https://codesearch.openstack.org/?q=decorator%3D%3D3.4.0&i=nope&literal=nope&files=&excludeFiles=&repos= > [3] > > https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2021-09-21.log.html#t2021-09-21T16:31:49 > [4] > > https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks/main.yaml#L28 > [5] https://tox.readthedocs.io/en/latest/config.html#conf-requires > [6] https://www.python.org/dev/peps/pep-0518/ > [7] https://review.opendev.org/c/openstack/placement/+/810293/1/tox.ini > [8] > > https://zuul.opendev.org/t/openstack/build/f78a0300c8734e7c8475309af1d2e1a4/log/tox/lower-constraints-0.log#7 > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210924/254a210f/attachment.htm>