On 11/02/2026 05:13, Takashi Kajinami wrote:
On 2/10/26 10:54 PM, Stephen Finucane wrote:
On Mon, 2026-02-09 at 15:08 +0000, Stephen Finucane wrote:
On Mon, 2026-02-09 at 11:50 +0000, Stephen Finucane wrote:
On Mon, 2026-02-09 at 18:49 +1100, Michael Still wrote:
Hey all,
I've just noticed that setuptools v82.0.0 released todayish and breaks a bunch of Kolla builds (and probably other things) by removing pkg_resources. https://setuptools.pypa.io/en/stable/history.html#v82-0-0 has further details of the release.
setuptools doesn't seem to be listed in upper-constraints, perhaps because it's often provided as a distro package?
Michael
We're seeing this on projects outside of opendev also, such as ORC [1]. The vast majority (though not all, as Takashi notes) of these are due to use of old versions of pbr. These issues are all resolved in recent pbr releases [2]. While we could bump the pbr constraint in upper- constraints for these stable branches, I think it would be wiser to pin setuptools as part of the devstack global venv creation since there were more users of pkg_resources that have only been removed in recent releases. I've proposed devstack changes to do just this for the affected stable branches, which you can find here [3].
I missed the fact that pip uses build isolation by default, which means it creates a custom venv with build dependencies installed before installing a package. Those build dependencies are not capped by default.
I have proposed a follow-up [1] to address this. However, I don't know if this will be sufficient since tools like tox also install packages and the devstack `pip_install` function obviously does not affect those. I don't currently have a good idea how to resolve these issues, since the only constraints mechanism we have for build-time dependencies is the `build-system.requires` setting in `pyproject.toml`. While we could pin setuptools for openstack projects on old stable branches (tedious though it would be), we are likely to run into chicken-and-egg cross-dependency problems that prevent us actually merging these fixes and, worse, we have no way to fix non- OpenStack dependencies.
I've started a discussion with the setuptools folks [1] but previous experience suggests we are unlikely to get much traction there (I'd love to be proven wrong though). If anyone else has good ideas I'd like to hear them.
Just a follow-up on this. I have had some engagement, though not yet from the person I suspect will ultimately make the call (jaraco). There are parallel discussions going on in [1] and [2]. [1] is my issue and asks to restore and simply freeze 'pkg_resources'. [2] is a re-opened issue that asks to publish 'pkg_resources' to PyPI. I'm heavily in favour of the former owing to complexities when 'pkg_resources' is a build-time dependency. Hopefully we'll get some traction here because the alternative, which I suspect will involve a combination of the existing devstack changes *plus* modifications to virtually all our 'tox.ini' / 'nox.py' files on stable branches *plus* changes to distributions like kolla, is going to an outrageous amount of work.
I'm also hoping the former. I've seen that some of the "unmaintained" dependencies (xstatic is the one mentioned earlier, but there are a few more like zake). While I know we should ultimately get rid of these, I don't think we can patch these libraries for timely manner (some of these are even abandoned). While we could try installing the pkg_resources package earlier (before installing any others, like we do for setuptools pin), that's the hack we may not prefer for long term...
im going to spend a few hours and see if i can recate a package on my github that i think we could move under oslo as oslo.pkg_resources that is litrally a snapshot of the setutools 81 content but package it so that it will export the module as pkg_resouces identically to when it was in setuptools for xstatic and other they could install that as a temporay messure and the code woudl "just work" we could perhaps but im hopeing that stpehsn propsal is adopted or a similar python upstream resoltuion is provided. given i dont think maintianing or packaging oslo.pkg_resources woudl be the correct long term solution. for xstatic i would argeu that instead fo fixign them we may want to condier dropting it sue entrily and just using yarn like zuul does to mange JavaScript deps in horizon adding oslo.pkg_resouces as a depency to xtatic however woudl fix all the relevent packages temporally.
Stephen
[1] https://github.com/pypa/setuptools/issues/5174 [2] https://github.com/pypa/setuptools/issues/863
Stephen
[1] https://review.opendev.org/c/openstack/devstack/+/976121
Cheers, Stephen
[1] https://github.com/k-orc/openstack-resource-controller/actions/runs/21818364... [2] https://review.opendev.org/q/project:openstack/pbr+topic:pep517 [3] https://review.opendev.org/q/topic:%22pin-setuptools%22+owner:stephenfin@red...