On 20-10-02 15:40:44, Sebastien Boyron wrote:
Hey all,
Almost all openstack projects are using pbr and setuptools.
A great majority are directly importing setuptools in the code (setup.py) while not explicitly requiring it in the requirements. In these cases , setuptools is only installed thanks to pbr dependency.
Example 1: Having a look on nova code : http://codesearch.openstack.org/?q=setuptools&i=nope&files=&repo...
We can see that setuptools is importer in setup.py to requires pbr whereas neither in *requirements.txt nor in *constraints.txt
Example 2: This is exactly the same for neutron : http://codesearch.openstack.org/?q=setuptools&i=nope&files=&repo...
I discovered this while making some cleaning on rpm-packaging spec files. Spec files should reflect the content of explicits requirements of the related project. Until now there is no issue on that, but to make it proper, setuptools has been removed from all the projects rpm dependencies and relies only on pbr rpm dependency except if there is an explicit requirement on it in the project. If tomorrow, unlikely, pbr evolves and no longer requires setuptools, many things can fail:
- All explicits imports in python code should break.
- RPM generation should break too since it won't be present as a
BuildRequirement.
- RPM installation of old versions will pull the latest pbr version which
will not require anymore and can break the execution.
- RPM build can be held by distribute if there is not setuptools
buildRequired anymore.
As the python PEP20 claims "Explicit is better than implicit." and it should be our mantra on Openstack, especially with this kind of nasty case. https://www.python.org/dev/peps/pep-0020/ I think we should explicitly require setuptools if, and only if, we need to make an explicit import on it.
This will help to have the right requirements into the RPMs while still staying simple and logical; keeping project requirements and RPM requirements in phase.
I am opening the discussion and pointing to this right now, but I think we should wait for the Wallaby release before doing anything on that point to insert this modification into the regular development cycle. On a release point of view all the changes related to this proposal will be released through the classic release process and they will be landed with other projects changes, in other words it will not require a range of specific releases for projects.
*SEBASTIEN BOYRON* Red Hat
Yes, both should be included in requirements.txt if needed/used. I'll also add that if entry_points exist in setup.cfg you will need to have setuptools installed as well.
setuptools/pbr is hard to version manage, (it's managed by the venv install done by devstack at the moment iirc). I'm not sure we have a way of specifying a requirement should be listed without also specifying a version, but it could be as easy as adding it to the minimal set (list) in the check code and having people's gate fail until they fix it :P
From a gentoo perspective, we've been going through and filing bugs for similiar reasons recently (setuptools being a runtime dep as well as a build dep). So this hits outside of Red Hat as well.