Open Stack

On 10/2/20 10:16 AM, Matthew Thode wrote:
> On 20-10-02 15:40:44, Sebastien Boyron wrote:
>> Hey all,
>>
>> Almost all openstack projects are using pbr and setuptools.
>>
>> A great majority are directly importing setuptools in the code (setup.py)
>> while not explicitly requiring it in the requirements.
>> In these cases , setuptools is only installed thanks to pbr dependency.
>>
>>
>> Example 1: Having a look on nova code :
>> http://codesearch.openstack.org/?q=setuptools&i=nope&files=&repos=openstack/nova
>>
>> We can see that setuptools is importer in setup.py to requires pbr whereas
>> neither in *requirements.txt nor in *constraints.txt
>>
>>
>> Example 2: This is exactly the same for neutron :
>> http://codesearch.openstack.org/?q=setuptools&i=nope&files=&repos=openstack/neutron
>>
>> I discovered this while making some cleaning on rpm-packaging spec files.
>> Spec files should reflect the content  of explicits requirements of the
>> related project.
>> Until now there is no issue on that, but to make it proper, setuptools has
>> been removed from all the projects rpm dependencies and
>> relies only on pbr rpm dependency except if there is an explicit
>> requirement on it in the project. If tomorrow, unlikely,
>>   pbr evolves and no longer requires setuptools, many things can fail:
>> - All explicits imports in python code should break.
>> - RPM generation should break too since it won't be present as a
>> BuildRequirement.
>> - RPM installation of old versions will pull the latest pbr version which
>> will not require anymore and can break the execution.
>> - RPM build can be held by distribute if there is not setuptools
>> buildRequired anymore.
>>
>> As the python PEP20 claims "Explicit is better than implicit." and it
>> should be our mantra on Openstack, especially with this kind of nasty case.
>> https://www.python.org/dev/peps/pep-0020/
>> I think we should explicitly require setuptools if, and only if, we need to
>> make an explicit import on it.
>>
>> This will help to have the right requirements into the RPMs while still
>> staying simple and logical; keeping project requirements and
>> RPM requirements in phase.
>>
>> I am opening the discussion and pointing to this right now, but I think we
>> should wait for the Wallaby release before doing anything on that point to
>> insert this modification
>> into the regular development cycle. On a release point of view all the
>> changes related to this proposal will be released through the classic
>> release process
>> and they will be landed with other projects changes, in other words it will
>> not require a range of specific releases for projects.
>>
>> *SEBASTIEN BOYRON*
>> Red Hat
> 
> Yes, both should be included in requirements.txt if needed/used.  I'll
> also add that if entry_points exist in setup.cfg you will need to have
> setuptools installed as well.
> 
> setuptools/pbr is hard to version manage, (it's managed by the venv
> install done by devstack at the moment iirc).  I'm not sure we have a
> way of specifying a requirement should be listed without also specifying
> a version, but it could be as easy as adding it to the minimal set
> (list) in the check code and having people's gate fail until they fix it :P
> 
>  From a gentoo perspective, we've been going through and filing bugs for
> similiar reasons recently (setuptools being a runtime dep as well as a
> build dep).  So this hits outside of Red Hat as well.
> 

This was the same conclusion we came to in the Oslo meeting last week. 
We mostly wanted to give people a chance to object before we patch 
bombed all of the non-compliant projects. :-)