[openstack-dev] [all] requirements-py{2, 3} and universal wheels

Monty Taylor mordred at inaugust.com
Tue Mar 17 13:33:10 UTC 2015


On 03/17/2015 09:07 AM, Monty Taylor wrote:
> On 03/16/2015 08:32 PM, Robert Collins wrote:
>> On 17 March 2015 at 13:22, Doug Hellmann <doug at doughellmann.com> wrote:
>>> Excerpts from Robert Collins's message of 2015-03-17 12:54:00 +1300:
>>>> I've raised this in reviews 157135 and 153966, but I think it deserves
>>>> a thread of its own.
>>>>
>>>> I think universal wheels are useful - they are simple to build and
>>>> publish - we don't need to do one wheel per Python version.
>>>>
>>>> However, right now I'm fairly sure that we're not exporting the
>>>> requirements from requirements-py2 / requirements-py3 as environment
>>>> markers (see PEP-426).
>>>>
>>>> That means that a wheel built on Python2 for a pbr project using
>>>> requirements-pyN files, even if marked as a universal wheel, will only
>>>> have the requirements for the Python2 deps.
>>>
>>> You're right that they only include the requirements for python 2. We
>>> try not to mark those packages as "universal" for that reason.
>>>
>>>>
>>>> This is broken - I've filed a bug about it (http://pad.lv/1431529).
>>>>
>>>> I think we should deprecate and remove the requirements-pyN files and
>>>> instead use environment markers directly in requirements.txt. That
>>>> will then flow into wheels and things should just work (plus we can
>>>> delete more pbr code).
>>>>
>>>> I haven't tested yet (and someone should) that it does all JUST WORK,
>>>> but thats easy: put an environment marker in a requirements.txt file
>>>> like so:
>>>>
>>>>  argparse; python_version < '3'
>>>
>>> I think the last time this came up the feature wasn't available in pip
>>> yet, and so using separate files was the work-around. Are environment
>>> markers fully supported by pip/setuptools/whatever now?
>>
>> Donald says yes, at least for pip (which is all we need, since we
>> advise folk to use pip install -e . locally).
> 
> Not just advise - setup.py install is _explicitly_ not supported since
> it is broken by design and insecure. I've spoken with Donald about
> trying to figure out a way to determine if we're being run via straight
> setup.py install rather than via pip so that we can error descriptively...
> 
> In any case:
> 
> a) woot
> b) I agree, pip support is all we need
> 
>>> If so, an option would be to have pbr recognize the version-specific
>>> input files as implying a particular rule, and adding that environment
>>> marker to the dependencies list automatically until we can migrate to a
>>> single requirements.txt (for which no rules would be implied).
>>
>> We could, or we could just migrate - I don't think its worth writing a
>> compat shim.
> 
> Also agree.

Actually - no, I just realized - we need to do a compat shim - because
pbr has no such thing as a stable release or ability to be versioned. We
have requirements-pyX in the wild, which means we must support them
basically until the end of time.

So I'm going to propose that we add a shim such as the one dhellmann
suggests above so that pbr will support our old releases, but moving
forward as a project, we should use markers and not requirements-pyX



More information about the OpenStack-dev mailing list