Open Stack

On 09/18/2015 04:29 PM, Robert Collins wrote:
> On 19 September 2015 at 08:03, Doug Hellmann <doug at doughellmann.com> wrote:
>> Excerpts from Robert Collins's message of 2015-09-19 07:32:18 +1200:
>>> I know this is terrible timing with the release and all, but
>>> constraints updates are failing. This is the first evidence - and it
>>> doesn't look like a race to me:
>>> http://logs.openstack.org/57/221157/10/check/gate-tempest-dsvm-full/18eb440/logs/devstacklog.txt.gz#_2015-09-18_13_51_46_902
>>>
>>> https://review.openstack.org/#/c/221157/ is the updated review to
>>> bring it all together. I'm worried that the incompatibility is going
>>> to impact distributors and/or may even be from one of our own recent
>>> library releases.
>>>
>>
>> It looks like this is a problem from os-client-config 1.7.0 and later.
>> The constraints file has not been updating on new releases, so we're
>> still constrained to 1.6.4 in jobs that use the constraints, which is
>> why it isn't showing up elsewhere.
>>
>> To debug, I ran:
>>
>> git clone openstack/python-openstackclient
>> tox -e py27 --notest
>> .tox/py27/bin/openstack --debug
>> (error)
>> .tox/py27/bin/pip install os-client-config==1.7.1
>> .tox/py27/bin/openstack --debug
>> (error)
>> .tox/py27/bin/pip install os-client-config==1.7.0
>> .tox/py27/bin/openstack --debug
>> (error)
>> .tox/py27/bin/pip install os-client-config==1.6.4
>> .tox/py27/bin/openstack --debug
>> (works)
>
> Monty seems to think that this is a case where we can just roll
> forward - I'm going to guess its a grouping thing:
> A = os-client-config
> B = python-openstackclient
>
> A < x + B < y works
> A < x + B >= y breaks
> A >= x + B < y breaks
> A >= x + B >= y works

Yeah. Actually, I believe we've found a case where the system is doing 
its job - it just took us a sec to see that.

I've got a new patch up to os-client-config which fixes the problem 
without requiring modifications to openstackclient- which is how this 
should work. It's an ugly patch - but meh, life is ugly.

> And API use of A and B is not itself incompatible at any point:
> clients of A and clients of B don't need to change - though one might
> argue that A and B are mutually incompatible once A at x released and
> that really we should have been able to detect that before releases
> were cut of them.
>
> That said, PyPI can express that situation (in that A < x can depend
> on B <y and A >= x can depend on B >=y) natively...
>
> But we can't express in OpenStack projects today due to limitations in
> pip combined with the g-r syncing process - we take a flattened view
> of everything and the algebra for specifiers is per package, not
> composable/groupable like this would require.
>
> We don't have a good canned answer here: while the transition is in
> progress we're protected (in functional tests, and shortly in unit
> tests), but anyone not using constraints will feel the pain.
>
> Even once the transition is complete anyone doing partial upgrades can
> be burnt (upgrade only B or only A and it breaks).
>
> Worse, because of limitations in pip (specifically that reverse deps
> are not checked when updating packages) there is little we can do to
> stop people being broken in the field: we're contributing to pip to
> get to the point that those things are possible, but currently its
> future work.
>
> So - I think the pragmatic thing here is to:
>   - review the CI for A and B here to see if we can prevent the
> incompatibilities occuring at all in future transitions like this
> [expand-contract is a much safer pattern and should be usable]
>   - document in the readme for python-openstackclient that this schism
> exists so its discoverable for the supported lifetime of liberty
>
> -Rob
>