Open Stack

On 19 September 2015 at 08:03, Doug Hellmann <doug at doughellmann.com> wrote:
> Excerpts from Robert Collins's message of 2015-09-19 07:32:18 +1200:
>> I know this is terrible timing with the release and all, but
>> constraints updates are failing. This is the first evidence - and it
>> doesn't look like a race to me:
>> http://logs.openstack.org/57/221157/10/check/gate-tempest-dsvm-full/18eb440/logs/devstacklog.txt.gz#_2015-09-18_13_51_46_902
>>
>> https://review.openstack.org/#/c/221157/ is the updated review to
>> bring it all together. I'm worried that the incompatibility is going
>> to impact distributors and/or may even be from one of our own recent
>> library releases.
>>
>
> It looks like this is a problem from os-client-config 1.7.0 and later.
> The constraints file has not been updating on new releases, so we're
> still constrained to 1.6.4 in jobs that use the constraints, which is
> why it isn't showing up elsewhere.
>
> To debug, I ran:
>
> git clone openstack/python-openstackclient
> tox -e py27 --notest
> .tox/py27/bin/openstack --debug
> (error)
> .tox/py27/bin/pip install os-client-config==1.7.1
> .tox/py27/bin/openstack --debug
> (error)
> .tox/py27/bin/pip install os-client-config==1.7.0
> .tox/py27/bin/openstack --debug
> (error)
> .tox/py27/bin/pip install os-client-config==1.6.4
> .tox/py27/bin/openstack --debug
> (works)

Monty seems to think that this is a case where we can just roll
forward - I'm going to guess its a grouping thing:
A = os-client-config
B = python-openstackclient

A < x + B < y works
A < x + B >= y breaks
A >= x + B < y breaks
A >= x + B >= y works

And API use of A and B is not itself incompatible at any point:
clients of A and clients of B don't need to change - though one might
argue that A and B are mutually incompatible once A at x released and
that really we should have been able to detect that before releases
were cut of them.

That said, PyPI can express that situation (in that A < x can depend
on B <y and A >= x can depend on B >=y) natively...

But we can't express in OpenStack projects today due to limitations in
pip combined with the g-r syncing process - we take a flattened view
of everything and the algebra for specifiers is per package, not
composable/groupable like this would require.

We don't have a good canned answer here: while the transition is in
progress we're protected (in functional tests, and shortly in unit
tests), but anyone not using constraints will feel the pain.

Even once the transition is complete anyone doing partial upgrades can
be burnt (upgrade only B or only A and it breaks).

Worse, because of limitations in pip (specifically that reverse deps
are not checked when updating packages) there is little we can do to
stop people being broken in the field: we're contributing to pip to
get to the point that those things are possible, but currently its
future work.

So - I think the pragmatic thing here is to:
 - review the CI for A and B here to see if we can prevent the
incompatibilities occuring at all in future transitions like this
[expand-contract is a much safer pattern and should be usable]
 - document in the readme for python-openstackclient that this schism
exists so its discoverable for the supported lifetime of liberty

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud