Open Stack

On 09/22/2013 01:47 PM, Thomas Goirand wrote:
> On 09/22/2013 11:36 PM, Jeremy Stanley wrote:
>> On 2013-09-22 23:00:26 +0800 (+0800), Thomas Goirand wrote:
>>> I already mentioned in this list that we need to be compatible with
>>> SQLAlchemy 0.8.x. However, I've seen both regressions happening in the
>>> global-requirements.txt and in Ceilometer:
>>>
>>> SQLAlchemy>=0.7.8,<=0.7.99
>>>
>>> Please, do not add <=0.7.99, this breaks badly, and all of OpenStack
>>> really needs to work with SQLAlchemy 0.8.x, which is already in Sid and
>>> Debian testing!
>>>
>>> It'd be nice if it didn't go back and forth for a 2nd time.
>>
>> Out of curiosity, I just did a pickaxe through the
>> openstack/requirements Git repository and confirmed that we have
>> never claimed support for SQLAlchemy 0.8.x at least back to its
>> inception over a year ago (when we first started trying to get a
>> consistent list of our dependencies across the entirety of
>> OpenStack). Individual projects have gone with SQLAlchemy uncapped
>> from time to time, but if they are to be co-installable then we need
>> them limited to the intersection of what the individual projects
>> support.
> 
> When a package has:
> 
> Depends: ${python:Depends}
> 
> then you try to build it with SQLAlchemy 0.7.x, then the following
> happens after the build:
> 
> Depends: python-sqlalchemy (>= 0.7~), python-sqlalchemy (<< 0.8)
> 
> When building with sqlalchemy 0.8.x installed, you get:
> 
> Depends: python-sqlalchemy (>= 0.8~), python-sqlalchemy (<< 0.9)
> 
> So no, we cannot have different versions of sqlalchemy installed for
> different project. However, if you build the whole of OpenStack with one
> single version of SQLAlchemy (0.7.x or 0.8.x), then it does work
> completely, including the various db-sync that we have. So in fact, what
> we should do is just leave the distribution decide which version of
> SQLAlchemy to use, and make sure that we are compatible with both.

Well, unfortunately, we can't do that, because we have to test with
pip-installed things in virtualenvs. But I'm certainly ok with
targetting 0.8.

BTW - we all really need to be better about more aggressively dealing
with things like this earlier in the cycle.

> And
> we currently are, since I raised the topic earlier, and that things have
> been fixed. Adding such a restriction (eg: <<0.8) in different project
> will only bring troubles with unit tests errors when parsing the
> requirements.txt.

We're very much against having requirements.txt differ across the
projects, for exactly this reason. So I certainly agree with you.

> Once more, it is important to note that both Ubuntu 13.10 and Debian
> Sid/Jessie both are using currently SQLAlchemy 0.8.x. In fact,
> SQLAlchemy 0.8 has just migrated to Jessie (from Sid), together with the
> whole of OpenStack grizzly (which I am very happy about, and for which I
> would like to thanks Julien Cristau from the release team who fixed the
> remaining package which was Alembic).
> 
>> Since we now have at least some integration testing on our global
>> requirements list, I uploaded https://review.openstack.org/47745 to
>> see what breaks (though unit tests for individual projects are not
>> covered by this test and may also need fixing).
> 
> The errors we can see in the Jenkins jobs are:
> 
> die 333 'keystone did not start'

I believe this is the key - to our knowledge, openstack does not yet
work with sqlalchemy 0.8. We can't update the requirements for _any_ of
the projects until we can upgrade it for _all_ of them.

> and the other one is an SSL error while trying to download
> thrift-0.9.1.tar.gz (which I believe isn't relevant).

You are correct - there are weird errors downloading thrift from pypi
currently.

> What's important here, is to understand that any error that we have with
> SQLAlchemy 0.8.x should be considered as a bug, and should be addressed.
> Otherwise, we'll have a big problem on both Ubuntu 13.10 and Debian.
> 
> If it was only down to me, I'd say put:
> 
> SQLAlchemy>=0.8.2,<=0.9
>
> in all of our requirements.txt, so that we have something predictable.
> Making a backport package of SQLAlchemy for Precise and Wheezy is
> trivial and not a problem.

I'm fine with that - as soon as it works. Step one is to make that
change to openstack/requirements. This will update the requirement for a
devstack run. As soon as that change _can_ land, then we also need to
make sure that our unittests in every openstack project work with 0.8.

Monty