Open Stack

On 07/10/2013 01:36 AM, David Ripton wrote:
> On 07/09/2013 12:46 PM, Thomas Goirand wrote:
>> On 07/08/2013 08:32 PM, Sean Dague wrote:
>>> On 07/08/2013 04:50 AM, Mark McLoughlin wrote:
>>>> On Mon, 2013-07-08 at 15:53 +0800, Thomas Goirand wrote:
>>>>> Hi,
>>>>>
>>>>> Since python-sqlalchemy 0.8.2 has been uploaded to Sid, Quantum is
>>>>> uninstallable there right now (see #715294).
>>>>>
>>>>> I am wondering: what's wrong with sqlalchemy >= 0.8, so that it is
>>>>> written explicitly in the requirements that we shouldn't use it? Is
>>>>> there a chance that having such a version of sqlalchemy will make
>>>>> all of
>>>>> OpenStack grizzly fail? What are the consequences? Would it be safe to
>>>>> simply patch the requirements file and ignore this?
>>>>
>>>> Don't really have a comment on the specifics, but ...
>>>
>>> The history of the 0.8 cap was the fact that 0.8beta1 (or something
>>> equiv) was uploaded near a freeze point. <0.8 didn't stop 0.8beta1 from
>>> being used (go version numbers).
>>>
>>> Also in 0.8 a piece was spun out as a separate library (I don't remember
>>> exactly which), which causes some build fails. Because it was around a
>>> freeze the cap was the right approach.
>>>
>>> However, projects really should be getting themselves on 0.8 in the
>>> Havana time frame. AFAIK it's really minor changes to work, so should be
>>> a simple review to bump it up.
>>>
>>>      -Sean
>>
>> Indeed, most projects seem to work with the new SQLAlchemy. Though heat
>> fails with a python crash dump:
>>
>>    File
>> "/home/zigo/sources/openstack/grizzly/heat/build-area/heat-2013.1.2/heat/db/sqlalchemy/models.py",
>>
>> line 32, in <module>
>>      class Json(types.TypeDecorator, types.MutableType):
>> AttributeError: 'module' object has no attribute 'MutableType'
>>
>> Indeed, MutableType is gone in SQLAlchemy >= 0.8. I'm therefore unsure
>> what to do to fix the heat package in Sid... :(
>> Any help would be appreciated.
> 
> Yeah, someone who understands the Heat model code well needs to make
> class Json no longer inherit from MutableType.  I hope it would be
> possible to do that in a backward-compatible way so it kept working with
> SQLAlchemy 0.7 in addition to working with 0.8.
> 
>> There's also a big problem with Cinder:
>>
>>    File "/usr/lib/python2.7/dist-packages/migrate/versioning/schema.py",
>> line 91, in runchange
>>      change.run(self.engine, step)
>>    File
>> "/usr/lib/python2.7/dist-packages/migrate/versioning/script/py.py", line
>> 145, in run
>>      script_func(engine)
>>    File
>> "/root/src/cinder/build-area/cinder-2013.1.2/cinder/db/sqlalchemy/migrate_repo/versions/002_quota_class.py",
>>
>> line 42, in upgrade
>>      _warn_on_bytestring=False),
>> TypeError: __init__() got an unexpected keyword argument 'assert_unicode'
>>
>> Unit tests aren't run at all, and cinder refuses to install (because I'm
>> doing the db_sync in the postinst, which fails).
>>
>> Help there would also be appreciated.
>>
>> AFAICT, these are the only packages affected by the SQLAlchemy upgrade.
> 
> Old openstack DB migrations contained a lot of "convert_unicode=True",
> "unicode_error=None" and "_warn_on_bytestring=False" in the Column
> creation code.  Dan Prince removed these from the Nova migrations in
> commit 93dec58156e , when he squashed all the migrations for Grizzly.
> Other projects still have them here and there, and that appears to be
> what is causing the above error.
> 
> I suspect, but haven't proven, that it's possible to get rid of them
> (because Nova did) and that getting rid of them will fix this problem
> (because Nova doesn't have the problem.)  Note that we don't like to
> modify database migrations because of compatibility concerns, so a
> change like this would need to receive a extra review scrutiny to prove
> it couldn't break anything.
> 
> I threw up a patch to excise these arguments from Cinder DB migrations.
> https://review.openstack.org/36302  I will add some comments about how
> scary this type of change is to the review.

Hey, thanks a lot David! I've used your patch, and rebased it for my
Grizzly package (well, it's rather easy to just remove these
convert_unicode stuff), then it built with no error on the unit tests!
Package uploaded, Cinder case closed. Remaining is heat...

Thomas Goirand (zigo)