Open Stack

On Wed, Jul 17, 2013 at 6:38 AM, Alexei Kornienko <
alexei.kornienko at gmail.com> wrote:

> Hello,
>
> Please see my comments inline.
>
> Best regards,
> Alexei Kornienko
>
>
> On 07/16/2013 09:16 PM, David Ripton wrote:
>
>> On 07/16/2013 01:58 PM, Boris Pavlovic wrote:
>>
>>  There is no "magic" and we have only 2 ways to end up with this problems
>>> and bugs that could be caused by "manually" migrations merging and tons
>>> of bugs in sqlalchemy-migrate.
>>>
>>> 1) step by step (openstack method)
>>>    There are special tests "test_migrations" that runs migrations on
>>> real data against all backends. So we should:
>>>
>>>    a) improve this tests to checks all behaviors // there is a lot of
>>> hidden bugs
>>>    b) replace migration (only one small migration) to alembic
>>>    c) check that in all backends we made the same changes in schema
>>>    d) Merge all old migrations in one using alembic (automatically).
>>>    So it could be done in safe way.
>>>
>>> 2.a) huge 2 steps
>>>    1. Merge all migrations in one huge manually (drop all tests in test
>>> migrations)
>>>        e.g. In Nova was patch https://review.openstack.org/#**/c/35748/<https://review.openstack.org/#/c/35748/>
>>>        I don't believe that there are no mistakes in this migration, and
>>> nobody is able to check it. // because of tons of hidden bugs in old
>>> migrations and sqla-migrate.
>>>    2. Replace this migration in Alembic
>>>         I don't believe that there will be way to check that there is no
>>> bugs
>>>
>>> 2.b) suicide mode (1 big step)
>>>    Merge and switch in one step=)
>>>
>>
>> We have compacted migrations before, and there's a test document for how
>> to verify that the big migration has exactly the same output as the series
>> of small migrations.  See https://wiki.openstack.org/**
>> wiki/Database_migration_**testing<https://wiki.openstack.org/wiki/Database_migration_testing> Dan Prince is the expert on this.
>>
>> I think the right process is:
>>
>> 1. Wait until the very beginning of Icehouse cycle.  (But not after we
>> have new migrations for Icehouse.)
>>
>> 2. Compact all migrations into 2xx_havana.py (for SQLAlchemy-migrate)
>>
> As Boris mentioned: "We are able only to convert only grizzly migrations
> not havana (because our customers should be able to switch from grizzly to
> havana)"
> so in the beginning of Icehouse we'll have:
> 2xx_grizzly.py (sqla-migrate)
> 2xx_havana.py (sqla-migrate)
> ...
> 2xx_havana.py (sqla-migrate)
> ___________________
> Icehouse migrations...
>
> So my question is: when we should start using alembic for new migrations?
> We won't be able to deprecate sqla-migrate and switch to alembic in one
> step since we'll have some migrations kept for backwards compatibility.
> I think that to accomplish your plan we should start using alembic for new
> migrations starting from Icehouse.
> If you are ok with it I can change ceilometer migrations implementation to
> work accordingly and we'll be able to use it as a reference in other
> projects.


As far as I'm concerned for ceilometer, we can start using alembic for
migrations as soon as the dbsync command runs both sets of migrations in
the right sequence. You have a patch in review for that now, so I don't
think we'll need to wait for Icehouse.

Doug


>
>
>
>> 3. Test that it's perfect via above test plan plus whatever enhancements
>> we think of.
>>
>> 4. Manually convert 2xx_havana.py (for SQLAlchemy-migrate) into Alembic,
>> and verify that it's still perfect.
>>
>> 5. Deprecate the SQLAlchemy-migrate version and announce that new
>> migrations should be in Alembic.
>>
>> #4 is hard work but not impossible.  I have some old code that does 90%
>> of the work, so we only have to do the other 90%.
>>
>>
>
> ______________________________**_________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.**org <OpenStack-dev at lists.openstack.org>
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130717/28de2586/attachment.html>