[openstack-dev] [DB][Migrations] Switching to using of Alembic
doug.hellmann at dreamhost.com
Wed Jul 17 12:48:41 UTC 2013
On Wed, Jul 17, 2013 at 6:38 AM, Alexei Kornienko <
alexei.kornienko at gmail.com> wrote:
> Please see my comments inline.
> Best regards,
> Alexei Kornienko
> On 07/16/2013 09:16 PM, David Ripton wrote:
>> On 07/16/2013 01:58 PM, Boris Pavlovic wrote:
>> There is no "magic" and we have only 2 ways to end up with this problems
>>> and bugs that could be caused by "manually" migrations merging and tons
>>> of bugs in sqlalchemy-migrate.
>>> 1) step by step (openstack method)
>>> There are special tests "test_migrations" that runs migrations on
>>> real data against all backends. So we should:
>>> a) improve this tests to checks all behaviors // there is a lot of
>>> hidden bugs
>>> b) replace migration (only one small migration) to alembic
>>> c) check that in all backends we made the same changes in schema
>>> d) Merge all old migrations in one using alembic (automatically).
>>> So it could be done in safe way.
>>> 2.a) huge 2 steps
>>> 1. Merge all migrations in one huge manually (drop all tests in test
>>> e.g. In Nova was patch https://review.openstack.org/#**/c/35748/<https://review.openstack.org/#/c/35748/>
>>> I don't believe that there are no mistakes in this migration, and
>>> nobody is able to check it. // because of tons of hidden bugs in old
>>> migrations and sqla-migrate.
>>> 2. Replace this migration in Alembic
>>> I don't believe that there will be way to check that there is no
>>> 2.b) suicide mode (1 big step)
>>> Merge and switch in one step=)
>> We have compacted migrations before, and there's a test document for how
>> to verify that the big migration has exactly the same output as the series
>> of small migrations. See https://wiki.openstack.org/**
>> wiki/Database_migration_**testing<https://wiki.openstack.org/wiki/Database_migration_testing> Dan Prince is the expert on this.
>> I think the right process is:
>> 1. Wait until the very beginning of Icehouse cycle. (But not after we
>> have new migrations for Icehouse.)
>> 2. Compact all migrations into 2xx_havana.py (for SQLAlchemy-migrate)
> As Boris mentioned: "We are able only to convert only grizzly migrations
> not havana (because our customers should be able to switch from grizzly to
> so in the beginning of Icehouse we'll have:
> 2xx_grizzly.py (sqla-migrate)
> 2xx_havana.py (sqla-migrate)
> 2xx_havana.py (sqla-migrate)
> Icehouse migrations...
> So my question is: when we should start using alembic for new migrations?
> We won't be able to deprecate sqla-migrate and switch to alembic in one
> step since we'll have some migrations kept for backwards compatibility.
> I think that to accomplish your plan we should start using alembic for new
> migrations starting from Icehouse.
> If you are ok with it I can change ceilometer migrations implementation to
> work accordingly and we'll be able to use it as a reference in other
As far as I'm concerned for ceilometer, we can start using alembic for
migrations as soon as the dbsync command runs both sets of migrations in
the right sequence. You have a patch in review for that now, so I don't
think we'll need to wait for Icehouse.
>> 3. Test that it's perfect via above test plan plus whatever enhancements
>> we think of.
>> 4. Manually convert 2xx_havana.py (for SQLAlchemy-migrate) into Alembic,
>> and verify that it's still perfect.
>> 5. Deprecate the SQLAlchemy-migrate version and announce that new
>> migrations should be in Alembic.
>> #4 is hard work but not impossible. I have some old code that does 90%
>> of the work, so we only have to do the other 90%.
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.**org <OpenStack-dev at lists.openstack.org>
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev