[openstack-dev] [DB][Migrations] Switching to using of Alembic
alexei.kornienko at gmail.com
Wed Jul 17 10:38:40 UTC 2013
Please see my comments inline.
On 07/16/2013 09:16 PM, David Ripton wrote:
> On 07/16/2013 01:58 PM, Boris Pavlovic wrote:
>> There is no "magic" and we have only 2 ways to end up with this problems
>> and bugs that could be caused by "manually" migrations merging and tons
>> of bugs in sqlalchemy-migrate.
>> 1) step by step (openstack method)
>> There are special tests "test_migrations" that runs migrations on
>> real data against all backends. So we should:
>> a) improve this tests to checks all behaviors // there is a lot of
>> hidden bugs
>> b) replace migration (only one small migration) to alembic
>> c) check that in all backends we made the same changes in schema
>> d) Merge all old migrations in one using alembic (automatically).
>> So it could be done in safe way.
>> 2.a) huge 2 steps
>> 1. Merge all migrations in one huge manually (drop all tests in test
>> e.g. In Nova was patch https://review.openstack.org/#/c/35748/
>> I don't believe that there are no mistakes in this migration, and
>> nobody is able to check it. // because of tons of hidden bugs in old
>> migrations and sqla-migrate.
>> 2. Replace this migration in Alembic
>> I don't believe that there will be way to check that there is no
>> 2.b) suicide mode (1 big step)
>> Merge and switch in one step=)
> We have compacted migrations before, and there's a test document for
> how to verify that the big migration has exactly the same output as
> the series of small migrations. See
> https://wiki.openstack.org/wiki/Database_migration_testing Dan Prince
> is the expert on this.
> I think the right process is:
> 1. Wait until the very beginning of Icehouse cycle. (But not after we
> have new migrations for Icehouse.)
> 2. Compact all migrations into 2xx_havana.py (for SQLAlchemy-migrate)
As Boris mentioned: "We are able only to convert only grizzly migrations
not havana (because our customers should be able to switch from grizzly
so in the beginning of Icehouse we'll have:
So my question is: when we should start using alembic for new migrations?
We won't be able to deprecate sqla-migrate and switch to alembic in one
step since we'll have some migrations kept for backwards compatibility.
I think that to accomplish your plan we should start using alembic for
new migrations starting from Icehouse.
If you are ok with it I can change ceilometer migrations implementation
to work accordingly and we'll be able to use it as a reference in other
> 3. Test that it's perfect via above test plan plus whatever
> enhancements we think of.
> 4. Manually convert 2xx_havana.py (for SQLAlchemy-migrate) into
> Alembic, and verify that it's still perfect.
> 5. Deprecate the SQLAlchemy-migrate version and announce that new
> migrations should be in Alembic.
> #4 is hard work but not impossible. I have some old code that does
> 90% of the work, so we only have to do the other 90%.
More information about the OpenStack-dev