[openstack-dev] [DB][Migrations] Switching to using of Alembic
dripton at redhat.com
Tue Jul 16 18:16:24 UTC 2013
On 07/16/2013 01:58 PM, Boris Pavlovic wrote:
> There is no "magic" and we have only 2 ways to end up with this problems
> and bugs that could be caused by "manually" migrations merging and tons
> of bugs in sqlalchemy-migrate.
> 1) step by step (openstack method)
> There are special tests "test_migrations" that runs migrations on
> real data against all backends. So we should:
> a) improve this tests to checks all behaviors // there is a lot of
> hidden bugs
> b) replace migration (only one small migration) to alembic
> c) check that in all backends we made the same changes in schema
> d) Merge all old migrations in one using alembic (automatically).
> So it could be done in safe way.
> 2.a) huge 2 steps
> 1. Merge all migrations in one huge manually (drop all tests in test
> e.g. In Nova was patch https://review.openstack.org/#/c/35748/
> I don't believe that there are no mistakes in this migration, and
> nobody is able to check it. // because of tons of hidden bugs in old
> migrations and sqla-migrate.
> 2. Replace this migration in Alembic
> I don't believe that there will be way to check that there is no
> 2.b) suicide mode (1 big step)
> Merge and switch in one step=)
We have compacted migrations before, and there's a test document for how
to verify that the big migration has exactly the same output as the
series of small migrations. See
https://wiki.openstack.org/wiki/Database_migration_testing Dan Prince
is the expert on this.
I think the right process is:
1. Wait until the very beginning of Icehouse cycle. (But not after we
have new migrations for Icehouse.)
2. Compact all migrations into 2xx_havana.py (for SQLAlchemy-migrate)
3. Test that it's perfect via above test plan plus whatever enhancements
we think of.
4. Manually convert 2xx_havana.py (for SQLAlchemy-migrate) into Alembic,
and verify that it's still perfect.
5. Deprecate the SQLAlchemy-migrate version and announce that new
migrations should be in Alembic.
#4 is hard work but not impossible. I have some old code that does 90%
of the work, so we only have to do the other 90%.
David Ripton Red Hat dripton at redhat.com
More information about the OpenStack-dev