Open Stack

Excerpts from Mike Bayer's message of 2015-05-14 11:40:44 -0400:
> 
> On 5/14/15 5:44 AM, John Garbutt wrote:
> > On 14 May 2015 at 07:18, Manickam, Kanagaraj <kanagaraj.manickam at hp.com> wrote:
> >> Hi Nova team,
> >>
> >> This mail is regarding an help required on the migration from
> >> sqlalchemy migration tool  to alembic tool.
> >>
> >> Heat is currently using sqlalchemy-migration tool and In liberty release,
> >> we are investigating on how to bring the alembic into heat. We found that
> >> nova has already tried the same (https://review.openstack.org/#/c/15196/ )
> >> almost 2 years back and in Kilo release, nova is still using sqlalchemy
> >> migration tool.
> >> (https://github.com/openstack/nova/tree/master/nova/db/sqlalchemy/migrate_repo/versions)
> >>
> >> So we are assuming that, in nova, you might have faced blockers to bring in
> >> alembic.
> >> And we would like to seek your  recommendation/suggestions based
> >> on your experience on this.  This will help us to take proper direction
> >> on using alembic in heat. Could you kindly share it.
> > We are currently looking at going this direction (its not merged yet though):
> > http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/online-schema-changes.html
> >
> > If we get that working, we wouldn't need to write or generate any
> > migrations, we end up doing a "diff" between the desired state and the
> > current database state, and split that into expand and contract
> > phases.
> >
> > To support that approach, we are no longer doing any data migrations
> > in our migration scripts, they are happening in a different way,
> > "online":
> > http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/flavor-from-sysmeta-to-blob.html#other-deployer-impact
> >
> > There are more details on how this is all planned to fit together here:
> > http://docs.openstack.org/developer/nova/devref/upgrade.html
> >
> > Anyways, this is why we have paused the alembic vs sqlalchemy debate
> > for the moment.
> 
> The "online schema changes" patch has been abandoned.    I regret that I 
> was not able to review the full nature of this spec in time to note some 
> concerns I have, namely that Alembic does not plan on ever acheiving 
> 100% "automation" of migration generation; such a thing is not possible 
> and would require a vast amount of development resources in any case to 
> constantly keep up with the ever changing features and behaviors of all 
> target databases.   The online migration spec, AFAICT, does not offer 
> any place for manual migrations to be added, and I think this will be a 
> major problem. The decisions made by "autogenerate" I have always stated 
> should always be manually reviewed and corrected, so I'd be very nervous 
> about a system that uses autogenerate on the fly and sends those changes 
> directly to a production database without any review.
> 
> FWIW in Nova I am planning to look into doing a simple one-shot of the 
> Nova migration scripts in nova/db/sqlalchemy/migrate_repo/versions to 
> Alembic.   The only large migration script in this repo is 
> 216_havana.py, which consists of all Table defs that do not need any 
> changes.  There are 39 remaining scripts, and all of these are extremely 
> short and simple, and just a direct manual change to Alembic style is 
> most expedient for these.

At one point we were exploring having both sqlalchemy-migrate and
alembic run, one after the other, so that we only need to create new
migrations with alembic and do not need to change any of the existing
migrations. Was that idea dropped?

Doug