On 08/29/2014 06:54 AM, Salvatore Orlando wrote:

If you are running version from a stable branch, changes in DB migrations should generally be forbidden as the policy states since those migrations are not likely to be executed again. Downgrading and then upgrading again is extremely risky and I don't think anybody would ever do that.

However, if one is running stable branch X-2 where X is the current development branch, back porting migration fixes could make sense for upgrading to version X-1 if the migration being fixed is in the path between X-2 and X-1. Therefore I would forbid every fix to migration earlier than X-2 release (there should not be any in theory but neutron has migrations back to folsom). For the path between X-2 and X-1 fixes might be ok.

I think it's safe to backport to X-1. The key bit is that the migration in master and the backported version must be reentrant. They need to inspect the schema and only perform the change if it hasn't already been applied. This is a good best practice to adopt for *all* migrations to make the backport option easier.

However, rather than amending existing migration is always better to add new migrations - even if it's a matter of enabling a given change for a particular plugin (*).

Agreed, in general.

It depends on the bug. If there's an error in the migration that will prevent the original code from running properly, breaking the migration, that obviously needs to be fixed.

As nova does, the best place for doing that is always immediately before release.

Doing what, adding placeholders?

Note that we actually add placeholders at the very *beginning* of a release cycle. The placeholders have to be put in place as the first set of migrations in a release. That way:

1) X-1 has those migration slots unused.

2) X has those slots reserved.

If we did it just *before* release, you can't actually backport into those positions. They've already run as no-op.

With alembic, we do not need to add placeholders, but just adjust pointers just like you would when inserting an element in a dynamic list.

Good point.

(*) we are getting rid of this conditional migration logic for juno anyway

Yay!

On Aug 29, 2014, at 7:23 AM, Alan Pevec apevec@gmail.com wrote:

...

It seems that currently it's hard to backport any database schema fix to Neutron [1] which uses alembic to manage db schema version. Nova has the same issue before and a workaround is to put some placeholder files before each release. So first do we allow db schema fixes to be backport to stable for Neutron ?

DB schema backports was a topic at StableBranch session last design summit [*] and policy did not change: not allowed in general but exceptions could always be discussed on stable-maint list.

...

If we do, then how about put some placeholder files similar to Nova at the end of each release cycle? or we have some better solution for alembic.

AFAIK you can't have placeholders in alembic, there was an action item from design session for Mark to summarize his best practices for db backports.

Alembic doesn’t need “placeholder” files, if we’re referring to the practice with migrate to have empty migration files present so that new migrations can be spliced in. Alembic migrations can be spliced anywhere in the series. The only current limitation, which is on deck to be opened up, is that the migrations ultimately have to be arranged linearly in some way (e.g. if two different environments are the product of two branches, and need to run the same series of migrations, but one needs to skip certain files and the other needs to skip others, only those migrations that are needed on each are applied. But SQLalchemy-migrate certainly has no capability for that either). If this issue needs to be fast-tracked I can move my efforts there.