Open Stack

Howdy,

At the moment database migrations are required to implement a downgrade
method to ensure updates can be rolled back. Currently downgrades are only
being tested by Jenkins/tox against sqlite databases. MySQL and Postgresql
are not tested and often have special edge cases. I have been working on
fixing broken downgrades to work correctly for these engines[0].

However, separate to that, there are cases where a migration may upgrade 
in a
way that is not backwards compatible. For example, a column or even whole
table may be dropped. The question of what to do in these circumstances is
unclear with many migrations taking different approaches.

For example, migration 209 creates a dump_table and populates it with data
that will be dropped in the upgrade. This then sits there until the 
migration
is downgraded at which point the data is copied back into place and the
dump_table is removed. A similar approach (in [1]) is also being reviewed
where lost data is copied into backup_table_migration_214 and restored on
downgrade. Then we have the example [2] where no lost data is preserved.

These all pose a few different questions. Namely:

1) Do we want to require migrations to downgrade data, or just schema?
2) If we do downgrade data then how should it look?

There is perhaps a secondary discussion around the necessity of downgrades
given most sysadmins would snapshot/backup a restore point before upgrading
themselves but I want to suspend that for the purpose of this discussion.

I think with #2 we should at least be listing the migration number in the
table name for reference sake. This will also give administrators an idea of
when they can safely drop backup tables (as a side note, perhaps this should
be documented).

We also have examples where data can be reconstructed without a backup 
table,
usually when the migration moves data into a new table (for example [3]). So
perhaps another option is we only require downgrading for data that can be
reconstructed without backups.

Cheers,
Josh

[0] https://review.openstack.org/#/c/40137/
[1] https://review.openstack.org/#/c/39685/
[2] https://review.openstack.org/#/c/42736/
[3] Migration 186

--
Rackspace Australia