Open Stack

Thanks Anna.

I've been following the issue so far, but I am happy to hand it over to you.
I think the problem assessment is complete, but if you have more questions
ping me on IRC.

Regarding the solution, I think we already have a fairly wide consensus on
the approach.
There are however a few details to discuss:
- Conflicting schemas. For instance two migrations for two distinct plugins
might create tables with the same name but different columns.
  We first need to look at existing migrations to verify where this
condition occurs, and then study a solution case by case.
- Logic for "corrective" migrations. For instance a corrective migration
for 569e98a8132b_metering is needed. However, such corrective migration
should have logic for understanding whether the original migration has been
executed or not.
- Corrective actions for corrupted schemas. This would be the case, for
instance, of somebody which enables metering while the database is at a
migration rev higher than the one when metering was introduced.

I reckon it might be the case of putting together a specification and push
it to the newly created neutron-specs repo, assuming that we feel confident
enough to start using this new process (Kyle and Mark might chime in on
this point). Also, I would like to see this work completed by Juno-1, which
I reckon is a reasonable target.

Of course I'm available for discussing design, implementation, reviewing
and writing code.

Salvatore



On 15 April 2014 12:44, Anna Kamyshnikova <akamyshnikova at mirantis.com>wrote:

> Hello everyone!
>
> I would like to try to solve this problem. I registered blueprint on this
> topic
> https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrationsand I'm going to experiment with options to solve this. I'm welcome any
> suggestions and ready to talk about it via IRC (akamyshnikova).
>
> Regards
> Ann
>
>
> On Mon, Apr 14, 2014 at 9:39 PM, Sean Dague <sean at dague.net> wrote:
>
>> On 04/14/2014 12:46 PM, Eugene Nikanorov wrote:
>> > Hi Salvatore,
>> >
>> > The described problem could be even worse if vendor drivers are
>> considered.
>> > Doesn't #1 require that all DB tables are named differently? Otherwise
>> > it seems that user can't be sure in DB schema even if all tables are
>> > present.
>> >
>> > I think the big part of the problem is that we need to support both
>> > online and offline migrations. Without the latter things could be a
>> > little bit simpler.
>> >
>> > Also it seems to me that problem statement should be changed to the
>> > following:
>> > One need to migrate from (Config1, MigrationID1) to (Config2,
>> > MigrationID2), and currently our code only accounts for MigrationIDs.
>> > We may consider amending DB with configuration metadata, at least that
>> > will allow to run migration code with full knowledge of what happened
>> > before (if online mode is considered).
>> > In offline mode that will require providing old and current
>> configurations.
>> >
>> > That was just thinking aloud, no concrete proposals yet.
>>
>> The root issue really is Migrations *must* be global, and config
>> invariant. That's the design point in both sqlalchemy-migrate and
>> alembic. The fact that there is one global migrations table per
>> database, with a single value in it, is indicative of this fact.
>>
>> I think that design point got lost somewhere along the way, and folks
>> assumed migrations were just a way to change schemas. They are much more
>> constrained than that.
>>
>> It does also sound like the data model is going to need some serious
>> reconsidering given what's allowed to be changed at the plugin or vendor
>> driver model. Contrast this with Nova, were virt drivers don't get to
>> define persistant data that's unique to them (only generic data that
>> they fit into the grander nova model).
>>
>> The one time we had a driver which needed persistent data (baremetal) it
>> managed it's own database entirely.
>>
>>         -Sean
>>
>> --
>> Sean Dague
>> Samsung Research America
>> sean at dague.net / sean.dague at samsung.com
>> http://dague.net
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140415/d7f7df14/attachment.html>