Open Stack

On Tue, Apr 15, 2014 at 7:03 AM, Salvatore Orlando <sorlando at nicira.com> wrote:
> Thanks Anna.
>
> I've been following the issue so far, but I am happy to hand it over to you.
> I think the problem assessment is complete, but if you have more questions
> ping me on IRC.
>
> Regarding the solution, I think we already have a fairly wide consensus on
> the approach.
> There are however a few details to discuss:
> - Conflicting schemas. For instance two migrations for two distinct plugins
> might create tables with the same name but different columns.
>   We first need to look at existing migrations to verify where this
> condition occurs, and then study a solution case by case.
> - Logic for "corrective" migrations. For instance a corrective migration for
> 569e98a8132b_metering is needed. However, such corrective migration should
> have logic for understanding whether the original migration has been
> executed or not.
> - Corrective actions for corrupted schemas. This would be the case, for
> instance, of somebody which enables metering while the database is at a
> migration rev higher than the one when metering was introduced.
>
> I reckon it might be the case of putting together a specification and push
> it to the newly created neutron-specs repo, assuming that we feel confident
> enough to start using this new process (Kyle and Mark might chime in on this
> point). Also, I would like to see this work completed by Juno-1, which I
> reckon is a reasonable target.
>
I'm working to get this new specification approval process ready,
hopefully by later
today. Once this is done, I agree with Salvatore, pushing a gerrit
review with the
specification for this work will be the right approach.

> Of course I'm available for discussing design, implementation, reviewing and
> writing code.
>
Thanks to Anna and Salvatore for taking this up!

Kyle

> Salvatore
>
>
>
> On 15 April 2014 12:44, Anna Kamyshnikova <akamyshnikova at mirantis.com>
> wrote:
>>
>> Hello everyone!
>>
>> I would like to try to solve this problem. I registered blueprint on this
>> topic
>> https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations
>> and I'm going to experiment with options to solve this. I'm welcome any
>> suggestions and ready to talk about it via IRC (akamyshnikova).
>>
>> Regards
>> Ann
>>
>>
>> On Mon, Apr 14, 2014 at 9:39 PM, Sean Dague <sean at dague.net> wrote:
>>>
>>> On 04/14/2014 12:46 PM, Eugene Nikanorov wrote:
>>> > Hi Salvatore,
>>> >
>>> > The described problem could be even worse if vendor drivers are
>>> > considered.
>>> > Doesn't #1 require that all DB tables are named differently? Otherwise
>>> > it seems that user can't be sure in DB schema even if all tables are
>>> > present.
>>> >
>>> > I think the big part of the problem is that we need to support both
>>> > online and offline migrations. Without the latter things could be a
>>> > little bit simpler.
>>> >
>>> > Also it seems to me that problem statement should be changed to the
>>> > following:
>>> > One need to migrate from (Config1, MigrationID1) to (Config2,
>>> > MigrationID2), and currently our code only accounts for MigrationIDs.
>>> > We may consider amending DB with configuration metadata, at least that
>>> > will allow to run migration code with full knowledge of what happened
>>> > before (if online mode is considered).
>>> > In offline mode that will require providing old and current
>>> > configurations.
>>> >
>>> > That was just thinking aloud, no concrete proposals yet.
>>>
>>> The root issue really is Migrations *must* be global, and config
>>> invariant. That's the design point in both sqlalchemy-migrate and
>>> alembic. The fact that there is one global migrations table per
>>> database, with a single value in it, is indicative of this fact.
>>>
>>> I think that design point got lost somewhere along the way, and folks
>>> assumed migrations were just a way to change schemas. They are much more
>>> constrained than that.
>>>
>>> It does also sound like the data model is going to need some serious
>>> reconsidering given what's allowed to be changed at the plugin or vendor
>>> driver model. Contrast this with Nova, were virt drivers don't get to
>>> define persistant data that's unique to them (only generic data that
>>> they fit into the grander nova model).
>>>
>>> The one time we had a driver which needed persistent data (baremetal) it
>>> managed it's own database entirely.
>>>
>>>         -Sean
>>>
>>> --
>>> Sean Dague
>>> Samsung Research America
>>> sean at dague.net / sean.dague at samsung.com
>>> http://dague.net
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>