Open Stack

I know alembic is designed to be global, but could we extend it to track multiple histories for a given database. In other words, various branches for different namespaces on a single database. Would this feature ameliorate the issues?

Amir

On Apr 15, 2014, at 8:24 AM, Kyle Mestery <mestery at noironetworks.com<mailto:mestery at noironetworks.com>> wrote:

On Tue, Apr 15, 2014 at 7:03 AM, Salvatore Orlando <sorlando at nicira.com<mailto:sorlando at nicira.com>> wrote:
Thanks Anna.

I've been following the issue so far, but I am happy to hand it over to you.
I think the problem assessment is complete, but if you have more questions
ping me on IRC.

Regarding the solution, I think we already have a fairly wide consensus on
the approach.
There are however a few details to discuss:
- Conflicting schemas. For instance two migrations for two distinct plugins
might create tables with the same name but different columns.
 We first need to look at existing migrations to verify where this
condition occurs, and then study a solution case by case.
- Logic for "corrective" migrations. For instance a corrective migration for
569e98a8132b_metering is needed. However, such corrective migration should
have logic for understanding whether the original migration has been
executed or not.
- Corrective actions for corrupted schemas. This would be the case, for
instance, of somebody which enables metering while the database is at a
migration rev higher than the one when metering was introduced.

I reckon it might be the case of putting together a specification and push
it to the newly created neutron-specs repo, assuming that we feel confident
enough to start using this new process (Kyle and Mark might chime in on this
point). Also, I would like to see this work completed by Juno-1, which I
reckon is a reasonable target.

I'm working to get this new specification approval process ready,
hopefully by later
today. Once this is done, I agree with Salvatore, pushing a gerrit
review with the
specification for this work will be the right approach.

Of course I'm available for discussing design, implementation, reviewing and
writing code.

Thanks to Anna and Salvatore for taking this up!

Kyle

Salvatore



On 15 April 2014 12:44, Anna Kamyshnikova <akamyshnikova at mirantis.com<mailto:akamyshnikova at mirantis.com>>
wrote:

Hello everyone!

I would like to try to solve this problem. I registered blueprint on this
topic
https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations
and I'm going to experiment with options to solve this. I'm welcome any
suggestions and ready to talk about it via IRC (akamyshnikova).

Regards
Ann


On Mon, Apr 14, 2014 at 9:39 PM, Sean Dague <sean at dague.net> wrote:

On 04/14/2014 12:46 PM, Eugene Nikanorov wrote:
Hi Salvatore,

The described problem could be even worse if vendor drivers are
considered.
Doesn't #1 require that all DB tables are named differently? Otherwise
it seems that user can't be sure in DB schema even if all tables are
present.

I think the big part of the problem is that we need to support both
online and offline migrations. Without the latter things could be a
little bit simpler.

Also it seems to me that problem statement should be changed to the
following:
One need to migrate from (Config1, MigrationID1) to (Config2,
MigrationID2), and currently our code only accounts for MigrationIDs.
We may consider amending DB with configuration metadata, at least that
will allow to run migration code with full knowledge of what happened
before (if online mode is considered).
In offline mode that will require providing old and current
configurations.

That was just thinking aloud, no concrete proposals yet.

The root issue really is Migrations *must* be global, and config
invariant. That's the design point in both sqlalchemy-migrate and
alembic. The fact that there is one global migrations table per
database, with a single value in it, is indicative of this fact.

I think that design point got lost somewhere along the way, and folks
assumed migrations were just a way to change schemas. They are much more
constrained than that.

It does also sound like the data model is going to need some serious
reconsidering given what's allowed to be changed at the plugin or vendor
driver model. Contrast this with Nova, were virt drivers don't get to
define persistant data that's unique to them (only generic data that
they fit into the grander nova model).

The one time we had a driver which needed persistent data (baremetal) it
managed it's own database entirely.

       -Sean

--
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140415/773cd9cd/attachment.html>