<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>I know alembic is designed to be global, but could we extend it to track multiple histories for a given database. In other words, various branches for different namespaces on a single database. Would this feature ameliorate the issues?</div>
<div><br>
</div>
<div>Amir</div>
<div><br>
<div>
<div>On Apr 15, 2014, at 8:24 AM, Kyle Mestery <<a href="mailto:mestery@noironetworks.com">mestery@noironetworks.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
On Tue, Apr 15, 2014 at 7:03 AM, Salvatore Orlando <<a href="mailto:sorlando@nicira.com">sorlando@nicira.com</a>> wrote:<br>
<blockquote type="cite">Thanks Anna.<br>
<br>
I've been following the issue so far, but I am happy to hand it over to you.<br>
I think the problem assessment is complete, but if you have more questions<br>
ping me on IRC.<br>
<br>
Regarding the solution, I think we already have a fairly wide consensus on<br>
the approach.<br>
There are however a few details to discuss:<br>
- Conflicting schemas. For instance two migrations for two distinct plugins<br>
might create tables with the same name but different columns.<br>
We first need to look at existing migrations to verify where this<br>
condition occurs, and then study a solution case by case.<br>
- Logic for "corrective" migrations. For instance a corrective migration for<br>
569e98a8132b_metering is needed. However, such corrective migration should<br>
have logic for understanding whether the original migration has been<br>
executed or not.<br>
- Corrective actions for corrupted schemas. This would be the case, for<br>
instance, of somebody which enables metering while the database is at a<br>
migration rev higher than the one when metering was introduced.<br>
<br>
I reckon it might be the case of putting together a specification and push<br>
it to the newly created neutron-specs repo, assuming that we feel confident<br>
enough to start using this new process (Kyle and Mark might chime in on this<br>
point). Also, I would like to see this work completed by Juno-1, which I<br>
reckon is a reasonable target.<br>
<br>
</blockquote>
I'm working to get this new specification approval process ready,<br>
hopefully by later<br>
today. Once this is done, I agree with Salvatore, pushing a gerrit<br>
review with the<br>
specification for this work will be the right approach.<br>
<br>
<blockquote type="cite">Of course I'm available for discussing design, implementation, reviewing and<br>
writing code.<br>
<br>
</blockquote>
Thanks to Anna and Salvatore for taking this up!<br>
<br>
Kyle<br>
<br>
<blockquote type="cite">Salvatore<br>
<br>
<br>
<br>
On 15 April 2014 12:44, Anna Kamyshnikova <<a href="mailto:akamyshnikova@mirantis.com">akamyshnikova@mirantis.com</a>><br>
wrote:<br>
<blockquote type="cite"><br>
Hello everyone!<br>
<br>
I would like to try to solve this problem. I registered blueprint on this<br>
topic<br>
<a href="https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations">https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations</a><br>
and I'm going to experiment with options to solve this. I'm welcome any<br>
suggestions and ready to talk about it via IRC (akamyshnikova).<br>
<br>
Regards<br>
Ann<br>
<br>
<br>
On Mon, Apr 14, 2014 at 9:39 PM, Sean Dague <sean@dague.net> wrote:<br>
<blockquote type="cite"><br>
On 04/14/2014 12:46 PM, Eugene Nikanorov wrote:<br>
<blockquote type="cite">Hi Salvatore,<br>
<br>
The described problem could be even worse if vendor drivers are<br>
considered.<br>
Doesn't #1 require that all DB tables are named differently? Otherwise<br>
it seems that user can't be sure in DB schema even if all tables are<br>
present.<br>
<br>
I think the big part of the problem is that we need to support both<br>
online and offline migrations. Without the latter things could be a<br>
little bit simpler.<br>
<br>
Also it seems to me that problem statement should be changed to the<br>
following:<br>
One need to migrate from (Config1, MigrationID1) to (Config2,<br>
MigrationID2), and currently our code only accounts for MigrationIDs.<br>
We may consider amending DB with configuration metadata, at least that<br>
will allow to run migration code with full knowledge of what happened<br>
before (if online mode is considered).<br>
In offline mode that will require providing old and current<br>
configurations.<br>
<br>
That was just thinking aloud, no concrete proposals yet.<br>
</blockquote>
<br>
The root issue really is Migrations *must* be global, and config<br>
invariant. That's the design point in both sqlalchemy-migrate and<br>
alembic. The fact that there is one global migrations table per<br>
database, with a single value in it, is indicative of this fact.<br>
<br>
I think that design point got lost somewhere along the way, and folks<br>
assumed migrations were just a way to change schemas. They are much more<br>
constrained than that.<br>
<br>
It does also sound like the data model is going to need some serious<br>
reconsidering given what's allowed to be changed at the plugin or vendor<br>
driver model. Contrast this with Nova, were virt drivers don't get to<br>
define persistant data that's unique to them (only generic data that<br>
they fit into the grander nova model).<br>
<br>
The one time we had a driver which needed persistent data (baremetal) it<br>
managed it's own database entirely.<br>
<br>
-Sean<br>
<br>
--<br>
Sean Dague<br>
Samsung Research America<br>
sean@dague.net / sean.dague@samsung.com<br>
http://dague.net<br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
OpenStack-dev@lists.openstack.org<br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br>
<br>
</blockquote>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
OpenStack-dev@lists.openstack.org<br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br>
<br>
</blockquote>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br>
<br>
</blockquote>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></div>
</blockquote>
</div>
<br>
</div>
</body>
</html>