<div dir="ltr">Thanks Anna.<div><br></div><div>I've been following the issue so far, but I am happy to hand it over to you.</div><div>I think the problem assessment is complete, but if you have more questions ping me on IRC.</div>
<div><br></div><div>Regarding the solution, I think we already have a fairly wide consensus on the approach.</div><div>There are however a few details to discuss:</div><div>- Conflicting schemas. For instance two migrations for two distinct plugins might create tables with the same name but different columns.</div>
<div> We first need to look at existing migrations to verify where this condition occurs, and then study a solution case by case.</div><div>- Logic for "corrective" migrations. For instance a corrective migration for 569e98a8132b_metering is needed. However, such corrective migration should have logic for understanding whether the original migration has been executed or not.</div>
<div>- Corrective actions for corrupted schemas. This would be the case, for instance, of somebody which enables metering while the database is at a migration rev higher than the one when metering was introduced.</div><div>
<br></div><div>I reckon it might be the case of putting together a specification and push it to the newly created neutron-specs repo, assuming that we feel confident enough to start using this new process (Kyle and Mark might chime in on this point). Also, I would like to see this work completed by Juno-1, which I reckon is a reasonable target.</div>
<div><br></div><div>Of course I'm available for discussing design, implementation, reviewing and writing code.</div><div><br></div><div>Salvatore</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On 15 April 2014 12:44, Anna Kamyshnikova <span dir="ltr"><<a href="mailto:akamyshnikova@mirantis.com" target="_blank">akamyshnikova@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div>Hello everyone!</div><div><br></div><div>I would like to try to solve this problem. I registered blueprint on this topic <a href="https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations" target="_blank">https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations</a> and I'm going to experiment with options to solve this. I'm welcome any suggestions and ready to talk about it via IRC (akamyshnikova).</div>
<div><br></div><div>Regards</div><div>Ann</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Mon, Apr 14, 2014 at 9:39 PM, Sean Dague <span dir="ltr"><<a href="mailto:sean@dague.net" target="_blank">sean@dague.net</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div>On 04/14/2014 12:46 PM, Eugene Nikanorov wrote:<br>
> Hi Salvatore,<br>
><br>
> The described problem could be even worse if vendor drivers are considered.<br>
> Doesn't #1 require that all DB tables are named differently? Otherwise<br>
> it seems that user can't be sure in DB schema even if all tables are<br>
> present.<br>
><br>
> I think the big part of the problem is that we need to support both<br>
> online and offline migrations. Without the latter things could be a<br>
> little bit simpler.<br>
><br>
> Also it seems to me that problem statement should be changed to the<br>
> following:<br>
> One need to migrate from (Config1, MigrationID1) to (Config2,<br>
> MigrationID2), and currently our code only accounts for MigrationIDs.<br>
> We may consider amending DB with configuration metadata, at least that<br>
> will allow to run migration code with full knowledge of what happened<br>
> before (if online mode is considered).<br>
> In offline mode that will require providing old and current configurations.<br>
><br>
> That was just thinking aloud, no concrete proposals yet.<br>
<br>
</div>The root issue really is Migrations *must* be global, and config<br>
invariant. That's the design point in both sqlalchemy-migrate and<br>
alembic. The fact that there is one global migrations table per<br>
database, with a single value in it, is indicative of this fact.<br>
<br>
I think that design point got lost somewhere along the way, and folks<br>
assumed migrations were just a way to change schemas. They are much more<br>
constrained than that.<br>
<br>
It does also sound like the data model is going to need some serious<br>
reconsidering given what's allowed to be changed at the plugin or vendor<br>
driver model. Contrast this with Nova, were virt drivers don't get to<br>
define persistant data that's unique to them (only generic data that<br>
they fit into the grander nova model).<br>
<br>
The one time we had a driver which needed persistent data (baremetal) it<br>
managed it's own database entirely.<br>
<span><font color="#888888"><br>
-Sean<br>
<br>
--<br>
Sean Dague<br>
Samsung Research America<br>
<a href="mailto:sean@dague.net" target="_blank">sean@dague.net</a> / <a href="mailto:sean.dague@samsung.com" target="_blank">sean.dague@samsung.com</a><br>
<a href="http://dague.net" target="_blank">http://dague.net</a><br>
<br>
</font></span><br></div></div><div class="">_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></div></blockquote></div><br></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>