<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jul 17, 2013 at 6:38 AM, Alexei Kornienko <span dir="ltr"><<a href="mailto:alexei.kornienko@gmail.com" target="_blank">alexei.kornienko@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
Please see my comments inline.<br>
<br>
Best regards,<br>
Alexei Kornienko<div><div class="h5"><br>
<br>
On 07/16/2013 09:16 PM, David Ripton wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 07/16/2013 01:58 PM, Boris Pavlovic wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
There is no "magic" and we have only 2 ways to end up with this problems<br>
and bugs that could be caused by "manually" migrations merging and tons<br>
of bugs in sqlalchemy-migrate.<br>
<br>
1) step by step (openstack method)<br>
There are special tests "test_migrations" that runs migrations on<br>
real data against all backends. So we should:<br>
<br>
a) improve this tests to checks all behaviors // there is a lot of<br>
hidden bugs<br>
b) replace migration (only one small migration) to alembic<br>
c) check that in all backends we made the same changes in schema<br>
d) Merge all old migrations in one using alembic (automatically).<br>
So it could be done in safe way.<br>
<br>
2.a) huge 2 steps<br>
1. Merge all migrations in one huge manually (drop all tests in test<br>
migrations)<br>
e.g. In Nova was patch <a href="https://review.openstack.org/#/c/35748/" target="_blank">https://review.openstack.org/#<u></u>/c/35748/</a><br>
I don't believe that there are no mistakes in this migration, and<br>
nobody is able to check it. // because of tons of hidden bugs in old<br>
migrations and sqla-migrate.<br>
2. Replace this migration in Alembic<br>
I don't believe that there will be way to check that there is no<br>
bugs<br>
<br>
2.b) suicide mode (1 big step)<br>
Merge and switch in one step=)<br>
</blockquote>
<br>
We have compacted migrations before, and there's a test document for how to verify that the big migration has exactly the same output as the series of small migrations. See <a href="https://wiki.openstack.org/wiki/Database_migration_testing" target="_blank">https://wiki.openstack.org/<u></u>wiki/Database_migration_<u></u>testing</a> Dan Prince is the expert on this.<br>
<br>
I think the right process is:<br>
<br>
1. Wait until the very beginning of Icehouse cycle. (But not after we have new migrations for Icehouse.)<br>
<br>
2. Compact all migrations into 2xx_havana.py (for SQLAlchemy-migrate)<br>
</blockquote></div></div>
As Boris mentioned: "We are able only to convert only grizzly migrations not havana (because our customers should be able to switch from grizzly to havana)"<br>
so in the beginning of Icehouse we'll have:<br>
2xx_grizzly.py (sqla-migrate)<br>
2xx_havana.py (sqla-migrate)<br>
...<br>
2xx_havana.py (sqla-migrate)<br>
___________________<br>
Icehouse migrations...<br>
<br>
So my question is: when we should start using alembic for new migrations?<br>
We won't be able to deprecate sqla-migrate and switch to alembic in one step since we'll have some migrations kept for backwards compatibility.<br>
I think that to accomplish your plan we should start using alembic for new migrations starting from Icehouse.<br>
If you are ok with it I can change ceilometer migrations implementation to work accordingly and we'll be able to use it as a reference in other projects.</blockquote><div><br></div><div>As far as I'm concerned for ceilometer, we can start using alembic for migrations as soon as the dbsync command runs both sets of migrations in the right sequence. You have a patch in review for that now, so I don't think we'll need to wait for Icehouse.</div>
<div><br></div><div>Doug</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im HOEnZb"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
3. Test that it's perfect via above test plan plus whatever enhancements we think of.<br>
<br>
4. Manually convert 2xx_havana.py (for SQLAlchemy-migrate) into Alembic, and verify that it's still perfect.<br>
<br>
5. Deprecate the SQLAlchemy-migrate version and announce that new migrations should be in Alembic.<br>
<br>
#4 is hard work but not impossible. I have some old code that does 90% of the work, so we only have to do the other 90%.<br>
<br>
</blockquote>
<br>
<br></div><div class="HOEnZb"><div class="h5">
______________________________<u></u>_________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.<u></u>org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
</div></div></blockquote></div><br></div></div>