<font face="arial" size="2"><p style="margin:0;padding:0;">I couldn't agree more with this.</p>
<p style="margin:0;padding:0;"> </p>
<p style="margin:0;padding:0;">-Alex</p>
<p style="margin:0;padding:0;"> </p>
<p style="margin:0;padding:0;">-----Original Message-----<br />From: "Brian Waldon" <bcwaldon@gmail.com><br />Sent: Tuesday, September 18, 2012 9:57pm<br />To: "OpenStack Development Mailing List" <openstack-dev@lists.openstack.org><br />Subject: [openstack-dev] DB Migration Testing<br /><br /></p>
<div id="SafeStyles1348160943">
<div>It has become painfully obvious over the past few weeks that we aren't testing our database migrations nearly well enough. Dean Troyer and myself have found several show-stopping bugs [1] [2] [3] [4] [5] [6] by simply applying the migrations to a running Essex system. I'd like to propose two things to prevent this from cropping up in the future:</div>
<div>1) Migrations must come with a test that exercises the database modifications with pre-loaded fixture test data. For examples of these tests see [7] and [8].</div>
<div>2) Run migration tests on a functional mysql backend in Jenkins.</div>
<div>I am concerned that there are hidden bugs in the existing migrations; to address this I'd like to propose a third action:</div>
<div>3) Revisit existing migrations and add the appropriate testing where necessary</div>
<div>If anybody has any additional suggestions, please throw them out there! If this all sounds sane, I'll start pushing to make it happen.</div>
<div>Waldon</div>
<div><br />
<div>[1] <a href="https://bugs.launchpad.net/nova/+bug/1047633">https://bugs.launchpad.net/nova/+bug/1047633</a></div>
<div>[2] <a href="https://bugs.launchpad.net/nova/+bug/1047665">https://bugs.launchpad.net/nova/+bug/1047665</a></div>
<div>[3] <a href="https://bugs.launchpad.net/nova/+bug/1052220">https://bugs.launchpad.net/nova/+bug/1052220</a></div>
<div>[4] <a href="https://bugs.launchpad.net/nova/+bug/1052244">https://bugs.launchpad.net/nova/+bug/1052244</a></div>
<div>[5] <a href="https://bugs.launchpad.net/nova/+bug/1052252">https://bugs.launchpad.net/nova/+bug/1052252</a></div>
<div>[6] <a href="https://bugs.launchpad.net/glance/+bug/1043041">https://bugs.launchpad.net/glance/+bug/1043041</a></div>
<div>[7] <a href="https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L307">https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L307</a></div>
</div>
<div>[8] <a href="https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L327">https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L327</a></div>
</div></font>