I couldn't agree more with this. -Alex -----Original Message----- From: "Brian Waldon" <bcwaldon at gmail.com> Sent: Tuesday, September 18, 2012 9:57pm To: "OpenStack Development Mailing List" <openstack-dev at lists.openstack.org> Subject: [openstack-dev] DB Migration Testing It has become painfully obvious over the past few weeks that we aren't testing our database migrations nearly well enough. Dean Troyer and myself have found several show-stopping bugs [1] [2] [3] [4] [5] [6] by simply applying the migrations to a running Essex system. I'd like to propose two things to prevent this from cropping up in the future: 1) Migrations must come with a test that exercises the database modifications with pre-loaded fixture test data. For examples of these tests see [7] and [8]. 2) Run migration tests on a functional mysql backend in Jenkins. I am concerned that there are hidden bugs in the existing migrations; to address this I'd like to propose a third action: 3) Revisit existing migrations and add the appropriate testing where necessary If anybody has any additional suggestions, please throw them out there! If this all sounds sane, I'll start pushing to make it happen. Waldon [1] [https://bugs.launchpad.net/nova/+bug/1047633] https://bugs.launchpad.net/nova/+bug/1047633 [2] [https://bugs.launchpad.net/nova/+bug/1047665] https://bugs.launchpad.net/nova/+bug/1047665 [3] [https://bugs.launchpad.net/nova/+bug/1052220] https://bugs.launchpad.net/nova/+bug/1052220 [4] [https://bugs.launchpad.net/nova/+bug/1052244] https://bugs.launchpad.net/nova/+bug/1052244 [5] [https://bugs.launchpad.net/nova/+bug/1052252] https://bugs.launchpad.net/nova/+bug/1052252 [6] [https://bugs.launchpad.net/glance/+bug/1043041] https://bugs.launchpad.net/glance/+bug/1043041 [7] [https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L307] https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L307 [8] [https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L327] https://github.com/openstack/nova/blob/0ffff7c490d54f9b932df4b2dcd62158242dae2a/nova/tests/test_migrations.py#L327 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20120920/ab4d8b35/attachment-0001.html>