Open Stack

On Sep 20, 2012, at 1:04 PM, Sean Dague wrote:

> On 09/18/2012 09:57 PM, Brian Waldon wrote:
>> It has become painfully obvious over the past few weeks that we aren't
>> testing our database migrations nearly well enough. Dean Troyer and
>> myself have found several show-stopping bugs [1] [2] [3] [4] [5] [6] by
>> simply applying the migrations to a running Essex system. I'd like to
>> propose two things to prevent this from cropping up in the future:
>> 
>> 1) Migrations must come with a test that exercises the database
>> modifications with pre-loaded fixture test data. For examples of these
>> tests see [7] and [8].
>> 2) Run migration tests on a functional mysql backend in Jenkins.
>> 
>> I am concerned that there are hidden bugs in the existing migrations; to
>> address this I'd like to propose a third action:
>> 
>> 3) Revisit existing migrations and add the appropriate testing where
>> necessary
>> 
>> If anybody has any additional suggestions, please throw them out there!
>> If this all sounds sane, I'll start pushing to make it happen.
> 
> +1.
> 
> If you could just carve off bugs or an etherpad for the items that need to be attacked, I think we could parallelyze it very quickly. This kind of approach has been working really well for the api sample testing.

It might be too late to really care about the Folsom migrations since I'm assuming we will compress the migrations as we did after the Essex release. It might be more reasonable to set strict guidelines for new migrations added in Grizzly.

> It would also be really good if it was easy to run this via run_tests.sh. I think the previous issues was the impact it had on people on Macs. Is there any better story there?

So I was testing on MySQL on Ubuntu, but we can get a lot of value in running testing on SQLite (which works find on Mac OS X).