Open Stack

On 09/18/2012 09:57 PM, Brian Waldon wrote:
> It has become painfully obvious over the past few weeks that we aren't
> testing our database migrations nearly well enough. Dean Troyer and
> myself have found several show-stopping bugs [1] [2] [3] [4] [5] [6] by
> simply applying the migrations to a running Essex system. I'd like to
> propose two things to prevent this from cropping up in the future:
>
> 1) Migrations must come with a test that exercises the database
> modifications with pre-loaded fixture test data. For examples of these
> tests see [7] and [8].
> 2) Run migration tests on a functional mysql backend in Jenkins.
>
> I am concerned that there are hidden bugs in the existing migrations; to
> address this I'd like to propose a third action:
>
> 3) Revisit existing migrations and add the appropriate testing where
> necessary
>
> If anybody has any additional suggestions, please throw them out there!
> If this all sounds sane, I'll start pushing to make it happen.

+1.

If you could just carve off bugs or an etherpad for the items that need 
to be attacked, I think we could parallelyze it very quickly. This kind 
of approach has been working really well for the api sample testing.

It would also be really good if it was easy to run this via 
run_tests.sh. I think the previous issues was the impact it had on 
people on Macs. Is there any better story there?

	-Sean

-- 
Sean Dague
IBM Linux Technology Center
email: sdague at linux.vnet.ibm.com
alt-email: sldague at us.ibm.com