Open Stack

I recently submitted a few fixes to the test suite in various components
of openstack. These fixes are being merged in master, but the code
remains broken in the stable/essex branch. Review requests for 
stable/essex either get rejected or stuck in limbo because it seems that 
people don't know what to do about them.

I am aware of the procedure for backporting fixes[1], but I think it
does not deal with this issue correctly.

Fixes to test scripts are for our benefit, the developers'. They don't
affect the users in any way. I don't think test code should be thrown
together with application code when thinking about making changes to
it.

Only making test changes to the master branches reflects a belief that
tests are only used during development. I don't think this is
true. Tests, especially functional tests, are also incredibly useful
during maintenance. e.g. They help us test against different library
versions/distro than the one that's used for development and using
different deployment configurations.

I suspect we're not the only downstream running the various testsuites
against their own packaged versions of different openstack
branches. Backporting these changes not only spares the time of other
projects who might run into these bugs on the stable branches later, it
also gives all of us the benefit of not having to fork the project just
so we can attach our patches. OTOH blocking test backports removes the
incentive that downstream projects have for reporting those bugs and
sending fixes for them upstream.

So can we talk about separating the tests from the application code at
least as far as the backports are concerned? What about having the 
'test/' directory as a git submodule?

Or maybe I don't understand this problem enough. What are the downsides
to backporting test-only fixes? Do they really outweigh the advantages?

Thanks,
Ionuț


[1] http://wiki.openstack.org/StableBranch