Open Stack

On Fri, Jun 14, 2013 at 09:12:11PM +0200, Giulio Fidente wrote:
> On 06/14/2013 10:33 AM, Attila Fazekas wrote:
> >- Gate time is continuously increasing and it will
> >   if we do not act soon.
> 
> It's an interesting topic but I need a little wrap on on the
> situation. Still, it'll hopefully be useful to other people also
> willing to contribute.
> 
> What are the blockers currently preventing us from using testr and
> "drop" nose?

So testr will technically run today, (to test it use 'testr run' or in parallel
'testr run --parallel') but you'll find that it is slower than just running
serially with nose. It also won't pass all the tests but that's a different
problem to solve after this one. The problem is with setUpClass(), in tempest
for a lot of tests we do initial setup for some shared resources at the class
level because they take a lot of time. So then the resources are just shared for
all the test methods. Testrepository just takes the list of tests that it got
from subunit partitions them up and then sends that to the testrunner. So for
each test setupClass gets run independently which significantly increases the
run time.

I have some ideas to get around this, see the thread on testing fixtures in
tempest: http://lists.openstack.org/pipermail/openstack-qa/2013-June/000517.html

I think that at this point the quickest path to success is going to be changing
how testrepository partitions tests. I've started working on doing this by adding
a new flag to partition on testCases instead of individual tests. This way we
should only be running setupClass once and hopefully give us the performance we
were expecting.


-Matt Treinish