Open Stack

On 06/14/2013 04:13 PM, Matthew Treinish wrote:
> On Fri, Jun 14, 2013 at 09:12:11PM +0200, Giulio Fidente wrote:
>> On 06/14/2013 10:33 AM, Attila Fazekas wrote:
>>> - Gate time is continuously increasing and it will
>>>    if we do not act soon.
>> It's an interesting topic but I need a little wrap on on the
>> situation. Still, it'll hopefully be useful to other people also
>> willing to contribute.
>>
>> What are the blockers currently preventing us from using testr and
>> "drop" nose?
> So testr will technically run today, (to test it use 'testr run' or in parallel
> 'testr run --parallel') but you'll find that it is slower than just running
> serially with nose. It also won't pass all the tests but that's a different
> problem to solve after this one. The problem is with setUpClass(), in tempest
> for a lot of tests we do initial setup for some shared resources at the class
> level because they take a lot of time. So then the resources are just shared for
> all the test methods. Testrepository just takes the list of tests that it got
> from subunit partitions them up and then sends that to the testrunner. So for
> each test setupClass gets run independently which significantly increases the
> run time.
>
> I have some ideas to get around this, see the thread on testing fixtures in
> tempest: http://lists.openstack.org/pipermail/openstack-qa/2013-June/000517.html
>
> I think that at this point the quickest path to success is going to be changing
> how testrepository partitions tests. I've started working on doing this by adding
> a new flag to partition on testCases instead of individual tests. This way we
> should only be running setupClass once and hopefully give us the performance we
> were expecting.
>
>
> -Matt Treinish
>
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
I think this might be the right thing anyway. We will continue to want 
to have multiple tests sharing a class setup. Given the huge number of 
test classes I don't think we lose much by setting the level of 
parallelism to be a class. Ideally a class could say whether it's tests 
should be executed in parallel or not.

  -David