Open Stack

Just my personal thoughts, but there's something else to take into account when parallelizing test suites. From my understanding, testr does class or module level parallelization. In a perfect scenario, this means that your test run will take as long as the longest module/class takes to run. The problem with many of the Tempest tests is that they are not optimized for this scenario. 

One good example is test_server_actions under the compute tests. Optimization by reusing a single server for this class helps, but does not resolve the problem that each scenario must be run in serial. To achieve an optimal run time, each scenario should be run in parallel. What I've done in my personal testing is to break each of these down into individual test classes. While this does require additional server resources, the speedup is much higher. I'll put together some benchmarks in the morning.

Daryl

________________________________________
From: Christopher Yeoh [cyeoh at au1.ibm.com]
Sent: Sunday, June 16, 2013 2:12 AM
To: openstack-qa at lists.openstack.org
Subject: Re: [openstack-qa] Priolity brainstorming my initial toughts

On Fri, 14 Jun 2013 16:13:24 -0400
Matthew Treinish <mtreinish at kortar.org> wrote:

> On Fri, Jun 14, 2013 at 09:12:11PM +0200, Giulio Fidente wrote:
> > On 06/14/2013 10:33 AM, Attila Fazekas wrote:
> > >- Gate time is continuously increasing and it will
> > >   if we do not act soon.
> >
> > It's an interesting topic but I need a little wrap on on the
> > situation. Still, it'll hopefully be useful to other people also
> > willing to contribute.
> >
> > What are the blockers currently preventing us from using testr and
> > "drop" nose?
>
> So testr will technically run today, (to test it use 'testr run' or
> in parallel 'testr run --parallel') but you'll find that it is slower
> than just running serially with nose.

What sort of machine are you finding that running in parallel runs
slower than run serially with nose? A few months ago I was seeing a 2x
speed up on 4 core machine when running with testr in parallel compared
to running with nose serially.

Chris
--
cyeoh at au1.ibm.com


_______________________________________________
openstack-qa mailing list
openstack-qa at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa