Open Stack

On 06/16/2013 10:45 AM, Daryl Walleck wrote:
> One good example is test_server_actions under the compute tests. Optimization by reusing a single server for this class helps, but does not resolve the problem that each scenario must be run in serial. To achieve an optimal run time, each scenario should be run in parallel. What I've done in my personal testing is to break each of these down into individual test classes. While this does require additional server resources, the speedup is much higher. I'll put together some benchmarks in the morning.

this seems to me to go somehow in contrast with the problem Matthew is 
trying to solve

in the ideal scenario you mention to get the best results you'll want 
the setUpClass steps (which I'm currently a fan of and writing a few) 
moved into the test itself so there won't be any need for it at all

I understand the logic behind it but to me it seems that it could only 
produce some benefits on systems capable of very large parallelization 
on both the client side AND the server side (which I don't see as a real 
near-term case for us)

Having to serialize only the tests which share a number of setUpClass 
steps instead seems to me a reasonable compromise

Did you manage to get any performance measurement around the two options?
-- 
Giulio Fidente
GPG KEY: 08D733BA | IRC: giulivo