Open Stack

+1 good idea

I was wondering this myself.

Sent from my really tiny device...

On Aug 14, 2013, at 1:07 PM, "Alexius Ludeman" <lex at lexinator.com<mailto:lex at lexinator.com>> wrote:

I kind of high jacked another thread with my testr problems, but I want to reiterate it directly on this one as they are my pain points I had from the transition.

testr does not appear to support --exclusion[1] or --stop[2].

I have a work around for --exclusion, by:
testr list-tests | egrep -v regex-exclude-list > unit-tests.txt
testr --load-list unit-tests.txt

I do not have a work around for --stop.

[1]https://bugs.launchpad.net/testrepository/+bug/1208610
[2]https://bugs.launchpad.net/testrepository/+bug/1211926


On Tue, Aug 13, 2013 at 1:25 PM, Matthew Treinish <mtreinish at kortar.org<mailto:mtreinish at kortar.org>> wrote:

Hi everyone,

So for the past month or so I've been working on getting tempest to work stably
with testr in parallel. As part of this you may have noticed the testr-full
jobs that get run on the zuul check queue. I was using that job to debug some
of the more obvious race conditions and stability issues with running tempest
in parallel. After a bunch of fixes to tempest and finding some real bugs in
some of the projects things seem to have smoothed out.

So I pushed the testr-full run to the gate queue earlier today. I'll be keeping
track of the success rate of this job vs the serial job and use this as the
determining factor before we push this live to be the default for all tempest
runs. So assuming that the success rate matches up well enough with serial job
on the gate queue then I will push out the change that will migrate all the
voting jobs to run in parallel hopefully either Friday afternoon or early next
week. Also, if anyone has any input on what threshold they feel is good enough
for this I'd welcome any input on that. For example, do we want to ensure
a >= 1:1 match for job success? Or would something like 90% as stable as the
serial job be good enough considering the speed advantage. (The parallel runs
take about half as much time as a full serial run, the parallel job normally
finishes in ~25-30min) Since this affects almost every project I don't want to
define this threshold without input from everyone.

After there is some more data for the gate queue's parallel job I'll have some
pretty graphite graphs that I can share comparing the success trends between
the parallel and serial jobs.

So at this point we're in the home stretch and I'm asking for everyone's help
in getting this merged. So, if everyone who is reviewing and pushing commits
could watch the results from these non-voting jobs and if things fail on the
parallel job but not the serial job please investigate the failure and open a
bug if necessary. If it turns out to be a bug in tempest please link it against
this blueprint:

https://blueprints.launchpad.net/tempest/+spec/speed-up-tempest

so that I'll give it the attention it deserves. I'd hate to get this close to
getting this merged and have a bit of racy code get merged at the last second
and block us for another week or two.

I feel that we need to get this in before the H3 rush starts up as it will help
everyone get through the extra review load faster.

Thanks,

Matt Treinish

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130815/479eb46e/attachment.html>