Open Stack

I kind of high jacked another thread with my testr problems, but I want to
reiterate it directly on this one as they are my pain points I had from the
transition.

testr does not appear to support --exclusion[1] or --stop[2].

I have a work around for --exclusion, by:

testr list-tests | egrep -v regex-exclude-list > unit-tests.txt
testr --load-list unit-tests.txt

I do not have a work around for --stop.

[1]https://bugs.launchpad.net/testrepository/+bug/1208610
[2]https://bugs.launchpad.net/testrepository/+bug/1211926


On Tue, Aug 13, 2013 at 1:25 PM, Matthew Treinish <mtreinish at kortar.org>wrote:

>
> Hi everyone,
>
> So for the past month or so I've been working on getting tempest to work
> stably
> with testr in parallel. As part of this you may have noticed the testr-full
> jobs that get run on the zuul check queue. I was using that job to debug
> some
> of the more obvious race conditions and stability issues with running
> tempest
> in parallel. After a bunch of fixes to tempest and finding some real bugs
> in
> some of the projects things seem to have smoothed out.
>
> So I pushed the testr-full run to the gate queue earlier today. I'll be
> keeping
> track of the success rate of this job vs the serial job and use this as the
> determining factor before we push this live to be the default for all
> tempest
> runs. So assuming that the success rate matches up well enough with serial
> job
> on the gate queue then I will push out the change that will migrate all the
> voting jobs to run in parallel hopefully either Friday afternoon or early
> next
> week. Also, if anyone has any input on what threshold they feel is good
> enough
> for this I'd welcome any input on that. For example, do we want to ensure
> a >= 1:1 match for job success? Or would something like 90% as stable as
> the
> serial job be good enough considering the speed advantage. (The parallel
> runs
> take about half as much time as a full serial run, the parallel job
> normally
> finishes in ~25-30min) Since this affects almost every project I don't
> want to
> define this threshold without input from everyone.
>
> After there is some more data for the gate queue's parallel job I'll have
> some
> pretty graphite graphs that I can share comparing the success trends
> between
> the parallel and serial jobs.
>
> So at this point we're in the home stretch and I'm asking for everyone's
> help
> in getting this merged. So, if everyone who is reviewing and pushing
> commits
> could watch the results from these non-voting jobs and if things fail on
> the
> parallel job but not the serial job please investigate the failure and
> open a
> bug if necessary. If it turns out to be a bug in tempest please link it
> against
> this blueprint:
>
> https://blueprints.launchpad.net/tempest/+spec/speed-up-tempest
>
> so that I'll give it the attention it deserves. I'd hate to get this close
> to
> getting this merged and have a bit of racy code get merged at the last
> second
> and block us for another week or two.
>
> I feel that we need to get this in before the H3 rush starts up as it will
> help
> everyone get through the extra review load faster.
>
> Thanks,
>
> Matt Treinish
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130814/548eb4d9/attachment.html>