Open Stack

On 06/18/2013 01:17 PM, Jeremy Stanley wrote:
> On 2013-06-18 18:43:58 +0200 (+0200), Martina Kollarova wrote:
> [...]
>> I propose that it should fail and stop all of the other tests once
>> there is a failure in a voting test. For non-voting tests, it should
>> stop only itself, not the others.
> [...]
> 
> We did this for a while (ran pep8 first in the check pipeline, then
> ran other jobs if that passed), but it turned out to have a rather
> confusing interaction with the requirements validation jobs.
> 
> The long and short of it is that there are roughly two classes of
> jobs we want to consider... quick ones which are not very resource
> intensive, and more involved tests which are unlikely to succeed
> anyway if the quick ones fail. To that end there has been discussion
> of adding a "fast-fail" option for certain jobs causing them to
> abort others already running and return the result set right away so
> that developers can benefit from a more immediate testing feedback
> loop.
> 
> I don't think we have a timeline for implementation there, but it's
> my recollection that's the direction we resolved to take it.

Yeah. What Jeremy said.

Actually, fast-fail is one of the reasons that we've been pushing so
hard to get people on to testr and thus have the option for the tests to
emit subunit output streams. Our current thinking is that we will teach
zuul to read subunit streams as they happen, so that fast-fail can
report back overall failure and also re-order the gate queue as needed
because we know a job will fail - but we can still continue running the
jobs.

Note, that since this involves us reading a streaming protocol, it
doesn't depend on a job failing first. We should be able to report
failure for the set of jobs on the first instance of a single test case
failing.

As of right now, we have not begun development on this feature. I
believe we'd be more than happy to point someone in the right direction
if they wanted to work on it before we can get to it.

Monty