Open Stack

On 2013-08-10 14:20:40 +0000 (+0000), Bob Ball wrote:
[...]
> My understanding at the moment is that there is a check
> immediately prior to merging to make sure that there aren't any
> -2's added by core reviewers between the merge being approved and
> the gating jobs completing - is that right? If so, I'm not sure I
> understand the differences between a reviewer preventing a merge
> in this fashion and a known test failure from preventing it.
[...]

Just so I understand the suggestion, you're saying you can make
smokestack test the changes stacked in the same sequence they're
being started by zuul, watch zuul's status mirroring the various
change ejections/test restarts... and finish testing on a given set
of scenarios before zuul does? (Remember that zuul can effectively
start testing an arbitrary number of sequenced changes in parallel
so any resource congestion at the smokestack end could quickly cause
it to fall behind at load.)

You had mentioned something about smokestack adding a -2 when it
begins testing and removing it on success, but if it doesn't
complete before zuul is ready to merge those commits then how do we
differentiate between smokestack still running (so we know to pause
the queue) and smokestack leaving that -2 indefinitely because the
test failed? Also worth noting, gerrit doesn't clear a -2 on a new
patchset upload like it does with a -1 or a positive vote, so the
proposed solution would additionally need to take care of rescinding
those -2 votes itself when new patchsets arrive to correct the
failure.

This all sounds very fragile and a probably a non-trivial amount of
effort to adapt to the situation you describe. I'm in agreement with
the other voiced opinions that we should continue to work toward
testing everything we can in a well-integrated and timely manner,
provide a way for less tightly-coupled systems to chime in with
automated advisory notes which can help reviewers, and always be
prepared deal with the unfortunate but inevitable clean-up when
something merges which breaks a particular configuration (we have to
do that anyway because nobody will every be able to write tests
which exercise every possible eventuality).
-- 
Jeremy Stanley