Open Stack

Matt Riedemann <mriedem at linux.vnet.ibm.com> writes:

> People get heads-down in their own projects and what they are working
> on and it's hard to keep up with what's going on in the infra channel
> (or nova channel for that matter), so sending out a recap that
> everyone can see in the mailing list is helpful to reset where things
> are at and focus possibly various isolated investigations (as we saw
> happen this week).

Further on that point, Joe and I and others have been brainstorming
about how to prevent this situation and improve things when it does
happen.  To that end, I'd like to propose we adopt some process around
gate-blocking bugs:

1) The QA team should have the ability to triage bugs in _all_ OpenStack
projects, specifically so that they may set gate-blocking bugs to
critical priority.

2) If there isn't an immediately obvious assignee for the bug, send an
email to the -dev list announcing it and asking for someone to take or
be assigned to the bug.

I think the expectation should be that the bug triage teams or PTLs
should help get someone assigned to the bug in a reasonable time (say,
24 hours, or ideally much less).

3) If things get really bad, as they have recently, we send a mail to
the list asking core devs to stop approving patches that don't address
gate-blocking bugs.

I don't think any of this is revolutionary -- we have more or less done
these things already in this situation, but we usually take a while to
get there.  I think setting expectations around this and standardizing
how we proceed will make us better able to handle it.

Separately we will be following up with information on some changes that
we hope will reduce the likelihood of nondeterministic bugs creeping in
in the first place.

-Jim