Open Stack

On Thu, 2015-04-02 at 07:03 -0400, Sean Dague wrote:
> On 04/02/2015 06:54 AM, James Bottomley wrote:
> > On Thu, 2015-04-02 at 06:45 -0400, Sean Dague wrote:
> >> On 04/02/2015 06:33 AM, James Bottomley wrote:
> >>> On Thu, 2015-04-02 at 11:32 +0200, Thierry Carrez wrote:
> >>>> Sean Dague wrote:
> >>>>> I just spent a chunk of the morning purging out some really old
> >>>>> Incomplete bugs because about 9 months ago we disabled the auto
> >>>>> expiration bit in launchpad -
> >>>>> https://bugs.launchpad.net/nova/+configure-bugtracker
> >>>>>
> >>>>> This is a manually grueling task, which by looking at these bugs, no one
> >>>>> else is doing. I'd like to turn that bit back on so we can actually get
> >>>>> attention focused on actionable bugs.
> >>>>>
> >>>>> Any objections here?
> >>>>
> >>>> No objection, just a remark:
> >>>>
> >>>> One issue with auto-expiration is that it usually results in the
> >>>> following story:
> >>>>
> >>>> 1. Someone reports bug
> >>>> 2. Triager notices NEW bug, asks reporter for details, sets INCOMPLETE
> >>>> 3. Reporter provides details
> >>>> 4. Triager does not notice reply on bug since they ignore INCOMPLETE
> >>>> 5. Bug expires after n months and disappears forever
> >>>> 6. Reporter is frustrated and won't ever submit issues again
> >>>>
> >>>> The problem is of course at step 4, not at step 5. Auto-expiration is
> >>>> very beneficial if your bug triaging routine includes checking Launchpad
> >>>> for "INCOMPLETE bugs with an answer" regularly. If nobody does this very
> >>>> boring task, then auto-expiration can be detrimental.
> >>>>
> >>>> Is anyone in Nova checking for "INCOMPLETE bugs with an answer" ? That's
> >>>> task 4 in https://wiki.openstack.org/wiki/BugTriage ...
> >>>
> >>> This actually looks to be a problem in the workflow to me.
> >>>
> >>> The OpenStack Incomplete/Confirmed seem to map roughly to the bugzilla
> >>> Need Info/Open states.  The difference is that in bugzilla, a reporter
> >>> can clear the Need Info flag.  This is also what needs to happen in
> >>> OpenStack (so the reporter doesn't need to wait on anyone looking at
> >>> thier input to move the bug on).
> >>>
> >>> I propose allowing the reporter to move the bug to Confirmed when they
> >>> supply the information making it incomplete.  If the triager thinks this
> >>> is wrong, they can set it back to incomplete again.  This has the net
> >>> effect that Incomplete needs no real review, it marks bugs the reporter
> >>> doesn't care enough about to reply... and these can be auto expired.
> >>>
> >>> This would make the initial state diagram
> >>>
> >>>
> >>> +---+    Review     +----------+
> >>> |New|-------------->|Incomplete|
> >>> +---+               +----------+
> >>>   |                     ^   |
> >>>   |    Still Needs Info |   | Reporter replies
> >>>   |                     |   v
> >>>   |     Review      +---------+
> >>>   +---------------->|Confirmed|
> >>>                     +---------+
> >>>
> >>>
> >>> James
> >>
> >> Reporters can definitely move it back to New, which is the expected
> >> flow, that means it gets picked up again on the next New bug sweep.
> >> That's Step #1 in triaging (for Nova we've agressively worked to keep
> >> that very near 0). I don't remember if they can also move it into
> >> Confirmed themselves if they aren't in the nova-bugs group, though that
> >> is an open group.
> >>
> >> Mostly the concern is people that don't understand the tools or bug
> >> flow. So they respond and leave in Incomplete. Or it's moved to
> >> Incomplete and they never respond because they don't understand that
> >> more info is needed. These things sit there for a year, and then there
> >> is some whiff of a real problem in them, but no path forward with that
> >> information.
> > 
> > But we have automation: the system can move it to Confirmed when they
> > reply.  The point is to try to make the states and timeouts self
> > classifying.  If incomplete means no-one cared enough about this bug to
> > supply requested information, then it's a no brainer candidate for
> > exipry.  The question I was asking is "could the states be set up so
> > this happens" and I believe the answer based on the above workflow is
> > "yes".
> > 
> > Now if it sits in Confirmed because the triager didn't read the supplied
> > information, it's not a candidate for expiry, it's a candidate for
> > kicking someone's arse.
> > 
> > The fundamental point is to make the states align with time triggered
> > actionable consequences.  That's what I believe the problem with the
> > current workflow is.  Someone has to look at each bug to determine what
> > Incomplete actually means which I'd view as unbelievably painful for
> > that person (or group of people).
> 
> So, it's launchpad, we only have so much control over things as we don't
> host it.

Hm ... I wonder if the company that hosts it might be responsive to
community requests ...?  They can only say no ... and it is (now,
finally) open source, so you could run your own.

> But that being said, New is a better state. It means that a bug triager
> actually looks at the response and says "yep, that's sufficient" and
> moves it to Confirmed. I'm not sure what the problem is there as long as
> New bugs are getting looked at regularly.
> 
> In almost all times Incomplete is set with a Question. We don't just
> flag things Incomplete without commentary.

OK, as long as the bug goes back to "New" automatically after the
reporter supplies a comment if it's in Incomplete, I think that serves
the purpose I was looking for (actionable states with timeouts).
Perhaps the state after a response should depend on what the initial
reviewer did.  If they assigned it a priority, it should go to Confirmed
and if they didn't it should go back to New.

> Realistically we've gotten a handle on the top end (Triaged which means
> directly actionable) the bottom end (New) in Nova. Where things are
> getting lost is in the giant pool of Confirmed Medium/Low that sit and
> rot and people don't tend to work on them.

Yes, so here, I think there should be some type of pinger for the bug
team.  I'd probably be happy to expire Low/Medium priority Confirmed
bugs (it's a bug, but perhaps it wasn't worth our time to fix) and save
the arse kicker for the High priority ones.  However there should also
be a useful mechanism to boost the priority ... say by number of me-too
flaggers on the bug.

James