Open Stack

On 14 December 2013 03:07, Russell Bryant <rbryant at redhat.com> wrote:
> On 12/12/2013 04:46 PM, Robert Collins wrote:
>> Hi, I'm trying to overhaul the bug triage process for nova (initially)
>> to make it much lighter and more effective.
>>
>> I'll be sending a more comprehensive mail shortly
>
> before you do, let's agree what we're trying to solve.  Perhaps you were
> going to cover that in your later message, but it wouldn't hurt
> discussing it now.

Sure.

> I actually didn't think our process was that broken.  It's more that I
> feel we need a person leading a small team that is working on it reguarly.

Yup, I agree. I wanted to get some data and check that the process is
actually straightforward before trying to build a team: it's easier to
build a group if the thing being built is straight foward.

http://webnumbr.com/.join%28nova-confirmed-undecided.all,untouched-nova-bugs.all%29.graph

 - that sawtooth pattern says to me that we're letting it build up,
then tackling it, then rinse and repeat. So either there isn't a team
around it, or folk burn out.

https://wiki.openstack.org/wiki/BugTriage is a 10 step process: thats
not something we can sensible do over coffee in the morning. It also
includes

In particular, the current definition has some significant
boil-the-ocean aspects such as:
"Review all medium and low bugs" - there are 340 open medium/low bugs,
230 bugs with no priority (both new and confirmed), and 500 high bugs.

We've fix-released 4000 bugs, and closed invalid 1200, opinion 46 and
have 1100 open That means we've fixed 4000/6300 bugs, or roughly
2/3rds. Thats *much* better than many other projects. OTOH the
practice we have of filing bugs when we put up patches may be
artificially inflating that: our bug database may not actually reflect
user visible issues.

We have 300 'confirmed', 200 'triaged' and 300 'in progress' bugs, 175
of which have been touched in the 2 months. By the current definition
of 'triaged', most of things we know actually are bugs still need
further handling. It may be that the current definition is part of the
very significant success we've had with actually fixing bugs... but
see above :)

> The idea with the tagging approach was to break up the triage problem
> into smaller work queues.  I haven't kept up with the tagging part and
> would really like to hand that off.  Then some of the work queues aren't
> getting triaged as regularly as they need to.  I'd like to see a small
> team making this a high priority with some of their time each week.

Yep. So I get the idea of the tagging approach, but it just shifts
work around AFAICT, and the total volume of work isn't all that high -
long as we keep on top of it.

> With all of that said, if you think an overhaul of the process is
> necessary to get to the end goal of a more well triaged bug queue, then
> I'm happy to entertain it.

I think we need to split out:
 - what the world would call triage - identify that it is an issue and
how severe - and thus important - it is for the project
   - requires reasonable understanding of nova at a deployer perspective
 - preparing bugs for developers to run with it - moving to the
OpenStack triaged state
   - essentially this is design review so requires reviewer - probably
-core experienced folk - partiticipation
 - the repeated boil-the-ocean stuff : for now this shouldn't be part
of the daily loop. Perhaps something for bug days, or once-a-cycle
review.

So essentially, I want to take this:

1 Task 1: Confirm new bugs (anyone)
2 Task 2: Prioritize confirmed bugs (bug supervisors)
3 Task 3: Solve inconsistencies (anyone)
3.1 New bugs with a priority set
3.2 In progress bugs without an assignee
4 Task 4: Review incomplete bugs (anyone)
5 Task 5: Review stale In Progress bugs (anyone)
6 Task 6: Review bugs with a patch (bug supervisors)
7 Task 7: Review Critical/High bugs (bug supervisors)
8 Task 8: Review Medium/Low bugs (bug supervisors)
9 Task 9: Deprecate old wishlist bugs (bug supervisors)
10 Task 10: Celebrate!

And turn it into something like this:
Daily tasks - first- layer - need to be broadly familiar with Nova but
doesn't require -core knowhow
1: Prioritize unprioritized bugs (bug supervisors)
  1.1 set bugs to Invalid if questions
  1.2 set bugs to Incomplete if questions asked
  1.4 patches are immediately directed to Gerrit
  1.5 subject area tags are applied
  1.3 output is Confirmed or Triaged + Priority set
2: Review Incomplete-with-response bugs
3: Review: In progress bugs without an assignee

Daily tasks - second layer - -core current and previous members
1. Assess the proposed approach in Confirmed+High[1] bugs
1.1. If good, move to Triaged
1.2  If not, suggest what would make the approach good[2]
2. If appropriate add low-hanging-fruit tag

Per-release:
1: Review stale In Progress bugs (anyone)
2: Review Critical/High bugs (bug supervisors)
3: Review Medium/Low bugs (bug supervisors)
4: Deprecate old wishlist bugs (bug supervisors)



1: I don't believe in putting a lot of expensive [read limited
resource] effort into every bug: lets stay focused on the priority
bugs. If we run out, we can always start doing that for medium bugs
too.

2: I really don't like this - we'll end up with a large set of bugs
that need to be manually scanned across to find ones that need
re-assessment, it's doing to be soul-sucking assessing that. I think
we need some tooling support here.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud