Open Stack

On 14/08/14 00:03, James Polley wrote:
> In recent history, we've been looking each week at stats
> from http://russellbryant.net/openstack-stats/tripleo-openreviews.html
> to get a gauge on how our review pipeline is tracking.
> 
> The main stats we've been tracking have been the "since the last
> revision without -1 or -2". I've included some history at [1], but the
> summary is that our 3rd quartile has slipped from 13 days to 16 days
> over the last 4 weeks or so. Our 1st quartile is fairly steady lately,
> around 1 day (down from 4 a month ago) and median is unchanged around 7
> days.
> 
> There was lots of discussion in our last meeting about what could be
> causing this[2]. However, the thing we wanted to bring to the list for
> the discussion is:
> 
> Are we tracking the right metric? Should we be looking to something else
> to tell us how well our pipeline is performing?
> 
> The meeting logs have quite a few suggestions about ways we could tweak
> the existing metrics, but if we're measuring the wrong thing that's not
> going to help.
> 
> I think that what we are looking for is a metric that lets us know
> whether the majority of patches are getting feedback quickly. Maybe
> there's some other metric that would give us a good indication?

Bring back auto abandon...

Gerrit at one stage not so long ago used to auto abandon patches that
had negative feedback and was over a week without activity, I believe
this was removed when a gerrit upgrade gave core reviewers the ability
to abandon other peoples patches.

This was the single best part of the entire process to keep things moving
o submitters were forced to keep patches current
o reviewers were not looking at stale or already known broken patches
o If something wasn't important it got abandoned and was never heard of
again, if it was important it would be reopened
o patch submitters were forced to engage with the reviewers quickly on
negative feedback instead of leaving a patch sitting there indefinitely

Here is the number I think we should be looking at
http://russellbryant.net/openstack-stats/tripleo-reviewers-30.txt
  Queue growth in the last 30 days: 72 (2.4/day)

http://russellbryant.net/openstack-stats/tripleo-reviewers-90.txt
  Queue growth in the last 90 days: 132 (1.5/day)

Obviously this isn't sustainable

re enabling auto abandon will ensure the majority of the patches we are
looking at are current/good quality and not lost in a sea of -1's.

How would people feel about turning it back on? Can it be done on a per
project basis?

To make the whole process a little friendlier we could increase the time
frame from 1 week to 2.

Derek.

> 
> --------
> 
> 
> [1] Current "Stats since the last revision without -1 or -2" :
> 
> Average wait time: 10 days, 17 hours, 6 minutes
> 1st quartile wait time: 1 days, 1 hours, 36 minutes
> Median wait time: 7 days, 5 hours, 33 minutes
> 3rd quartile wait time: 16 days, 8 hours, 16 minutes
> 
> At last week's meeting we had: 3rd quartile wait time: 15 days, 13
> hours, 47 minutes
> A week before that: 3rd quartile wait time: 13 days, 9 hours, 11 minutes
> The week before that was the mid-cycle, but the week before that:
> 
> 19:53:38 <lifeless> Stats since the last revision without -1 or -2 :
> 19:53:38 <lifeless> Average wait time: 10 days, 17 hours, 49 minutes
> 19:53:38 <lifeless> 1st quartile wait time: 4 days, 7 hours, 57 minutes
> 19:53:38 <lifeless> Median wait time: 7 days, 10 hours, 52 minutes
> 19:53:40 <lifeless> 3rd quartile wait time: 13 days, 13 hours, 25 minutes
> 
> [2] Some of the things suggested as potential causes of the long 3rd
> median times:
> 
> * We have small number of really old reviews that have only positive
> scores but aren't being landed
> * Some reviews get a -1 but then sit for a long time waiting for the
> author to reply
> * We have some really old reviews that suddenly get revived after a long
> period being in WIP or abandoned, which reviewstats seems to miscount
> * Reviewstats counts weekends, we don't (so a change that gets pushed at
> 5pm US Friday and gets reviewed at 9am Aus Monday would be seen by us as
> having no wait time, but by reviewstats as ~36 hours)
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>