[openstack-dev] [all][stackalytics] Gaming the Stackalytics stats
rbryant at redhat.com
Mon Apr 11 12:29:26 UTC 2016
On Mon, Apr 11, 2016 at 6:48 AM, Ihar Hrachyshka <ihrachys at redhat.com>
> Clint Byrum <clint at fewbar.com> wrote:
> Excerpts from Morgan Fainberg's message of 2016-04-10 16:47:28 -0700:
>>> On Sun, Apr 10, 2016 at 4:37 PM, Clint Byrum <clint at fewbar.com> wrote:
>>> Excerpts from Matt Riedemann's message of 2016-04-09 06:42:54 -0700:
>>>>> There is also disincentive in +1ing a change that you don't understand
>>>>> and is wrong and then a core comes along and -1s it (you get dinged for
>>>>> the disagreement). And there is disincentive in -1ing a change for the
>>>>> wrong reasons (silly nits or asking questions for understanding). I ask
>>>>> a lot of questions in a lot of changes and I don't vote on those
>>>>> it would be inappropriate.
>>>> Why is disagreement a negative thing? IMO, reviewers who agree too much
>>>> are just part of the echo chamber.
>>> There is no problem with disagreement IMHO. However, we track it as a
>>> and people don't want to feel as though they are in disagreement with the
>>> cores. I think this is just some level of psychology.
>>> I very, very rarely look at disagreement stat for anything (now or when I
>>> was PTL).
>> Agreed, as a number, it can be highly misleading and is especially hard
>> to compare to any of the other numbers.
>> However, in meta-reviews, I found actual occurrences very useful to
>> analyze how a reviewer handles confronting the other cores and how
>> confident they are in their understanding of the code base. So it worries
>> me that new people might be somehow discouraged from disagreement.
>> So let me just say it here, disagreeing with the core reviewers when
>> there is a valid reason _is what somebody who wants to be a core reviewer
>> should be doing_.
> Amen to that! I find that people who have higher disagreement stats are
> actually the people that add value to review process, since they obviously
> look at patches from perspectives that are different from existing core
> Now, I agree that if the disagreements are solely for nits in commit
> messages or random misunderstandings, then it’s not of value. But if those
> are legit concerns, that’s usually a good sign, not a bad one.
Note that the original definition of "disagreement" from reviewstats 
paid particular attention to ordering. A disagreement is only when a -core
team member votes against you, not the other way around. It was kind of
an experimental thing to see if it could help expose overly eager +1
reviewers (lots of reviews for stats, missing lots of errors). Maybe it
hasn't proved to be that valuable.
I haven't looked at how stackalytics implements it, though.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev