Open Stack

On 09/12/13 06:31, Steven Hardy wrote:
> Hi all,
>
> So I've been getting concerned about $subject recently, and based on some
> recent discussions so have some other heat-core folks, so I wanted to start
> a discussion where we can agree and communicate our expectations related to
> nomination for heat-core membership (becuase we do need more core
> reviewers):
>
> The issues I have are:
> - Russell's stats (while very useful) are being used by some projects as
>    the principal metric related to -core membership (ref TripleO's monthly
>    cull/name&shame, which I am opposed to btw).  This is in some cases
>    encouraging some stats-seeking in our review process, IMO.
>
> - Review quality can't be measured mechanically - we have some folks who
>    contribute fewer, but very high quality reviews, and are also very active
>    contributors (so knowledge of the codebase is not stale).  I'd like to
>    see these people do more reviews, but removing people from core just
>    because they drop below some arbitrary threshold makes no sense to me.

+1

Fun fact: due to the quirks of how Gerrit produces the JSON data dump, 
it's not actually possible for the reviewstats tools to count +0 
reviews. So, for example, one can juice one's review stats by actively 
obstructing someone else's work (voting -1) when a friendly comment 
would have sufficed. This is one of many ways in which metrics offer 
perverse incentives.

Statistics can be useful. They can be particularly useful *in the 
aggregate*. But as soon as you add a closed feedback loop you're no 
longer measuring what you originally thought - mostly you're just 
measuring the gain of the feedback loop.

> So if you're aiming for heat-core nomination, here's my personal wish-list,
> but hopefully others can proide their input and we can update the wiki with
> the resulting requirements/guidelines:
>
> - Make your reviews high-quality.  Focus on spotting logical errors,
>    reducing duplication, consistency with existing interfaces, opportunities
>    for reuse/simplification etc.  If every review you do is +1, or -1 for a
>    trivial/cosmetic issue, you are not making a strong case for -core IMHO.
>
> - Send patches.  Some folks argue that -core membership is only about
>    reviews, I disagree - There are many aspects of reviews which require
>    deep knowledge of the code, e.g spotting structural issues, logical
>    errors caused by interaction with code not modified by the patch,
>    effective use of test infrastructure, etc etc.  This deep knowledge comes
>    from writing code, not only reviewing it.  This also gives us a way to
>    verify your understanding and alignment with our sylistic conventions.

I agree, though I have also heard a lot of folks say it should be just 
about the reviews. Of course the job of core is reviewing - to make sure 
good changes get in and bad changes get turned into good changes. But 
there are few better ways to acquire and demonstrate the familiarity 
with the codebase and conventions of the project necessary to be an 
effective reviewer than to submit patches. It makes no sense to blind 
ourselves to code contributions when considering whom to add to core - a 
single patch contains far more information than a thousand "+1 no 
comment" or "-1 typo in commit message" reviews.

> - Fix bugs.  Related to the above, help us fix real problems by testing,
>    reporting bugs, and fixing them, or take an existing bug and post a patch
>    fixing it.  Ask an existing team member to direct you if you're not sure
>    which bug to tackle.  Sending patches doing trivial cosmetic cleanups is
>    sometimes worthwhile, but make sure that's not all you do, as we need
>    -core folk who can find, report, fix and review real user-impacting
>    problems (not just new features).  This is also a great way to build
>    trust and knowledge if you're aiming to contribute features to Heat.
>
> - Engage in discussions related to the project (here on the ML, helping
>    users on the general list, in #heat on Freenode, attend our weekly
>    meeting if it's not an anti-social time in your TZ)
>
> Anyone have any more thoughts to add here?

+1

The way to be recognised in an Open Source project should be to 
consistently add value to the community. Concentrate on that and the 
stats will look after themselves.

cheers,
Zane.