Open Stack

Michael Still wrote:
> On Fri, Apr 5, 2013 at 2:47 AM, Stefano Maffulli <stefano at openstack.org> wrote:
>> On 04/04/2013 08:24 AM, Mark McLoughlin wrote:
> 
> [snip]
> 
>>> Frankly, though, I'm losing faith in this "number of commits" statistic
>>> being all that useful. Most of my commits were probably trivial cfg
>>> cleanups in Nova.
>>
>> Not one single number has meanings by itself but only acquires a meaning
>> inside a story. If you look at count of commits together with the number of
>> bugs closed over time, so you can see trends, for example, you may have an
>> idea of how much effort goes into stabilization vs new features. Looking at
>> commits and lines of code added/removed can also help understand where the
>> development effort is going. Or at least the numbers will seem to tell
>> something and force you to want to dive deeper.
> 
> Yes. I am increasingly worried about employers managing to these
> numbers as well. I feel that the numbers that get discussed in the
> community are the ones employees are likely to optimise for.
> 
> I'd like to see a more general discussion about what contributions we
> value as a community, and how we encourage those contributions.

Yeah, I'm having the same concern, and the official "activity board" is
only making it more pressing:

On one hand, it's extremely difficult to get those stats right. Tracking
where everyone works is painful (the Bitergia report, for example, still
counts me under "Rackspace"). The metrics themselves can be a bit
unfair: we already mentioned the issue with "number of commits"... but
others fail as well (like "bug activity": stable branch management will
result in opening a lot of bugs for process tracking, which will be an
order of magnitude bigger than the number of bugs opened due to QA
research).

On the other hand, more and more people/companies want to be able to
brag about their OpenStack contribution, or want to not appear too bad
on those lists. That leads to the incentive to game the system, once
some stats are made "official".

Until now, the stats posted were mostly one-shot and ad-hoc, to serve as
a reputational pressure encouraging people to contribute more,
especially in strategic areas. But if they become official and stable
metrics, they will be gamed, defeating their original purpose.

Bruce Schneier puts it in better words than I do in "Liars and Outliers":

"When you start measuring something and then judge people based on that
measurement, you encourage people to game the measurement instead of
doing whatever it is you wanted in the first place."

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack