Open Stack

On 04/05/2013 12:21 PM, Stefano Maffulli wrote:
> On 04/05/2013 06:08 AM, Sean Dague wrote:
>> Maybe we just need public mocking when any organization puts out
>> metric based numbers like that. Treat it as a back end problem. :)
> 
> Wonderful, this is what I've been thinking reading the thread so far. I
> don't see any real incentive for gaming the system but I see clearly a
> dis-incentive on gaming the system.
> 
> Let's avoid getting too excited about this: numbers don't tell stories
> by themselves, lines of code are not a meaningful metric, number of bugs
> closed/opened have no inner value. But when you look at all these dots
> from a distance you can use these numbers to tell a story and understand
> the enviroment you're in. If you look at the blog post by Bitergia
> you'll see what I mean: there is a story that has to be read with the
> numbers. And there are even more stories to tell if you look at the
> 'long tail' of our community, if you beyond the "top 10".

I understand that folks are righfully reluctant for us to publish
numbers due to possibility of gaming. HOWEVER - _every_ large company
involved in OpenStack is going to do this by themselves if we don't.
When you get as large as many of these players are, and when you are
spending as much money on the project as each of them are, it is
unavoidable that various people who are not involved daily will want a
pulse on what's going on in terms that they understand.

On the one hand, the numbers are a potential for gaming. On the other
hand, there is an extremely beneficial positive use for them as well -
shaming your management in to hiring more developers, or providing
incentive for the developers they do have to contribute their patches back.

Numbers can lie - but look at the broad-strokes of many of these reports
and tell me if you see anything that's just totally off base. All of the
devs in the gitdm git list seem like reasonable choices as active devs
to me - and the gerrit review list both on individual and employer seems
about right. Now, is the exact relative activity level between Rackspace
IBM and RedHat 100% accurate? Who knows. Were all three of them super
involved this last cycle? Yes.

In any case - I totally hear everyone who has concerns - but I know that
if there isn't an OpenStack list of numbers that's published - people
will make them internally. And if you don't think that various
middle-managers who aren't actually contributing aren't going to cook up
some slice of the numbers that makes them look good... well, then you
should talk about how awesome your company is more often.

> On 04/05/2013 02:03 AM, Thierry Carrez wrote:
>> On the other hand, more and more people/companies want to be able to
>>  brag about their OpenStack contribution, or want to not appear too
>> bad on those lists. That leads to the incentive to game the system,
>> once some stats are made "official".
> 
> We'll cross that bridge when we get there: I don't think that there is a
> shortcut to appear on those stats than do a lot of work and let the
> facts talk. Are there lots of smaller companies whose contributions  are
> dwarfed by the top 10 contributors? Sure. Would they want to build a
> story about their one developer pushing a very good patch or 100 "low
> hanging fruit" bug fixes for a petty press release? We can't be
> responsible for the behaviour of incompetent marketing managers :)
> 
>> Until now, the stats posted were mostly one-shot and ad-hoc, to
>> serve as a reputational pressure encouraging people to contribute
>> more,
> 
> That's not why I started the Activity board. My objective is to
> understand the dynamics of the community. Think of the car's dashboard:
> it is necessary to give information about the car's speed and the
> engine's internals, and to give the driver information to make decisions.
> 
> Same thing for the community dashboards: we need it to get information
> and take better decisions. Is somebody increasing contributions? Good,
> let's go see why. Is somebody decreasing development efforts? What
> happened? How can we fix that?
> 
>> The metrics themselves can be a bit unfair: we already mentioned
>> the issue with "number of commits"... but others fail as well
> 
> I'll keep asking for help to understand how the numbers work and what
> they mean, how they relate to our development cycles and processes.
> We'll have a long session at the summit to discuss the Activity Board,
> its tools and meanings. The intention I have is to make this a community
> project, with gated public repository. Since evidently all companies are
> interested in getting these data, I ask all to join the session
> 
> http://openstacksummitapril2013.sched.org/event/ebbf37e343246e0b80bf99540a0c8d86#.UVNw0lGJSoM
> 
> 
> /stef
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>