<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Oct 10, 2013 at 5:43 PM, Tim Smith <span dir="ltr"><<a href="mailto:tsmith@gridcentric.com" target="_blank">tsmith@gridcentric.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>On Thu, Oct 10, 2013 at 1:50 PM, Russell Bryant <span dir="ltr"><<a href="mailto:rbryant@redhat.com" target="_blank">rbryant@redhat.com</a>></span> wrote:<br>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Please understand that I only want to help here. Perhaps a good way for<br>
you to get more review attention is get more karma in the dev community<br>
by helping review other patches. It looks like you don't really review<br>
anything outside of your own stuff, or patches that touch hyper-v. In<br>
the absence of significant interest in hyper-v from others, the only way<br>
to get more attention is by increasing your karma.<br></blockquote><div><br></div></div><div>NB: I don't have any vested interest in this discussion except that I want to make sure OpenStack stays "Open", i.e. inclusive. I believe the concept of "reviewer karma", while seemingly sensible, is actually subtly counter to the goals of openness, innovation, and vendor neutrality, and would also lead to overall lower commit quality.</div>
<div><br></div></div></div></div></blockquote><div><br></div><div>The way I see it there are a few parts to 'karma' including:</div><div><br></div><div>* The ratio of reviewers to open patches is way off. In nova there are only 21 reviewers who have done on average two reviews a day for the past 30 days [1], and there are 226 open reviews, 125 of which are waiting for a reviewer. So one part of the karma is helping out the team as a whole with the review work load (and the more insightful the review the better). If we have more reviewers, more patches get looked at faster.</div>
<div>* The more I see someone being active, through reviews or through patches, the more I trust there +1/-1s and patches.</div><div><br></div><div><br></div><div>While there are some potentially negative sides to karma, I don't see how the properties above, which to me are the major elements of karma, can be considered negative.</div>
<div><br></div><div><br></div><div>[1] <a href="http://www.russellbryant.net/openstack-stats/nova-reviewers-30.txt" target="_blank">http://www.russellbryant.net/openstack-stats/nova-reviewers-30.txt</a></div>
<div>[2] <a href="http://www.russellbryant.net/openstack-stats/nova-openreviews.html" target="_blank">http://www.russellbryant.net/openstack-stats/nova-openreviews.html</a></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>Brian Kernighan famously wrote: "Debugging is twice as hard as writing the code in the first place." A corollary is that constructing a mental model of code is hard; perhaps harder than writing the code in the first place. It follows that reviewing code is not an easy task, especially if one has not been intimately involved in the original development of the code under review. In fact, if a reviewer is not intimately familiar with the code under review, and therefore only able to perform the functions of human compiler and style-checker (functions which can be and typically are performed by automatic tools), the rigor of their review is at best less-than-ideal, and at worst purely symbolic.</div>
<div><br></div></div></div></div></blockquote><div><br></div><div>FWIW, we have automatic style-checking.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>It is logical, then, that a reviewer should review changes to code that he/she is familiar with. Attempts to gamify the implicit review prioritization system through a "karma" scheme are sadly doomed to fail, as contributors hoping to get their patches reviewed will have no option but to "build karma" reviewing patches in code they are unfamiliar with, leading to a higher number of low quality reviews.</div>
<div><br></div><div>So, if a cross-functional "karma" system won't accomplish the desired result (high-quality reviews of commits across all functional units), what will it accomplish (besides overall lower commit quality)?</div>
<div><br></div><div>Because the "karma" system inherently favors entrenched (read: heavily deployed) code, it forms a slippery slope leading to a mediocre "one-size-fits-all" stack, where contributors of new technologies, approaches, and hardware/software drivers will see their contributions die on the vine due to lack of core reviewer attention. If the driver team for a widely deployed hypervisor (outside of the OpenStack space - they can't really be expected to have wide OpenStack deployment without a mature driver) is having difficulty with reviews due to an implicit "karma" deficit, imagine the challenges that will be faced by the future SDN/SDS/SDx innovators of the world hoping to find a platform for their innovation in OpenStack.</div>
<div><br></div><div>Again, I don't have any vested interest in this discussion, except that I believe the concept of "reviewer karma" to be counter to both software quality and openness. In this particular case it would seem that the simplest solution to this problem would be to give one of the hyper-v team members core reviewer status, but perhaps there are consequences to that that elude me.</div>
</div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra">
<div class="gmail_quote">
<div><br></div><div>Regards,</div><div>Tim</div><div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<a href="https://review.openstack.org/#/q/reviewer:3185+project:openstack/nova,n,z" target="_blank">https://review.openstack.org/#/q/reviewer:3185+project:openstack/nova,n,z</a><br>
<span><font color="#888888"><br>
--<br>
Russell Bryant<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</font></span></blockquote></div></div><br></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>