<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Oct 10, 2013 at 6:57 PM, Matt Riedemann <span dir="ltr"><<a href="mailto:mriedem@us.ibm.com" target="_blank">mriedem@us.ibm.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font face="sans-serif">Getting integration testing hooked up for
the hyper-v driver with tempest should go a long way here which is a good
reason to have it. As has been mentioned, there is a core team of
people that understand the internals of the hyper-v driver and the subtleties
of when it won't work, and only those with a vested interest in using it
will really care about it.</font>
<br>
<br><font face="sans-serif">My team has the same issue with the
powervm driver. We don't have community integration testing hooked
up yet. We run tempest against it internally so we know what works
and what doesn't, but besides standard code review practices that apply
throughout everything (strong unit test coverage, consistency with other
projects, hacking rules, etc), any other reviewer has to generally take
it on faith that what's in there works as it's supposed to. Sure,
there is documentation available on what the native commands do and anyone
can dig into those to figure it out, but I wouldn't expect that low-level
of review from anyone that doesn't regularly work on the powervm driver.
I think the same is true for anything here. So the equalizer
is a rigorously tested and broad set of integration tests, which is where
we all need to get to with tempest and continuous integration.</font>
<br></blockquote><div><br></div><div>Well said, I couldn't agree more!</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br><font face="sans-serif">We've had the same issues as mentioned
in the original note about things slipping out of releases or taking a
long time to get reviewed, and we've had to fork code internally because
of it which we then have to continue to try and get merged upstream - and
it's painful, but it is what it is, that's the nature of the business.</font>
<br>
<br><font face="sans-serif">Personally my experience has been that
the more I give the more I get. The more I'm involved in what others
are doing and the more I review other's code, the more I can build a relationship
which is mutually beneficial. Sometimes I can only say 'hey, you
need unit tests for this or this doesn't seem right but I'm not sure',
but unless you completely automate code coverage metrics and build that
back into reviews, e.g. does your 1000 line blueprint have 95% code coverage
in the tests, you still need human reviewers on everything, regardless
of context. Even then it's not going to be enough, there will always
be a need for people with a broader vision of the project as a whole that
can point out where things are going in the wrong direction even if it
fixes a bug.</font>
<br>
<br><font face="sans-serif">The point is I see both sides of the
argument, I'm sure many people do. In a large complicated project
like this it's inevitable. But I think the quality and adoption of
OpenStack speaks for itself and I believe a key component of that is the
review system and that's only as good as the people which are going to
uphold the standards across the project. I've been on enough development
projects that give plenty of lip service to code quality and review standards
which are always the first thing to go when a deadline looms, and those
projects are always ultimately failures.<br>
</font>
<br><font face="sans-serif"><br>
</font>
<br><font size="1" face="Arial">Thanks,</font>
<br>
<br><font size="3" color="#8f8f8f" face="Arial"><b>MATT RIEDEMANN</b></font><font size="1" face="Arial"><br>
Advisory Software Engineer<br>
Cloud Solutions and OpenStack Development</font>
<table width="680" style="border-collapse:collapse">
<tbody><tr height="8">
<td width="680" colspan="2" style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px">
<hr>
</td></tr><tr valign="top" height="8">
<td width="418" style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#4181c0" face="Arial"><b>Phone:</b></font><font size="1" color="#5f5f5f" face="Arial">
<a href="tel:1-507-253-7622" value="+15072537622" target="_blank">1-507-253-7622</a></font><font size="1" color="#4181c0" face="Arial"> | <b>Mobile:</b></font><font size="1" color="#5f5f5f" face="Arial">
<a href="tel:1-507-990-1889" value="+15079901889" target="_blank">1-507-990-1889</a></font><font size="1" color="#4181c0" face="Arial"><b><br>
E-mail:</b></font><font size="1" color="#5f5f5f" face="Arial"> </font><a href="mailto:mriedem@us.ibm.com" target="_blank"><font size="1" color="#5f5f5f" face="Arial"><u>mriedem@us.ibm.com</u></font></a>
</td><td width="261" style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px">
<div align="right"><img src="cid:_1_09DE5D3C09DE57A8000ACC6E86257C01" width="83" height="30" alt="IBM"><font size="1" color="#5f5f5f" face="Arial"><br>
<br>
3605 Hwy 52 N<br>
Rochester, MN 55901-1407<br>
United States</font></div></td></tr></tbody></table>
<br>
<br>
<br>
<br>
<br><font size="1" color="#5f5f5f" face="sans-serif">From:
</font><font size="1" face="sans-serif">Tim Smith <<a href="mailto:tsmith@gridcentric.com" target="_blank">tsmith@gridcentric.com</a>></font>
<br><font size="1" color="#5f5f5f" face="sans-serif">To:
</font><font size="1" face="sans-serif">OpenStack Development
Mailing List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>>, </font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Date:
</font><font size="1" face="sans-serif">10/10/2013 07:48 PM</font>
<br><font size="1" color="#5f5f5f" face="sans-serif">Subject:
</font><font size="1" face="sans-serif">Re: [openstack-dev]
[Hyper-V] Havana status</font>
<br>
<hr noshade><div class="HOEnZb"><div class="h5">
<br>
<br>
<br><font size="3">On Thu, Oct 10, 2013 at 1:50 PM, Russell Bryant <</font><a href="mailto:rbryant@redhat.com" target="_blank"><font size="3" color="blue"><u>rbryant@redhat.com</u></font></a><font size="3">>
wrote:</font>
<br><font size="3"> </font>
<br><font size="3">Please understand that I only want to help here. Perhaps
a good way for<br>
you to get more review attention is get more karma in the dev community<br>
by helping review other patches. It looks like you don't really review<br>
anything outside of your own stuff, or patches that touch hyper-v. In<br>
the absence of significant interest in hyper-v from others, the only way<br>
to get more attention is by increasing your karma.</font>
<br>
<br><font size="3">NB: I don't have any vested interest in this discussion
except that I want to make sure OpenStack stays "Open", i.e.
inclusive. I believe the concept of "reviewer karma", while seemingly
sensible, is actually subtly counter to the goals of openness, innovation,
and vendor neutrality, and would also lead to overall lower commit quality.</font>
<br>
<br><font size="3">Brian Kernighan famously wrote: "Debugging is
twice as hard as writing the code in the first place." A corollary
is that constructing a mental model of code is hard; perhaps harder than
writing the code in the first place. It follows that reviewing code is
not an easy task, especially if one has not been intimately involved in
the original development of the code under review. In fact, if a reviewer
is not intimately familiar with the code under review, and therefore only
able to perform the functions of human compiler and style-checker (functions
which can be and typically are performed by automatic tools), the rigor
of their review is at best less-than-ideal, and at worst purely symbolic.</font>
<br>
<br><font size="3">It is logical, then, that a reviewer should review changes
to code that he/she is familiar with. Attempts to gamify the implicit review
prioritization system through a "karma" scheme are sadly doomed
to fail, as contributors hoping to get their patches reviewed will have
no option but to "build karma" reviewing patches in code they
are unfamiliar with, leading to a higher number of low quality reviews.</font>
<br>
<br><font size="3">So, if a cross-functional "karma" system won't
accomplish the desired result (high-quality reviews of commits across all
functional units), what will it accomplish (besides overall lower commit
quality)?</font>
<br>
<br><font size="3">Because the "karma" system inherently favors
entrenched (read: heavily deployed) code, it forms a slippery slope leading
to a mediocre "one-size-fits-all" stack, where contributors of
new technologies, approaches, and hardware/software drivers will see their
contributions die on the vine due to lack of core reviewer attention. If
the driver team for a widely deployed hypervisor (outside of the OpenStack
space - they can't really be expected to have wide OpenStack deployment
without a mature driver) is having difficulty with reviews due to an implicit
"karma" deficit, imagine the challenges that will be faced by
the future SDN/SDS/SDx innovators of the world hoping to find a platform
for their innovation in OpenStack.</font>
<br>
<br><font size="3">Again, I don't have any vested interest in this discussion,
except that I believe the concept of "reviewer karma" to be counter
to both software quality and openness. In this particular case it would
seem that the simplest solution to this problem would be to give one of
the hyper-v team members core reviewer status, but perhaps there are consequences
to that that elude me.</font>
<br>
<br><font size="3">Regards,</font>
<br><font size="3">Tim</font>
<br><font size="3"> </font>
<br>
<br><font size="3" color="blue"><u><br>
</u></font><a href="https://review.openstack.org/#/q/reviewer:3185+project:openstack/nova,n,z" target="_blank"><font size="3" color="blue"><u>https://review.openstack.org/#/q/reviewer:3185+project:openstack/nova,n,z</u></font></a><font size="3" color="#8f8f8f"><br>
<br>
--<br>
Russell Bryant<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list</font><font size="3" color="blue"><u><br>
</u></font><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank"><font size="3" color="blue"><u>OpenStack-dev@lists.openstack.org</u></font></a><font size="3" color="blue"><u><br>
</u></font><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><font size="3" color="blue"><u>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</u></font></a>
<br><tt><font>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
</font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><tt><font>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</font></tt></a><tt><font><br>
</font></tt>
<br></div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>