[openstack-dev] [All][Neutron] Improving development and review velocity - Is it time for a drastic change?

Nikhil Komawar nik.komawar at gmail.com
Fri Apr 1 01:46:40 UTC 2016

And now I wish I could delete my messages sent to ML.

On 3/31/16 9:38 PM, Nikhil Komawar wrote:
> 2) is a giveaway it's Apr 1 (in some TZs)!
> On 3/31/16 9:12 PM, Assaf Muller wrote:
>> Have you been negatively impacted by slow development and review
>> velocity? Read on.
>> OpenStack has had a slow review velocity for as long as I can
>> remember. This has a cascading effect where people take up multiple
>> tasks, so that they can work on something while the other is being
>> reviewed. This adds even more patches to ever growing queues. Features
>> miss releases and bugs never get fixed. Even worse, we turn away new
>> contributors due to an agonizing process.
>> In the Neutron community, we've tried a few things over the years.
>> Neutron's growing scope was identified and load balancing, VPN and
>> firewall as a service were split out to their own repositories.
>> Neutron core reviewers had less load, *aaS contributors could iterate
>> faster, it was a win win. Following that, Neutron plugins were split
>> off as well. Neutron core reviewers did not have the expertise or
>> access to specialized hardware of vendors anyway, vendors could
>> iterate faster, and everybody were happy. Finally, a specialization
>> system was created. Areas of the Neutron code base were determined and
>> a "Lieutenant" was chosen for each area. That lieutenant could then
>> nominate core reviewers, and those reviewers were then expected to +2
>> only within their area. This led to doubling the core team, and for my
>> money was a great success. Leading us to today.
>> Today, I think it's clear we still have a grave problem. Patches sit
>> idle for months, turning contributors away. I believe we've reached a
>> tipping point, and now is the time for out of the box thinking. I am
>> proposing two changes:
>> 1) Changing what a core reviewer is. It is time to move to a system of
>> trust: Everyone have +2 rights to begin with, and the system
>> self-regulates by shaming offending individuals and eventually taking
>> away rights for repeated errors in judgement. I've proposed a Neutron
>> governance change here:
>> https://review.openstack.org/300271
>> 2) Now, transform yourself six to twelve months in the future. We now
>> face a new problem. Patches are flying in. You're no longer working on
>> a dozen patches in parallel. You push up something, it is reviewed
>> promptly, and you move on to the next thing. Our next issue is then CI
>> run-time. The time it takes to test (Check queue), approve and test a
>> patch again (Gate queue) is simply too long. How do we cut this down?
>> Again, by using a proven open source methodology of trust. As
>> Neutron's testing lieutenant, I hereby propose that we remove the
>> tests. Why deal with a problem you can avoid in the first place? The
>> Neutron team has been putting out fires in the form of gate issues on
>> a weekly basis, double so late in to a release cycle. The gate has so
>> many false negatives, the tests are riddled with race conditions,
>> we've clearly failed to get testing right. Needless to say, my
>> proposal keeps pep8 in place. We all know how important a consistent
>> style is. I've proposed a patch that removes Neutron's tests here:
>> https://review.openstack.org/300272
> Well played! But 2) gave it away =]
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list