[openstack-dev] [All][Neutron] Improving development and review velocity - Is it time for a drastic change?

Neil Jerram Neil.Jerram at metaswitch.com
Fri Apr 1 08:57:04 UTC 2016


On 01/04/16 02:16, Assaf Muller wrote:
> Have you been negatively impacted by slow development and review
> velocity? Read on.
>
> OpenStack has had a slow review velocity for as long as I can
> remember. This has a cascading effect where people take up multiple
> tasks, so that they can work on something while the other is being
> reviewed. This adds even more patches to ever growing queues. Features
> miss releases and bugs never get fixed. Even worse, we turn away new
> contributors due to an agonizing process.
>
> In the Neutron community, we've tried a few things over the years.
> Neutron's growing scope was identified and load balancing, VPN and
> firewall as a service were split out to their own repositories.
> Neutron core reviewers had less load, *aaS contributors could iterate
> faster, it was a win win. Following that, Neutron plugins were split
> off as well. Neutron core reviewers did not have the expertise or
> access to specialized hardware of vendors anyway, vendors could
> iterate faster, and everybody were happy. Finally, a specialization
> system was created. Areas of the Neutron code base were determined and
> a "Lieutenant" was chosen for each area. That lieutenant could then
> nominate core reviewers, and those reviewers were then expected to +2
> only within their area. This led to doubling the core team, and for my
> money was a great success. Leading us to today.
>
> Today, I think it's clear we still have a grave problem. Patches sit
> idle for months, turning contributors away. I believe we've reached a
> tipping point, and now is the time for out of the box thinking. I am
> proposing two changes:
>
> 1) Changing what a core reviewer is. It is time to move to a system of
> trust: Everyone have +2 rights to begin with, and the system
> self-regulates by shaming offending individuals and eventually taking
> away rights for repeated errors in judgement. I've proposed a Neutron
> governance change here:
>
> https://review.openstack.org/300271
>
> 2) Now, transform yourself six to twelve months in the future. We now
> face a new problem. Patches are flying in. You're no longer working on
> a dozen patches in parallel. You push up something, it is reviewed
> promptly, and you move on to the next thing. Our next issue is then CI
> run-time. The time it takes to test (Check queue), approve and test a
> patch again (Gate queue) is simply too long. How do we cut this down?
> Again, by using a proven open source methodology of trust. As
> Neutron's testing lieutenant, I hereby propose that we remove the
> tests.

You had me until this sentence. :-)  Nice one!

	Neil

> Why deal with a problem you can avoid in the first place? The
> Neutron team has been putting out fires in the form of gate issues on
> a weekly basis, double so late in to a release cycle. The gate has so
> many false negatives, the tests are riddled with race conditions,
> we've clearly failed to get testing right. Needless to say, my
> proposal keeps pep8 in place. We all know how important a consistent
> style is. I've proposed a patch that removes Neutron's tests here:
>
> https://review.openstack.org/300272
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>




More information about the OpenStack-dev mailing list