Open Stack

On Fri, Aug 15, 2014 at 2:26 PM, Kevin Benton <blak111 at gmail.com> wrote:
> I definitely agree that reviewer time is wasted reviewing changes. However,
> I don't think moving them to a different repo with different cores is going
> to make them less brittle without some very strict guidelines about what a
> driver/plugin is allowed to do.
>
Agreed.

> For example, without neutron core reviewer oversight, what prevents a plugin
> author from monkey patching parts of the neutron API? If nothing, that will
> immediately break on any kind of API refactoring, module renaming, etc.
>
The fact that this will need to be merged in a repo under the
networking program means we would catch this. However, the person who
wants to monkey patch like this could easily move their plugin out of
tree and monkey patch to their hearts content.

> That scenario also brings up another concern. Will changes to neutron that
> break a vendor plugin even be blocked on the neutron side? If so, the
> neutron repo will be held hostage by third-party code that isn't in Neutron
> and lacks the quality control it would have in Neutron. If not, this will
> immediately break the gate on the driver repo, forcing maintainers to
> disable the gating job for that vendor plugin. Neither of these scenarios
> seem less brittle to me.
>
If we had cross-repo CI running (like was suggested for the
incubator), we would catch things like this. In other words, if the
driver repo ran for patches to the neutron repo and posted back, we
could catch this.

> What the PLUMgrid folks did works; however, IIUC it was at the sacrifice of
> unit tests verifying any calls into the plumlib. There is just a fake driver
> that accepts anything for the unit tests. [1] They could implement a lot of
> mock logic to simulate the real driver, but then we are back to square one
> and they might as well put the actual driver there.
>
> I'm all for moving drivers/plugins out of Neutron, but we need to be really
> careful here because we are going to lose a lot of quality control that
> Neutron could end up taking the blame for since these drivers/plugins are
> still in a public repo.
>
++, this is a critical area here. On the other hand, the current model
of adding 5-6 new plugins/drivers for proprietary backends each cycle
won't scale going forward, and the level of involvement of most of
these companies ends at their plugin. So something needs to change to
make this scalable going forward.

Kyle

> 1.
> https://github.com/openstack/neutron/blob/08529376f16837083c28b009411cc52e0e2a8d33/neutron/plugins/plumgrid/drivers/fake_plumlib.py
>
>
> On Fri, Aug 15, 2014 at 8:50 AM, Kyle Mestery <mestery at mestery.com> wrote:
>>
>> I think the review time alone is a huge issue. Even worse, for the
>> most part, core reviewers are reviewing code for which they themselves
>> can't test because it requires proprietary hardware or software,
>> making the situation brittle at best. Having a separate git repository
>> which is gated by stringent third-party CI requirements, with separate
>> (and possibly overlapping) core reviewers would help to alleviate this
>> problem. Another alternative is to move most intelligence out of the
>> plugins/drivers and into vendor owned packages which can live on pypi.
>> This is similar to what the PLUMgrid folks did for their plugin. This
>> allows vendor control over most of their bits, removes the constant
>> churn for simple bug fixes in the neutron tree, and adds the benefit
>> of being a part of the simultaneous release, which is the only thing
>> most vendors care about.
>>
>> On Thu, Aug 14, 2014 at 10:34 PM, Kevin Benton <blak111 at gmail.com> wrote:
>> >>I also feel like the drivers/plugins are currently BEYOND a tipping
>> > point, and are in fact dragging down velocity of the core project in
>> > many ways.
>> >
>> > Can you elaborate on the ways that they are slowing down the velocity? I
>> > know they take up reviewer time, but are there other ways that you think
>> > they slow down the project?
>> >
>> >
>> > On Thu, Aug 14, 2014 at 6:07 AM, Kyle Mestery <mestery at mestery.com>
>> > wrote:
>> >>
>> >> I also feel like the drivers/plugins are currently BEYOND a tipping
>> >> point, and are in fact dragging down velocity of the core project in
>> >> many ways. I'm working on a proposal for Kilo where we move all
>> >> drivers/plugins out of the main Neutron tree and into a separate git
>> >> repository under the networking program. We have way too many drivers,
>> >> requiring way too man review cycles, for this to be a sustainable
>> >> model going forward. Since the main reason plugin/driver authors want
>> >> their code upstream is to be a part of the simultaneous release, and
>> >> thus be packaged by distributions, having a separate repository for
>> >> these will satisfy this requirement. I'm still working through the
>> >> details around reviews of this repository, etc.
>> >>
>> >> Also, I feel as if the level of passion on the mailing list has died
>> >> down a bit, so I thought I'd send something out to try and liven
>> >> things up a bit. It's been somewhat non-emotional here for a day or
>> >> so. :)
>> >>
>> >> Thanks,
>> >> Kyle
>> >>
>> >> On Thu, Aug 14, 2014 at 5:09 AM, Salvatore Orlando
>> >> <sorlando at nicira.com>
>> >> wrote:
>> >> > I think there will soon be a discussion regarding what the
>> >> > appropriate
>> >> > location for plugin and drivers should be.
>> >> > My personal feeling is that Neutron has simply reached the tipping
>> >> > point
>> >> > where the high number of drivers and plugins is causing unnecessary
>> >> > load
>> >> > for
>> >> > the core team and frustration for the community.
>> >> >
>> >> > There I would totally support Luke's initiative about maintaining an
>> >> > out-of-tree ML2 driver. On the other hand, a plugin/driver "diaspora"
>> >> > might
>> >> > also have negative consequences such as frequent breakages such as
>> >> > those
>> >> > Bob
>> >> > was mentioning or confusion for users which might need to end up
>> >> > fetching
>> >> > drivers from disparate sources.
>> >> >
>> >> > As mentioned during the last Neutron IRC meeting this is another
>> >> > "process"
>> >> > aspect which will be discussed soon, with the aim of defining a plan
>> >> > for:
>> >> > - drastically reduce the number of plugins and drivers which must be
>> >> > maintained in the main source tree
>> >> > - enhance control of plugin/driver maintainers over their own code
>> >> > - preserve the ability of doing CI checks on gerrit as we do today
>> >> > - raise the CI bar (maybe finally set the smoketest as a minimum
>> >> > requirement?)
>> >> >
>> >> > Regards,
>> >> > Salvatore
>> >> >
>> >> >
>> >> >
>> >> > On 14 August 2014 11:47, loy wolfe <loywolfe at gmail.com> wrote:
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Thu, Aug 14, 2014 at 4:22 PM, Mathieu Rohon
>> >> >> <mathieu.rohon at gmail.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> Hi,
>> >> >>>
>> >> >>> I would like to add that it would be harder for the community to
>> >> >>> help
>> >> >>> maintaining drivers.
>> >> >>> such a work [1] wouldn't have occured with an out of tree ODL
>> >> >>> driver.
>> >> >>
>> >> >>
>> >> >> +1.
>> >> >> It's better to move all MD for none built-in backend out of tree,
>> >> >> maintaining these drivers shouldn't be the responsibility of
>> >> >> community.
>> >> >> Not
>> >> >> only MD, but also plugin, agent should all obey this rule
>> >> >>
>> >> >>>
>> >> >>>
>> >> >>> [1] https://review.openstack.org/#/c/96459/
>> >> >>>
>> >> >>> On Wed, Aug 13, 2014 at 1:09 PM, Robert Kukura
>> >> >>> <kukura at noironetworks.com>
>> >> >>> wrote:
>> >> >>> > One thing to keep in mind is that the ML2 driver API does
>> >> >>> > sometimes
>> >> >>> > change,
>> >> >>> > requiring updates to drivers. Drivers that are in-tree get
>> >> >>> > updated
>> >> >>> > along
>> >> >>> > with the driver API change. Drivers that are out-of-tree must be
>> >> >>> > updated by
>> >> >>> > the owner.
>> >> >>> >
>> >> >>> > -Bob
>> >> >>> >
>> >> >>> >
>> >> >>> > On 8/13/14, 6:59 AM, ZZelle wrote:
>> >> >>> >
>> >> >>> > Hi,
>> >> >>> >
>> >> >>> >
>> >> >>> > The important thing to understand is how to integrate with
>> >> >>> > neutron
>> >> >>> > through
>> >> >>> > stevedore/entrypoints:
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> > https://github.com/dave-tucker/odl-neutron-drivers/blob/master/setup.cfg#L32-L34
>> >> >>> >
>> >> >>> >
>> >> >>> > Cedric
>> >> >>> >
>> >> >>> >
>> >> >>> > On Wed, Aug 13, 2014 at 12:17 PM, Dave Tucker
>> >> >>> > <dave at dtucker.co.uk>
>> >> >>> > wrote:
>> >> >>> >>
>> >> >>> >> I've been working on this for OpenDaylight
>> >> >>> >> https://github.com/dave-tucker/odl-neutron-drivers
>> >> >>> >>
>> >> >>> >> This seems to work for me (tested Devstack w/ML2) but YMMV.
>> >> >>> >>
>> >> >>> >> -- Dave
>> >> >>> >>
>> >> >>> >> _______________________________________________
>> >> >>> >> OpenStack-dev mailing list
>> >> >>> >> OpenStack-dev at lists.openstack.org
>> >> >>> >>
>> >> >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> > _______________________________________________
>> >> >>> > OpenStack-dev mailing list
>> >> >>> > OpenStack-dev at lists.openstack.org
>> >> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> > _______________________________________________
>> >> >>> > OpenStack-dev mailing list
>> >> >>> > OpenStack-dev at lists.openstack.org
>> >> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >>> >
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> OpenStack-dev mailing list
>> >> >>> OpenStack-dev at lists.openstack.org
>> >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >>
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> OpenStack-dev mailing list
>> >> >> OpenStack-dev at lists.openstack.org
>> >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > OpenStack-dev mailing list
>> >> > OpenStack-dev at lists.openstack.org
>> >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >
>> >>
>> >> _______________________________________________
>> >> OpenStack-dev mailing list
>> >> OpenStack-dev at lists.openstack.org
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> >
>> > --
>> > Kevin Benton
>> >
>> > _______________________________________________
>> > OpenStack-dev mailing list
>> > OpenStack-dev at lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Kevin Benton
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>