Open Stack

On 9 December 2014 at 13:59, Ivar Lazzaro <ivarlazzaro at gmail.com> wrote:

> I agree with Salvatore that the split is not an easy thing to achieve for
> vendors, and I would like to bring up my case to see if there are ways to
> make this at least a bit simpler.
>
> At some point I had the need to backport vendor code from Juno to Icehouse
> (see first attempt here [0]). That in [0] was some weird approach that put
> unnecessary burden on infra, neutron cores and even packagers, so I decided
> to move to a more decoupled approach that was basically completely
> splitting my code from Neutron. You can find the result here [1].
> The focal points of this approach are:
>
> * **all** the vendor code is removed;
> * Neutron is used as a dependency, pulled directly from github for UTs
> (see test-requirements [2]) and explicitly required when installing the
> plugin;
> * The Database and Schema is the same as Neutron's;
> * A migration script exists for this driver, which uses a different (and
> unique) version_table (see env.py [3]);
> * Entry points are properly used in setup.cfg [4] in order to provide
> migration scripts and Driver/Plugin shortcuts for Neutron;
> * UTs are run by including Neutron in the venv [2].
> * The boilerplate is taken care by cookiecutter [5].
>

> The advantage of the above approach, is that it's very simple to pull off
> (only thing you need is cookiecutter, a repo, and then you can just
> replicate the same tree structure that existed in Neutron for your own
> vendor code). Also it has the advantage to remove all the vendor code from
> Neutron (did I say that already?). As far as the CI is concerned, it just
> needs to "learn" how to install the new plugin, which will require Neutron
> to be pre-existent.
>
> The typical installation workflow would be:
> - Install Neutron normally;
> - pull from pypi the vendor driver;
> - run the vendor db migration script;
> - Do everything else (configuration and execution) just like it was done
> before.
>
> Note that this same satellite approach is used by GBP (I know this is a
> bad word that once brought hundreds of ML replies, but that's just an
> example :) ) for the Juno timeframe [6]. This shows that the very same
> thing can be easily done for services.
>

Okay, so if I understand you correctly, you're saying that was easier for
you to go entirely out of tree, and that you have done so already. Okay,
good for you, problem solved!

One point that should be clear here is that, if someone is completely
comfortable with being entirely out of tree, and he/she has done so already
(I know of a few other examples besides the apic driver), then this
proposal does not apply to them.

They are way ahead of us, and kudos to them!

As far you're concerned, Ivar, if you want to promote this model for new
plugins/drivers contributions, by all means, I encourage you to document
this in a blog or the wiki and disseminate your findings so that people can
adopt your model if they wanted to.


>
> As far as ML2 is concerned, I think we should split it as well in order to
> treat all the plugins equally, but with the following caveats:
>
> * ML2 will be in a openstack repo under the networking program (kind of
> obvious);
> * The drivers can decide wether to stay in tree with ML2 or not (for a
> better community effort, but they will definitively evolve slower);
> * Don't care about the governance, Neutron will be in charge of this repo
> and will have the ability to promote whoever they want when needed.
>

We discussed this point before, and the decision is that we don't want to
be prescriptive about this. If people interested in ML2 drivers want to
stay close to each other or not, we should not mandate one or the other.


> As far as cogating is concerned, I think that using the above approach the
> breaking will exist just as long as the infra job understands how to
> install the ML2 driver from it's own repo. I don't see it as a big issue,
> But maybe it's just me, and my fabulous world where stuff works for no good
> reason. We could at least ask the infra team to understand it it's feasible.
>

Well, you live in a nice wonderland, I envy you! This co-gating decision is
the result of a discussion with infra folks.


> Moreover, this is a work that we may need to do anyway! So it's better to
> just start it now thus creating an example for all the vendors that have to
> go through the split (back on Gary's point).
>

Plenty of examples to go by, as mentioned time and time again.


>
> Appreciate your feedback,
> Ivar.
>
> [0] https://review.openstack.org/#/c/123596/
> [1] https://github.com/noironetworks/apic-ml2-driver/tree/icehouse
> [2]
> https://github.com/noironetworks/apic-ml2-driver/blob/icehouse/test-requirements.txt
> [3]
> https://github.com/noironetworks/apic-ml2-driver/blob/icehouse/apic_ml2/neutron/db/migration/alembic_migrations/env.py
> [4]
> https://github.com/noironetworks/apic-ml2-driver/blob/icehouse/setup.cfg
> [5] https://github.com/openstack-dev/cookiecutter
> [6] https://github.com/stackforge/group-based-policy
>
> On Tue, Dec 9, 2014 at 9:53 AM, Salvatore Orlando <sorlando at nicira.com>
> wrote:
>
>>
>>
>> On 9 December 2014 at 17:32, Armando M. <armamig at gmail.com> wrote:
>>
>>>
>>>
>>> On 9 December 2014 at 09:41, Salvatore Orlando <sorlando at nicira.com>
>>> wrote:
>>>
>>>> I would like to chime into this discussion wearing my plugin developer
>>>> hat.
>>>>
>>>> We (the VMware team) have looked very carefully at the current proposal
>>>> for splitting off drivers and plugins from the main source code tree.
>>>> Therefore the concerns you've heard from Gary are not just ramblings but
>>>> are the results of careful examination of this proposal.
>>>>
>>>> While we agree with the final goal, the feeling is that for many plugin
>>>> maintainers this process change might be too much for what can be
>>>> accomplished in a single release cycle.
>>>>
>>> We actually gave a lot more than a cycle:
>>>
>>>
>>> https://review.openstack.org/#/c/134680/16/specs/kilo/core-vendor-decomposition.rst
>>> LINE 416
>>>
>>> And in all honestly, I can only tell that getting this done by such an
>>> experienced team like the Neutron team @VMware shouldn't take that long.
>>>
>>
>> We are probably not experienced enough. We always love to learn new
>> things.
>>
>>
>>>
>>> By the way, if Kyle can do it in his teeny tiny time that he has left
>>> after his PTL duties, then anyone can do it! :)
>>>
>>> https://review.openstack.org/#/c/140191/
>>>
>>
>> I think I should be able to use mv & git push as well - I think however
>> there's a bit more than that to it.
>>
>>
>>>
>>> As a member of the drivers team, I am still very supportive of the
>>>> split, I just want to make sure that it’s made in a sustainable way; I also
>>>> understand that “sustainability” has been one of the requirements of the
>>>> current proposal, and therefore we should all be on the same page on this
>>>> aspect.
>>>>
>>>> However, we did a simple exercise trying to assess the amount of work
>>>> needed to achieve something which might be acceptable to satisfy the
>>>> process. Without going into too many details, this requires efforts for:
>>>>
>>>> - refactor the code to achieve a plugin module simple and thin enough
>>>> to satisfy the requirements. Unfortunately a radical approach like the one
>>>> in [1] with a reference to an external library is not pursuable for us
>>>>
>>>> - maintaining code repositories outside of the neutron scope and the
>>>> necessary infrastructure
>>>>
>>>> - reinforcing our CI infrastructure, and improve our error detection
>>>> and log analysis capabilities to improve reaction times upon failures
>>>> triggered by upstream changes. As you know, even if the plugin interface is
>>>> solid-ish, the dependency on the db base class increases the chances of
>>>> upstream changes breaking 3rd party plugins.
>>>>
>>>
>>> No-one is advocating for approach laid out in [1], but a lot of code can
>>> be moved elsewhere (like the nsxlib) without too much effort. Don't forget
>>> that not so long ago I was the maintainer of this plugin and the one who
>>> built the VMware NSX CI; I know very well what it takes to scope this
>>> effort, and I can support you in the process.
>>>
>>
>> Thanks for this clarification. I was sure that you guys were not
>> advocating for a ninja-split thing, but I wanted just to be sure of that.
>> I'm also pretty sure our engineering team values your support.
>>
>>> The feedback from our engineering team is that satisfying the
>>>> requirements of this new process might not be feasible in the Kilo
>>>> timeframe, both for existing plugins and for new plugins and drivers that
>>>> should be upstreamed (there are a few proposed on neutron-specs at the
>>>> moment, which are all in -2 status considering the impending approval of
>>>> the split out).
>>>>
>>> No new plugins can and will be accepted if they do not adopt the
>>> proposed model, let's be very clear about this.
>>>
>>
>> This is also what I gathered from the proposal. It seems that you're
>> however stating that there might be some flexibility in defining how much a
>> plugin complies with the new model. I will need to go back to the drawing
>> board with the rest of my team and see in which way this can work for us.
>>
>>
>>> The questions I would like to bring to the wider community are therefore
>>>> the following:
>>>>
>>>> 1 - Is there a possibility of making a further concession on the
>>>> current proposal, where maintainers are encouraged to experiment with the
>>>> plugin split in Kilo, but will actually required to do it in the next
>>>> release?
>>>>
>>> This is exactly what the spec is proposing: get started now, and it does
>>> not matter if you don't finish in time.
>>>
>>
>> I think the deprecation note at line 416 still scares people off a bit.
>> To me your word is enough, no change is needed.
>>
>>> 2 - What could be considered as a acceptable as a new plugin? I
>>>> understand that they would be accepted only as “thin integration modules”,
>>>> which ideally should just be a pointer to code living somewhere else. I’m
>>>> not questioning the validity of this approach, but it has been brought to
>>>> my attention that this will actually be troubling for teams which have made
>>>> an investment in the previous release cycles to upstream plugins following
>>>> the “old” process
>>>>
>>> You are not alone. Other efforts went through the same process [1, 2,
>>> 3]. Adjusting is a way of life. No-one is advocating for throwing away
>>> existing investment. This proposal actually promotes new and pre-existing
>>> investment.
>>>
>>> [1] https://review.openstack.org/#/c/104452/
>>> [2] https://review.openstack.org/#/c/103728/
>>> [3] https://review.openstack.org/#/c/136091/
>>>
>> 3 - Regarding the above discussion on "ML2 or not ML2". The point on
>>>> co-gating is well taken. Eventually we'd like to remove this binding -
>>>> because I believe the ML2 subteam would also like to have more freedom on
>>>> their plugin. Do we already have an idea about how doing that without
>>>> completely moving away from the db_base class approach?
>>>>
>>> Sure, if you like to participate in the process, we can only welcome
>>> you!
>>>
>>
>> I actually asked you if you already had an idea... should I take that as
>> a no?
>>
>>> Thanks for your attention and for reading through this
>>>>
>>>> Salvatore
>>>>
>>>> [1]
>>>> http://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/vmware/plugin.py#n22
>>>>
>>>> On 8 December 2014 at 21:51, Maru Newby <marun at redhat.com> wrote:
>>>>
>>>>>
>>>>> On Dec 7, 2014, at 10:51 AM, Gary Kotton <gkotton at vmware.com> wrote:
>>>>>
>>>>> > Hi Kyle,
>>>>> > I am not missing the point. I understand the proposal. I just think
>>>>> that it has some shortcomings (unless I misunderstand, which will certainly
>>>>> not be the first time and most definitely not the last). The thinning out
>>>>> is to have a shim in place. I understand this and this will be the entry
>>>>> point for the plugin. I do not have a concern for this. My concern is that
>>>>> we are not doing this with the ML2 off the bat. That should lead by example
>>>>> as it is our reference architecture. Lets not kid anyone, but we are going
>>>>> to hit some problems with the decomposition. I would prefer that it be done
>>>>> with the default implementation. Why?
>>>>>
>>>>> The proposal is to move vendor-specific logic out of the tree to
>>>>> increase vendor control over such code while decreasing load on reviewers.
>>>>> ML2 doesn’t contain vendor-specific logic - that’s the province of ML2
>>>>> drivers - so it is not a good target for the proposed decomposition by
>>>>> itself.
>>>>>
>>>>>
>>>>> >       • Cause we will fix them quicker as it is something that
>>>>> prevent Neutron from moving forwards
>>>>> >       • We will just need to fix in one place first and not in N
>>>>> (where N is the vendor plugins)
>>>>> >       • This is a community effort – so we will have a lot more eyes
>>>>> on it
>>>>> >       • It will provide a reference architecture for all new plugins
>>>>> that want to be added to the tree
>>>>> >       • It will provide a working example for plugin that are
>>>>> already in tree and are to be replaced by the shim
>>>>> > If we really want to do this, we can say freeze all development
>>>>> (which is just approvals for patches) for a few days so that we will can
>>>>> just focus on this. I stated what I think should be the process on the
>>>>> review. For those who do not feel like finding the link:
>>>>> >       • Create a stack forge project for ML2
>>>>> >       • Create the shim in Neutron
>>>>> >       • Update devstack for the to use the two repos and the shim
>>>>> > When #3 is up and running we switch for that to be the gate. Then we
>>>>> start a stopwatch on all other plugins.
>>>>>
>>>>> As was pointed out on the spec (see Miguel’s comment on r15), the ML2
>>>>> plugin and the OVS mechanism driver need to remain in the main Neutron repo
>>>>> for now.  Neutron gates on ML2+OVS and landing a breaking change in the
>>>>> Neutron repo along with its corresponding fix to a separate ML2 repo would
>>>>> be all but impossible under the current integrated gating scheme.
>>>>> Plugins/drivers that do not gate Neutron have no such constraint.
>>>>>
>>>>>
>>>>> Maru
>>>>>
>>>>>
>>>>> > Sure, I’ll catch you on IRC tomorrow. I guess that you guys will
>>>>> bash out the details at the meetup. Sadly I will not be able to attend – so
>>>>> you will have to delay on the tar and feathers.
>>>>> > Thanks
>>>>> > Gary
>>>>> >
>>>>> >
>>>>> > From: "mestery at mestery.com" <mestery at mestery.com>
>>>>> > Reply-To: OpenStack List <openstack-dev at lists.openstack.org>
>>>>> > Date: Sunday, December 7, 2014 at 7:19 PM
>>>>> > To: OpenStack List <openstack-dev at lists.openstack.org>
>>>>> > Cc: "openstack at lists.openstack.org" <openstack at lists.openstack.org>
>>>>> > Subject: Re: [openstack-dev] [Neutron] Core/Vendor code decomposition
>>>>> >
>>>>> > Gary, you are still miss the point of this proposal. Please see my
>>>>> comments in review. We are not forcing things out of tree, we are thinning
>>>>> them. The text you quoted in the review makes that clear. We will look at
>>>>> further decomposing ML2 post Kilo, but we have to be realistic with what we
>>>>> can accomplish during Kilo.
>>>>> >
>>>>> > Find me on IRC Monday morning and we can discuss further if you
>>>>> still have questions and concerns.
>>>>> >
>>>>> > Thanks!
>>>>> > Kyle
>>>>> >
>>>>> > On Sun, Dec 7, 2014 at 2:08 AM, Gary Kotton <gkotton at vmware.com>
>>>>> wrote:
>>>>> >> Hi,
>>>>> >> I have raised my concerns on the proposal. I think that all plugins
>>>>> should be treated on an equal footing. My main concern is having the ML2
>>>>> plugin in tree whilst the others will be moved out of tree will be
>>>>> problematic. I think that the model will be complete if the ML2 was also
>>>>> out of tree. This will help crystalize the idea and make sure that the
>>>>> model works correctly.
>>>>> >> Thanks
>>>>> >> Gary
>>>>> >>
>>>>> >> From: "Armando M." <armamig at gmail.com>
>>>>> >> Reply-To: OpenStack List <openstack-dev at lists.openstack.org>
>>>>> >> Date: Saturday, December 6, 2014 at 1:04 AM
>>>>> >> To: OpenStack List <openstack-dev at lists.openstack.org>, "
>>>>> openstack at lists.openstack.org" <openstack at lists.openstack.org>
>>>>> >> Subject: [openstack-dev] [Neutron] Core/Vendor code decomposition
>>>>> >>
>>>>> >> Hi folks,
>>>>> >>
>>>>> >> For a few weeks now the Neutron team has worked tirelessly on [1].
>>>>> >>
>>>>> >> This initiative stems from the fact that as the project matures,
>>>>> evolution of processes and contribution guidelines need to evolve with it.
>>>>> This is to ensure that the project can keep on thriving in order to meet
>>>>> the needs of an ever growing community.
>>>>> >>
>>>>> >> The effort of documenting intentions, and fleshing out the various
>>>>> details of the proposal is about to reach an end, and we'll soon kick the
>>>>> tires to put the proposal into practice. Since the spec has grown pretty
>>>>> big, I'll try to capture the tl;dr below.
>>>>> >>
>>>>> >> If you have any comment please do not hesitate to raise them here
>>>>> and/or reach out to us.
>>>>> >>
>>>>> >> tl;dr >>>
>>>>> >>
>>>>> >> From the Kilo release, we'll initiate a set of steps to change the
>>>>> following areas:
>>>>> >>      • Code structure: every plugin or driver that exists or wants
>>>>> to exist as part of Neutron project is decomposed in an slim vendor
>>>>> integration (which lives in the Neutron repo), plus a bulkier vendor
>>>>> library (which lives in an independent publicly available repo);
>>>>> >>      • Contribution process: this extends to the following aspects:
>>>>> >>              • Design and Development: the process is largely
>>>>> unchanged for the part that pertains the vendor integration; the maintainer
>>>>> team is fully auto governed for the design and development of the vendor
>>>>> library;
>>>>> >>              • Testing and Continuous Integration: maintainers will
>>>>> be required to support their vendor integration with 3rd CI testing; the
>>>>> requirements for 3rd CI testing are largely unchanged;
>>>>> >>              • Defect management: the process is largely unchanged,
>>>>> issues affecting the vendor library can be tracked with whichever
>>>>> tool/process the maintainer see fit. In cases where vendor library fixes
>>>>> need to be reflected in the vendor integration, the usual OpenStack defect
>>>>> management apply.
>>>>> >>              • Documentation: there will be some changes to the way
>>>>> plugins and drivers are documented with the intention of promoting
>>>>> discoverability of the integrated solutions.
>>>>> >>      • Adoption and transition plan: we strongly advise maintainers
>>>>> to stay abreast of the developments of this effort, as their code, their
>>>>> CI, etc will be affected. The core team will provide guidelines and support
>>>>> throughout this cycle the ensure a smooth transition.
>>>>> >> To learn more, please refer to [1].
>>>>> >>
>>>>> >> Many thanks,
>>>>> >> Armando
>>>>> >>
>>>>> >> [1] https://review.openstack.org/#/c/134680
>>>>> >>
>>>>> >> _______________________________________________
>>>>> >> OpenStack-dev mailing list
>>>>> >> OpenStack-dev at lists.openstack.org
>>>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> >>
>>>>> >
>>>>> > _______________________________________________
>>>>> > OpenStack-dev mailing list
>>>>> > OpenStack-dev at lists.openstack.org
>>>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141209/ea86fe90/attachment.html>