<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On 9 December 2014 at 17:32, Armando M. <span dir="ltr"><<a href="mailto:armamig@gmail.com" target="_blank">armamig@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On 9 December 2014 at 09:41, Salvatore Orlando <span dir="ltr"><<a href="mailto:sorlando@nicira.com" target="_blank">sorlando@nicira.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">I would like to chime into this discussion wearing my plugin developer hat.<div><br></div><div>
<p><span>We (the VMware team) have looked very carefully at the current proposal for splitting off drivers and plugins from the main source code tree. Therefore the concerns you've heard from Gary are not just ramblings but are the results of careful examination of this proposal.</span></p><p>While we agree with the final goal, the feeling is that for many plugin maintainers this process change might be too much for what can be accomplished in a single release cycle. </p></div></div></blockquote></span><div>We actually gave a lot more than a cycle:</div><div><br></div><div><a href="https://review.openstack.org/#/c/134680/16/specs/kilo/core-vendor-decomposition.rst" target="_blank">https://review.openstack.org/#/c/134680/16/specs/kilo/core-vendor-decomposition.rst</a> LINE 416<br></div><div><br></div><div>And in all honestly, I can only tell that getting this done by such an experienced team like the Neutron team @VMware shouldn't take that long.</div></div></div></div></blockquote><div><br></div><div>We are probably not experienced enough. We always love to learn new things.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>By the way, if Kyle can do it in his teeny tiny time that he has left after his PTL duties, then anyone can do it! :)</div><div><br></div><div><a href="https://review.openstack.org/#/c/140191/" target="_blank">https://review.openstack.org/#/c/140191/</a></div></div></div></div></blockquote><div><br></div><div>I think I should be able to use mv & git push as well - I think however there's a bit more than that to it.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><p>As a member of the drivers team, I am still very supportive of the split, I just want to make sure that it’s made in a sustainable way; I also understand that “sustainability” has been one of the requirements of the current proposal, and therefore we should all be on the same page on this aspect.</p><p>However, we did a simple exercise trying to assess the amount of work needed to achieve something which might be acceptable to satisfy the process. Without going into too many details, this requires efforts for:<br></p><p><span>- refactor the code to achieve a plugin module simple and thin enough to satisfy the requirements. Unfortunately a radical approach like the one in [1] with a reference to an external library is not pursuable for us</span></p><p>- maintaining code repositories outside of the neutron scope and the necessary infrastructure</p><p><span>- reinforcing our CI infrastructure, and improve our error detection and log analysis capabilities to improve reaction times upon failures triggered by upstream changes. As you know, even if the plugin interface is solid-ish, the dependency on the db base class increases the chances of upstream changes breaking 3rd party plugins.</span></p></div></div></blockquote><div><br></div></span><div>No-one is advocating for approach laid out in [1], but a lot of code can be moved elsewhere (like the nsxlib) without too much effort. Don't forget that not so long ago I was the maintainer of this plugin and the one who built the VMware NSX CI; I know very well what it takes to scope this effort, and I can support you in the process.</div></div></div></div></blockquote><div><br></div><div>Thanks for this clarification. I was sure that you guys were not advocating for a ninja-split thing, but I wanted just to be sure of that.</div><div>I'm also pretty sure our engineering team values your support. </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><p><span>The feedback from our engineering team is that satisfying the requirements of this new process might not be feasible in the Kilo timeframe, both for existing plugins and for new plugins and drivers that should be upstreamed (there are a few proposed on neutron-specs at the moment, which are all in -2 status considering the impending approval of the split out).</span></p></div></div></blockquote></span><div>No new plugins can and will be accepted if they do not adopt the proposed model, let's be very clear about this. </div></div></div></div></blockquote><div><br></div><div>This is also what I gathered from the proposal. It seems that you're however stating that there might be some flexibility in defining how much a plugin complies with the new model. I will need to go back to the drawing board with the rest of my team and see in which way this can work for us.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><p>The questions I would like to bring to the wider community are therefore the following:<br><span></span></p><p><span>1 - Is there a possibility of making a further concession on the current proposal, where maintainers are encouraged to experiment with the plugin split in Kilo, but will actually required to do it in the next release?</span></p></div></div></blockquote></span><div>This is exactly what the spec is proposing: get started now, and it does not matter if you don't finish in time. </div></div></div></div></blockquote><div><br></div><div>I think the deprecation note at line 416 still scares people off a bit. To me your word is enough, no change is needed. </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><p><span>2 - What could be considered as a acceptable as a new plugin? I understand that they would be accepted only as “thin integration modules”, which ideally should just be a pointer to code living somewhere else. I’m not questioning the validity of this approach, but it has been brought to my attention that this will actually be troubling for teams which have made an investment in the previous release cycles to upstream plugins following the “old” process</span></p></div></div></blockquote></span><div>You are not alone. Other efforts went through the same process [1, 2, 3]. Adjusting is a way of life. No-one is advocating for throwing away existing investment. This proposal actually promotes new and pre-existing investment.</div><div><br></div><div>[1] <a href="https://review.openstack.org/#/c/104452/" target="_blank">https://review.openstack.org/#/c/104452/</a><br></div><div>[2] <a href="https://review.openstack.org/#/c/103728/" target="_blank">https://review.openstack.org/#/c/103728/</a><br></div><div>[3] <a href="https://review.openstack.org/#/c/136091/" target="_blank">https://review.openstack.org/#/c/136091/</a> </div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><p><span>3 - Regarding the above discussion on "ML2 or not ML2". The point on co-gating is well taken. Eventually we'd like to remove this binding - because I believe the ML2 subteam would also like to have more freedom on their plugin. Do we already have an idea about how doing that without completely moving away from the db_base class approach?</span></p></div></div></blockquote></span><div>Sure, if you like to participate in the process, we can only welcome you! <br></div></div></div></div></blockquote><div><br></div><div>I actually asked you if you already had an idea... should I take that as a no? </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><p><span>Thanks for your attention and for reading through this</span></p><p><span></span>Salvatore</p><p>
</p><p><span>[1] <a href="http://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/vmware/plugin.py#n22" target="_blank"><span>http://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/vmware/plugin.py#n22</span></a></span></p></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 8 December 2014 at 21:51, Maru Newby <span dir="ltr"><<a href="mailto:marun@redhat.com" target="_blank">marun@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span><br>
On Dec 7, 2014, at 10:51 AM, Gary Kotton <<a href="mailto:gkotton@vmware.com" target="_blank">gkotton@vmware.com</a>> wrote:<br>
<br>
> Hi Kyle,<br>
> I am not missing the point. I understand the proposal. I just think that it has some shortcomings (unless I misunderstand, which will certainly not be the first time and most definitely not the last). The thinning out is to have a shim in place. I understand this and this will be the entry point for the plugin. I do not have a concern for this. My concern is that we are not doing this with the ML2 off the bat. That should lead by example as it is our reference architecture. Lets not kid anyone, but we are going to hit some problems with the decomposition. I would prefer that it be done with the default implementation. Why?<br>
<br>
</span>The proposal is to move vendor-specific logic out of the tree to increase vendor control over such code while decreasing load on reviewers. ML2 doesn’t contain vendor-specific logic - that’s the province of ML2 drivers - so it is not a good target for the proposed decomposition by itself.<br>
<br>
<br>
> • Cause we will fix them quicker as it is something that prevent Neutron from moving forwards<br>
<span>> • We will just need to fix in one place first and not in N (where N is the vendor plugins)<br>
> • This is a community effort – so we will have a lot more eyes on it<br>
> • It will provide a reference architecture for all new plugins that want to be added to the tree<br>
> • It will provide a working example for plugin that are already in tree and are to be replaced by the shim<br>
> If we really want to do this, we can say freeze all development (which is just approvals for patches) for a few days so that we will can just focus on this. I stated what I think should be the process on the review. For those who do not feel like finding the link:<br>
> • Create a stack forge project for ML2<br>
> • Create the shim in Neutron<br>
> • Update devstack for the to use the two repos and the shim<br>
> When #3 is up and running we switch for that to be the gate. Then we start a stopwatch on all other plugins.<br>
<br>
</span>As was pointed out on the spec (see Miguel’s comment on r15), the ML2 plugin and the OVS mechanism driver need to remain in the main Neutron repo for now. Neutron gates on ML2+OVS and landing a breaking change in the Neutron repo along with its corresponding fix to a separate ML2 repo would be all but impossible under the current integrated gating scheme. Plugins/drivers that do not gate Neutron have no such constraint.<br>
<span><font color="#888888"><br>
<br>
Maru<br>
</font></span><div><div><br>
<br>
> Sure, I’ll catch you on IRC tomorrow. I guess that you guys will bash out the details at the meetup. Sadly I will not be able to attend – so you will have to delay on the tar and feathers.<br>
> Thanks<br>
> Gary<br>
><br>
><br>
> From: "<a href="mailto:mestery@mestery.com" target="_blank">mestery@mestery.com</a>" <<a href="mailto:mestery@mestery.com" target="_blank">mestery@mestery.com</a>><br>
> Reply-To: OpenStack List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
> Date: Sunday, December 7, 2014 at 7:19 PM<br>
> To: OpenStack List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
> Cc: "<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>" <<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>><br>
> Subject: Re: [openstack-dev] [Neutron] Core/Vendor code decomposition<br>
><br>
> Gary, you are still miss the point of this proposal. Please see my comments in review. We are not forcing things out of tree, we are thinning them. The text you quoted in the review makes that clear. We will look at further decomposing ML2 post Kilo, but we have to be realistic with what we can accomplish during Kilo.<br>
><br>
> Find me on IRC Monday morning and we can discuss further if you still have questions and concerns.<br>
><br>
> Thanks!<br>
> Kyle<br>
><br>
> On Sun, Dec 7, 2014 at 2:08 AM, Gary Kotton <<a href="mailto:gkotton@vmware.com" target="_blank">gkotton@vmware.com</a>> wrote:<br>
>> Hi,<br>
>> I have raised my concerns on the proposal. I think that all plugins should be treated on an equal footing. My main concern is having the ML2 plugin in tree whilst the others will be moved out of tree will be problematic. I think that the model will be complete if the ML2 was also out of tree. This will help crystalize the idea and make sure that the model works correctly.<br>
>> Thanks<br>
>> Gary<br>
>><br>
>> From: "Armando M." <<a href="mailto:armamig@gmail.com" target="_blank">armamig@gmail.com</a>><br>
>> Reply-To: OpenStack List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
>> Date: Saturday, December 6, 2014 at 1:04 AM<br>
>> To: OpenStack List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>>, "<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>" <<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>><br>
>> Subject: [openstack-dev] [Neutron] Core/Vendor code decomposition<br>
>><br>
>> Hi folks,<br>
>><br>
>> For a few weeks now the Neutron team has worked tirelessly on [1].<br>
>><br>
>> This initiative stems from the fact that as the project matures, evolution of processes and contribution guidelines need to evolve with it. This is to ensure that the project can keep on thriving in order to meet the needs of an ever growing community.<br>
>><br>
>> The effort of documenting intentions, and fleshing out the various details of the proposal is about to reach an end, and we'll soon kick the tires to put the proposal into practice. Since the spec has grown pretty big, I'll try to capture the tl;dr below.<br>
>><br>
>> If you have any comment please do not hesitate to raise them here and/or reach out to us.<br>
>><br>
>> tl;dr >>><br>
>><br>
>> From the Kilo release, we'll initiate a set of steps to change the following areas:<br>
>> • Code structure: every plugin or driver that exists or wants to exist as part of Neutron project is decomposed in an slim vendor integration (which lives in the Neutron repo), plus a bulkier vendor library (which lives in an independent publicly available repo);<br>
>> • Contribution process: this extends to the following aspects:<br>
>> • Design and Development: the process is largely unchanged for the part that pertains the vendor integration; the maintainer team is fully auto governed for the design and development of the vendor library;<br>
>> • Testing and Continuous Integration: maintainers will be required to support their vendor integration with 3rd CI testing; the requirements for 3rd CI testing are largely unchanged;<br>
>> • Defect management: the process is largely unchanged, issues affecting the vendor library can be tracked with whichever tool/process the maintainer see fit. In cases where vendor library fixes need to be reflected in the vendor integration, the usual OpenStack defect management apply.<br>
>> • Documentation: there will be some changes to the way plugins and drivers are documented with the intention of promoting discoverability of the integrated solutions.<br>
>> • Adoption and transition plan: we strongly advise maintainers to stay abreast of the developments of this effort, as their code, their CI, etc will be affected. The core team will provide guidelines and support throughout this cycle the ensure a smooth transition.<br>
>> To learn more, please refer to [1].<br>
>><br>
>> Many thanks,<br>
>> Armando<br>
>><br>
>> [1] <a href="https://review.openstack.org/#/c/134680" target="_blank">https://review.openstack.org/#/c/134680</a><br>
>><br>
>> _______________________________________________<br>
>> OpenStack-dev mailing list<br>
>> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>