<div dir="ltr">Yes I'm considering writing it as a plugin for Fuel, but did not have time yet to do so. I miss writing code so hopefully I can make some time for it :P</div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 24, 2015 at 3:13 PM, Mike Scherbakov <span dir="ltr"><<a href="mailto:mscherbakov@mirantis.com" target="_blank">mscherbakov@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Sam,<div>very well & easy to read blog post. Did you consider trying out Fuel plugable framework for enabling SRIOV? </div><div><br></div><div>Just curios if you considered it and saw any gaps, I just want to know about those.</div><div><br></div><div>Thank you,</div></div><div class="HOEnZb"><div class="h5"><br><div class="gmail_quote"><div dir="ltr">On Mon, Jul 20, 2015 at 7:05 AM Sam Stoelinga <<a href="mailto:sammiestoel@gmail.com" target="_blank">sammiestoel@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I think it's a lot clearer already, I also made a minor modification to SRIOV agent part. Next to that I will see if I can add the content to OpenStack networking manual under advanced configuration. Have filed a bug against openstack-manuals here: <a href="https://bugs.launchpad.net/openstack-manuals/+bug/1476242" target="_blank">https://bugs.launchpad.net/openstack-manuals/+bug/1476242</a> will move forward if bug get's consensus that content is ok to be added to networking guide.<br></div></div><div dir="ltr"><div><br></div><div>Sam</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 20, 2015 at 6:56 PM, Moshe Levi <span dir="ltr"><<a href="mailto:moshele@mellanox.com" target="_blank">moshele@mellanox.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
I have update the SR-IOV wiki <a href="https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking</a><br>
Let me know if it clearer now.<br>
<br>
Thanks,<br>
Moshe Levi.<br>
<span><br>
> -----Original Message-----<br>
> From: Andreas Scheuring [mailto:<a href="mailto:scheuran@linux.vnet.ibm.com" target="_blank">scheuran@linux.vnet.ibm.com</a>]<br>
</span><div><div>> Sent: Monday, July 20, 2015 12:34 PM<br>
> To: Sam Stoelinga<br>
> Cc: Moshe Levi; <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
> Subject: Re: [Openstack] [Neutron][SRIOV][docs] Enabling SRIOV on<br>
> OpenStack Juno step-by-step guide<br>
><br>
> That's right.<br>
><br>
> But the firewall_driver = NoopFirewallDriver is an agent configuration, so if<br>
> you would have the sriov-agent configured with this driver, you still could<br>
> configure an ovs agent with another driver I guess. Does that make sense?<br>
> If so, maybe we could add this information to the wiki as well!<br>
><br>
> Thanks!<br>
><br>
> Andreas<br>
><br>
><br>
><br>
> On Mo, 2015-07-20 at 16:58 +0800, Sam Stoelinga wrote:<br>
> > I want to add that I'm not using the NoopFirewall as I'm using<br>
> > agent_required = False. So all instances that are not using SRIOV can<br>
> > still use security groups like normal. Instances that are using SRIOV<br>
> > won't have security groups applied though.<br>
> ><br>
> > On Mon, Jul 20, 2015 at 3:21 PM, Moshe Levi <<a href="mailto:moshele@mellanox.com" target="_blank">moshele@mellanox.com</a>><br>
> > wrote:<br>
> ><br>
> ><br>
> > > -----Original Message-----<br>
> > > From: Andreas Scheuring [mailto:<a href="mailto:scheuran@linux.vnet.ibm.com" target="_blank">scheuran@linux.vnet.ibm.com</a>]<br>
> > > Sent: Monday, July 20, 2015 10:04 AM<br>
> > > To: Moshe Levi<br>
> > > Cc: Sam Stoelinga; <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
> > > Subject: Re: [Openstack] [Neutron][SRIOV][docs] Enabling<br>
> > SRIOV on<br>
> > > OpenStack Juno step-by-step guide<br>
> > ><br>
> > > +1 for updating the wiki<br>
> > > +1 for adding a section to the docs<br>
> > ><br>
> > ><br>
> > > Moshe,<br>
> > > what about the firewall support Sam mentioned? I assume<br>
> > fwaas is<br>
> > > supported, as it runs on the network node which uses ovs,<br>
> > but Security<br>
> > > Groups are not working as you're using the<br>
> > NoopFirewallDriver, right?<br>
> > Yes that is correct. I will update that as well in the wiki.<br>
> ><br>
> > > Or is there another FW driver that could be used?<br>
> > ><br>
> > > Thanks<br>
> > ><br>
> > ><br>
> > ><br>
> > > On So, 2015-07-19 at 08:12 +0000, Moshe Levi wrote:<br>
> > > > See my comments inline<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > From: Sam Stoelinga [mailto:<a href="mailto:sammiestoel@gmail.com" target="_blank">sammiestoel@gmail.com</a>]<br>
> > > > Sent: Sunday, July 19, 2015 10:37 AM<br>
> > > > To: Moshe Levi<br>
> > > > Cc: <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
> > > > Subject: Re: [Openstack] [Neutron][SRIOV][docs] Enabling<br>
> > SRIOV on<br>
> > > > OpenStack Juno step-by-step guide<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > I think it was not fair to say it's not up to date. It<br>
> > seems it's up<br>
> > > > to date, but current downsides of existing OpenStack wikis<br>
> > on SRIOV<br>
> > > > are missing info, many different Wikis and hard to consume<br>
> > the info:<br>
> > > ><br>
> > > ><br>
> > > > 1.<br>
> > <a href="https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking</a><br>
> > > ><br>
> > > ><br>
> > > > 2. <a href="https://wiki.openstack.org/wiki/Nova-neutron-sriov" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/Nova-neutron-sriov</a><br>
> > > ><br>
> > > ><br>
> > > > 3.<br>
> > <a href="https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support</a><br>
> > > ><br>
> > > ><br>
> > > > 4. <a href="https://wiki.openstack.org/wiki/Pci_passthrough" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/Pci_passthrough</a><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > [ML] – I think the only wiki users should be using is<br>
> > > ><br>
> > <a href="https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking</a><br>
> > > > [2],[3] are the blueprints and [4] is PCI-Pass-through<br>
> > without SR-IOV<br>
> > > > and it refer to [1] which is good.<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > Things that I noticed that were missing / could be better:<br>
> > > ><br>
> > > ><br>
> > > > 1. Adding PCIDeviceFilter to nova-scheduler<br>
> > > ><br>
> > > > [ML] I think you mean the PciPassthroughFilter and you are<br>
> > correct indeed<br>
> > > it is missing. I will update the wiki.<br>
> > > > 2. How to enable VFs on compute nodes<br>
> > > ><br>
> > > > [ML] – This vendor specific but we can add links to<br>
> > vendor wiki page<br>
> > > > on how to configure their NIC to support SR-IOV<br>
> > > ><br>
> > > ><br>
> > > > 3. This wiki:<br>
> > <a href="https://wiki.openstack.org/wiki/Nova-neutron-sriov" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/Nova-neutron-sriov</a><br>
> > > > contains incorrect information.<br>
> > > ><br>
> > > ><br>
> > > > Current wrong: "neutron port-create <net-uuid-from-step-1><br>
> > --name<br>
> > > > sriov_port --vnic-type direct "<br>
> > > ><br>
> > > ><br>
> > > > should be "neutron port-create <net-uuid-from-step-1><br>
> > --name<br>
> > > > sriov_port --binding:vnic_type direct"<br>
> > > ><br>
> > > > [ML] – this is blueprint but I will check if I can update<br>
> > it.<br>
> > > ><br>
> > > ><br>
> > > > 3. Make it more clear that agent_required = False is<br>
> > totally fine<br>
> > > > and may be better. From what I read you have to disable<br>
> > the firewall<br>
> > > > functionality if you enable sriov-agent? Not sure if that<br>
> > > > understanding is correct.<br>
> > > ><br>
> > > > [ML] – I agree this is totally need clarification. The<br>
> > > > agent_required=False is used when you have Intel NIC that<br>
> > doesn’t<br>
> > > > support admin up/down change. SR-IOV NIC that support<br>
> > admin up/down<br>
> > > > change should be configured with agent_required =True.<br>
> > > ><br>
> > > > I will update the wiki explaining that flag. (by the way<br>
> > we hope to<br>
> > > > change it in liberty and deprecate the agent_required<br>
> > flag)<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > I would prefer this information to have release bound<br>
> > documentation in<br>
> > > > for example the Networking<br>
> > > > Guide: <a href="http://docs.openstack.org/networking-guide/" rel="noreferrer" target="_blank">http://docs.openstack.org/networking-guide/</a> or the<br>
> > Cloud<br>
> > > > Administrator<br>
> > > > Guide:<br>
> > <a href="http://docs.openstack.org/admin-guide-cloud/content/" rel="noreferrer" target="_blank">http://docs.openstack.org/admin-guide-cloud/content/</a><br>
> > > ><br>
> > > > [ML] – Ok, I guess this is required change in<br>
> > > > openstack/openstack-manuals repository. Can you help and<br>
> > adding<br>
> > > > documentations there? Just put me as review. If not I<br>
> > will try do it<br>
> > > > myself or find someone in Mellanox.<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > I believe that using the pci sys interface is vendor<br>
> > compatible. Would<br>
> > > > be great if you could confirm. Could you try testing $<br>
> > echo '7'<br>
> > > > > /sys/class/net/eth3/device/sriov_numvfs on a mellanox<br>
> > card?<br>
> > > ><br>
> > > ><br>
> > > > This way we don't have to write vendor specific docs on<br>
> > enabling VFs<br>
> > > > :)<br>
> > > ><br>
> > > ><br>
> > > > When using modprobe ixgbe max_vfs=7 it tells you that<br>
> > using max_vfs is<br>
> > > > deprecated and that the pci sys interface should be used.<br>
> > That's how I<br>
> > > > found out about this.<br>
> > > ><br>
> > > > [ML] – unfortunately it is not generic in Melllanox you<br>
> > need to<br>
> > > > configure number of VFs and number of probes (also Single<br>
> > Port or<br>
> > > > Duel Port ) it is more complicateL<br>
> > > ><br>
> > > > see <a href="https://community.mellanox.com/docs/DOC-1484" rel="noreferrer" target="_blank">https://community.mellanox.com/docs/DOC-1484</a><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > On Sun, Jul 19, 2015 at 2:44 PM, Moshe Levi<br>
> > <<a href="mailto:moshele@mellanox.com" target="_blank">moshele@mellanox.com</a>><br>
> > > > wrote:<br>
> > > ><br>
> > > > Hi Sam,<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > Can you explain why you think that the<br>
> > > ><br>
> > <a href="https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking</a><br>
> > > is out of date?<br>
> > > ><br>
> > > > Moreover you blog explain how to configure SR-IOV<br>
> > on Intel<br>
> > > > NIC, but keep in mind Neutron SRIOV is generic and<br>
> > can support<br>
> > > > any other vendors such as Mellanox.<br>
> > > ><br>
> > > > Maybe will should add links to how to configure<br>
> > SR-IOV NIC for<br>
> > > > several Vendors. We can start with Mellanox and<br>
> > Intel NIC.<br>
> > > > What do you think?<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > From: Sam Stoelinga [mailto:<a href="mailto:sammiestoel@gmail.com" target="_blank">sammiestoel@gmail.com</a>]<br>
> > > > Sent: Saturday, July 18, 2015 5:55 PM<br>
> > > > To: <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
> > > > Subject: [Openstack] [Neutron][SRIOV][docs]<br>
> > Enabling SRIOV on<br>
> > > > OpenStack Juno step-by-step guide<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > Hi networking gurus,<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > While it may be easy for many of you to enable<br>
> > Neutron SRIOV<br>
> > > > on OpenStack it wasn't a smooth ride for me. I<br>
> > documented<br>
> > > > exactly which steps were required to enable SRIOV<br>
> > on OpenStack<br>
> > > > on my<br>
> > > > blog:<br>
> > > ><br>
> > <a href="http://samos-it.com/posts/sriov-openstack-juno-fuel-6-1.html" rel="noreferrer" target="_blank">http://samos-it.com/posts/sriov-openstack-juno-fuel-6-1.html</a><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > It seems there is no official documentation yet<br>
> > other than 2<br>
> > > > out of date wiki pages. I would like take the<br>
> > content of my<br>
> > > > blog post to official OpenStack docs if you<br>
> > guys/girls think<br>
> > > > it's useful for the broader audience.<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > Regards,<br>
> > > ><br>
> > > ><br>
> > > > Sam Stoelinga<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > _______________________________________________<br>
> > > > Mailing list:<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
> > > > Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
> > > > Unsubscribe :<br>
> > > ><br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
> > ><br>
> > > --<br>
> > > Andreas<br>
> > > (IRC: scheuran)<br>
> > ><br>
> ><br>
> ><br>
> ><br>
> ><br>
><br>
> --<br>
> Andreas<br>
> (IRC: scheuran)<br>
><br>
<br>
</div></div></blockquote></div><br></div>
_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
</blockquote></div><div dir="ltr">-- <br></div></div></div><span class="HOEnZb"><font color="#888888"><div dir="ltr">Mike Scherbakov<br>#mihgen</div>
</font></span></blockquote></div><br></div>