Open Stack

> -----Original Message-----
> From: Mikyung Kang [mailto:mkkang at isi.edu]
> Sent: Tuesday, September 22, 2015 7:20 PM
> To: openstack at lists.openstack.org
> Subject: [Openstack] Is it possible to setup both SR-IOV-IB and Non-SR-IOV-
> Ethernet within 1 VM?
> 
> Hello,
> 
> I'm working for Icehouse-SR-IOV testbed on CentOS7.1.
> 
> I could setup SR-IOV-Infiniband testbed and SR-IOV-IB-VM is working fine:
> https://wiki.openstack.org/wiki/Mellanox-Neutron-Icehouse-Redhat-
> InfiniBand
> 
> Now, on SR-IOV-Infiniband testbed, I'm going to add one more normal
> ethernet (10G) NIC into VM, that is, 2 different types of network (SR-IOV and
> Non-SR-IOV) within 1 VM. Is it possible?
> 
> Even though I added one more net/subnet/bridge/port for normal ethernet
> NIC, VM can't get normal ethernet IP. Only SR-IOV-IB NIC is working. Failed
> to make both SR-IOV-IB and Non-SR-IOV-Ethernet work within 1 VM.
> 
> I think that it's not possible to support SR-IOV and Non-SR-IOV on the same
> compute node when using Mellanox vif/eswitch/mlnx-agent packages.
> 
> There are 3 limitations:
> 
> (1) dhcp_driver in /etc/neutron/dhcp_agent.ini @controller ---> we have to
> choose one between linux.dhcp.Dnsmasq (Ethernet) or
> minx_dhcp.MlnxDnsmasq (IB).
Yes indeed the mlnx_dhcp added the client-id so that the VM would get a dhcp for  InfiniBand (in  InfiniBand  there is not mac), but I think you can work around it by sending  the excepted client id 
also  for  Ethernet interface.  It should be something like ff:00:00:00:00:00:02:00:00:02:c9:00:<first 3 mac bytes>:00:00: <last 3 mac bytes>
neutron support adding dhcp opt for a port so you can add the client-id

> (2) vif_driver in /etc/nova/nova.conf @compute --> we have to choose one
> between libvirt.vif (Non-SR-IOV) and mlnxvif.vif (SR-IOV). Also, minx
> [eswitch] requires mellanox's product/vendor ID.
The mlnx mechanism driver in Icehouse is working with a heck on the Normal vnic_type in liberty I checked it to be using "direct" vnic_type so it is possible to create vm with 
One normal port and on direct port the normal port will be bounded by the openvswitch-agent and the direct port will be bounded by the mellanox agent for SR-IOV InfiniBand.
Please see https://wiki.openstack.org/wiki/Mellanox-Neutron-Liberty-Redhat-InfiniBand for more information. (the page is still under construction)

> (3) neutron-mlnx-agent @compute --> we had to stop neutron-
> openvswitch-agent (Non-SR-IOV) and then start only neutron-mlnx-agent
> (SR-IOV) to make SR-IOV-VM work.
> 
The change in liberty will resolve this issue as explained above.

> Nobody tried to support both SR-IOV and Non-SR-IOV within 1 VM? It seems
> that Juno/Kilo version also uses same way.
> Is it possible or impossible to support both at the same time using current
> icehouse-mellanox packages??
On icehouse probably not, but you can try liberty. I haven't test it myself. 

> Any idea to support this? Any help is welcome!
> 
> Thanks,
> Mikyung
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack