[neutron] Neutron sriov vf-lag port binding
Hello everyone, I have tested sriov with Connectx-5 cards, it works well but I also want to try ovs with hardware offload feature. I believe I have properly configured pf, vfs and bonding :). PF’s are working in switchdev mode (I have followed stackhpcs article[1]). Unfortunately, I see port binding errors when creating instances. I have created a vlan provider network and a port with vnic_type=direct under that network. I am trying to create instance with that port. nova-compute.log
2025-04-04 17:37:28.014 7 ERROR nova.compute.manager [instance: bf8408e8-20a9-4963-9e50-033e54aba9fe] nova.exception.PortBindingFailed: Binding failed for port 6cbd8181-5707-4349-9fa0-c35e7370bfcd, please check neutron logs for more information.
neutron-server.log
025-04-04 17:37:27.000 25 DEBUG neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driver [req-ff074be0-e410-4df8-8938-7e2b59a84bfd req-d9bffb08-7e74-4976-b3ae-8cb544ec71f5 fbbdf9e30cad498180c3b637d1bba6d9 3e7a1867f1c24a158b4bb2d2f3f68b9c - - default default] Refusing to bind due to unsupported vnic_type: direct with switchdev capability bind_port /var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py:130
neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py
if (vnic_type == portbindings.VNIC_DIRECT and 'switchdev' in capabilities): LOG.debug("Refusing to bind due to unsupported vnic_type: %s " "with switchdev capability", portbindings.VNIC_DIRECT)
nova.conf
[pci] passthrough_whitelist = [{"physical_network": "physnet5", "devname": "ens10f0"},{"physical_network": "physnet6", "devname": "ens10f1"}]
[sriov_nic] physical_device_mappings = physnet5:ens10f0,physnet6:ens10f1 Is vnic_type=direct wrong if switchdev in capabilities or is this a configuration issue? Environment: Openstack version: Caracal Operating System: Ubuntu 22.04 Thanks, Serhat [1] - https://www.stackhpc.com/vflag-kayobe.html
Hello Serhat: From the Neutron point of view, a port with "vnic_type=direct" and "switchdev capabilities" is not a ML2/SR-IOV port. This kind of port can be used in ML2/OVS or ML2/OVN. The NIC driver will create a VF port representor that will act as a TAP device and will connect to an OVS bridge. This port will have the same properties as a normal one (you can use port security, for example). You need to enable one of the ML2 mechanism drivers supported and deploy in the compute node the needed services (depending on the driver). Regards. On Fri, Apr 4, 2025 at 8:01 PM Serhat Rıfat Demircan < demircan.serhat@gmail.com> wrote:
Hello everyone,
I have tested sriov with Connectx-5 cards, it works well but I also want to try ovs with hardware offload feature. I believe I have properly configured pf, vfs and bonding :). PF’s are working in switchdev mode (I have followed stackhpcs article[1]). Unfortunately, I see port binding errors when creating instances.
I have created a vlan provider network and a port with vnic_type=direct under that network. I am trying to create instance with that port.
nova-compute.log
2025-04-04 17:37:28.014 7 ERROR nova.compute.manager [instance: bf8408e8-20a9-4963-9e50-033e54aba9fe] nova.exception.PortBindingFailed: Binding failed for port 6cbd8181-5707-4349-9fa0-c35e7370bfcd, please check neutron logs for more information.
neutron-server.log
025-04-04 17:37:27.000 25 DEBUG neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driver [req-ff074be0-e410-4df8-8938-7e2b59a84bfd req-d9bffb08-7e74-4976-b3ae-8cb544ec71f5 fbbdf9e30cad498180c3b637d1bba6d9 3e7a1867f1c24a158b4bb2d2f3f68b9c - - default default] Refusing to bind due to unsupported vnic_type: direct with switchdev capability bind_port /var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py:130
neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py
if (vnic_type == portbindings.VNIC_DIRECT and 'switchdev' in capabilities): LOG.debug("Refusing to bind due to unsupported vnic_type: %s " "with switchdev capability", portbindings.VNIC_DIRECT)
nova.conf
[pci] passthrough_whitelist = [{"physical_network": "physnet5", "devname": "ens10f0"},{"physical_network": "physnet6", "devname": "ens10f1"}]
[sriov_nic] physical_device_mappings = physnet5:ens10f0,physnet6:ens10f1
Is vnic_type=direct wrong if switchdev in capabilities or is this a configuration issue?
Environment: Openstack version: Caracal Operating System: Ubuntu 22.04
Thanks, Serhat
as an aside form a nova point fo view no backend supprot VF-lag even on the neutron side it was not peroperly enabled with a user facing api. people just figured out how make backedn use it without actually makeign neutron and nova aware of what was happening. upstream openstack does not offically supprot VF-lag even if it technically possible to use the melonox functionality in an openstack deployment. the reason im calling that out is if you find any bugs or issues with using VF-lag they are not Openstack (nova/neutron) bugs. On 07/04/2025 13:19, Rodolfo Alonso Hernandez wrote:
Hello Serhat:
From the Neutron point of view, a port with "vnic_type=direct" and "switchdev capabilities" is not a ML2/SR-IOV port. This kind of port can be used in ML2/OVS or ML2/OVN. The NIC driver will create a VF port representor that will act as a TAP device and will connect to an OVS bridge. This port will have the same properties as a normal one (you can use port security, for example).
You need to enable one of the ML2 mechanism drivers supported and deploy in the compute node the needed services (depending on the driver).
Regards.
On Fri, Apr 4, 2025 at 8:01 PM Serhat Rıfat Demircan <demircan.serhat@gmail.com> wrote:
Hello everyone,
I have tested sriov with Connectx-5 cards, it works well but I also want to try ovs with hardware offload feature. I believe I have properly configured pf, vfs and bonding :). PF’s are working in switchdev mode (I have followed stackhpcs article[1]). Unfortunately, I see port binding errors when creating instances.
I have created a vlan provider network and a port with vnic_type=direct under that network. I am trying to create instance with that port.
nova-compute.log
2025-04-04 17:37:28.014 7 ERROR nova.compute.manager [instance: bf8408e8-20a9-4963-9e50-033e54aba9fe] nova.exception.PortBindingFailed: Binding failed for port 6cbd8181-5707-4349-9fa0-c35e7370bfcd, please check neutron logs for more information.
neutron-server.log
025-04-04 17:37:27.000 25 DEBUG neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driver [req-ff074be0-e410-4df8-8938-7e2b59a84bfd req-d9bffb08-7e74-4976-b3ae-8cb544ec71f5 fbbdf9e30cad498180c3b637d1bba6d9 3e7a1867f1c24a158b4bb2d2f3f68b9c - - default default] Refusing to bind due to unsupported vnic_type: direct with switchdev capability bind_port /var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py:130
neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py
if (vnic_type == portbindings.VNIC_DIRECT and 'switchdev' in capabilities): LOG.debug("Refusing to bind due to unsupported vnic_type: %s " "with switchdev capability", portbindings.VNIC_DIRECT)
nova.conf
[pci] passthrough_whitelist = [{"physical_network": "physnet5", "devname": "ens10f0"},{"physical_network": "physnet6", "devname": "ens10f1"}]
[sriov_nic] physical_device_mappings = physnet5:ens10f0,physnet6:ens10f1
Is vnic_type=direct wrong if switchdev in capabilities or is this a configuration issue?
Environment: Openstack version: Caracal Operating System: Ubuntu 22.04
Thanks, Serhat
Hello guys, Thank you for explanation. I successfully created instances after switching from ML2/SR-IOV to ML2/OVS for my test network. Thanks, Serhat On Mon, Apr 7, 2025 at 3:44 PM Sean Mooney <smooney@redhat.com> wrote:
as an aside form a nova point fo view no backend supprot VF-lag
even on the neutron side it was not peroperly enabled with a user facing api.
people just figured out how make backedn use it without actually makeign neutron and nova aware of what was happening.
upstream openstack does not offically supprot VF-lag even if it technically possible to use the melonox functionality in an openstack
deployment. the reason im calling that out is if you find any bugs or issues with using VF-lag they are not Openstack (nova/neutron) bugs.
On 07/04/2025 13:19, Rodolfo Alonso Hernandez wrote:
Hello Serhat:
From the Neutron point of view, a port with "vnic_type=direct" and "switchdev capabilities" is not a ML2/SR-IOV port. This kind of port can be used in ML2/OVS or ML2/OVN. The NIC driver will create a VF port representor that will act as a TAP device and will connect to an OVS bridge. This port will have the same properties as a normal one (you can use port security, for example).
You need to enable one of the ML2 mechanism drivers supported and deploy in the compute node the needed services (depending on the driver).
Regards.
On Fri, Apr 4, 2025 at 8:01 PM Serhat Rıfat Demircan <demircan.serhat@gmail.com> wrote:
Hello everyone,
I have tested sriov with Connectx-5 cards, it works well but I also want to try ovs with hardware offload feature. I believe I have properly configured pf, vfs and bonding :). PF’s are working in switchdev mode (I have followed stackhpcs article[1]). Unfortunately, I see port binding errors when creating instances.
I have created a vlan provider network and a port with vnic_type=direct under that network. I am trying to create instance with that port.
nova-compute.log
2025-04-04 17:37:28.014 7 ERROR nova.compute.manager [instance: bf8408e8-20a9-4963-9e50-033e54aba9fe] nova.exception.PortBindingFailed: Binding failed for port 6cbd8181-5707-4349-9fa0-c35e7370bfcd, please check neutron logs for more information.
neutron-server.log
025-04-04 17:37:27.000 25 DEBUG neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driver [req-ff074be0-e410-4df8-8938-7e2b59a84bfd req-d9bffb08-7e74-4976-b3ae-8cb544ec71f5 fbbdf9e30cad498180c3b637d1bba6d9 3e7a1867f1c24a158b4bb2d2f3f68b9c - - default default] Refusing to bind due to unsupported vnic_type: direct with switchdev capability bind_port
/var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py:130
neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py
if (vnic_type == portbindings.VNIC_DIRECT and 'switchdev' in capabilities): LOG.debug("Refusing to bind due to unsupported vnic_type: %s " "with switchdev capability", portbindings.VNIC_DIRECT)
nova.conf
[pci] passthrough_whitelist = [{"physical_network": "physnet5", "devname": "ens10f0"},{"physical_network": "physnet6", "devname": "ens10f1"}]
[sriov_nic] physical_device_mappings = physnet5:ens10f0,physnet6:ens10f1
Is vnic_type=direct wrong if switchdev in capabilities or is this a configuration issue?
Environment: Openstack version: Caracal Operating System: Ubuntu 22.04
Thanks, Serhat
participants (3)
-
Rodolfo Alonso Hernandez
-
Sean Mooney
-
Serhat Rıfat Demircan