<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">Hi!<br>
</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"><br>
I have deployed Openstack Xena using kolla-ansible (v13.0.1) on a multi-node setup (1 controller+network, multiple computes).</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">On this cluster I would like to have support for normal, as well as direct (SR-IOV) vNIC types.</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">I have done all the pre-requirement like configure VF on SR-IOV network interface.</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">In the current deployment, I have created two physnets, one for flat network (called physnet1 on br-ex), and one for SR-IOV (called sriovnet1 on Intel 10G 2P X520 card). I am creating one network of VXLAN
type for normal vNIC, and one of VLAN type on sriovnet1 (called sriov_network) for direct vNIC.</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">Mapping of PF with provider in sriov-agent.ini:</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"># cat /etc/kolla/neutron-sriov-agent/sriov_agent.ini</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">[sriov_nic]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">physical_device_mappings = sriovnet1:enp8s0f1</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">exclude_devices =</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">[securitygroup]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">firewall_driver = neutron.agent.firewall.NoopFirewallDriver</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">----------------------------------------------</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">ml2_conf.ini on controller node:</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"># cat /etc/kolla/neutron-server/ml2_conf.ini</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">[ml2]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">type_drivers = flat,vlan,vxlan</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">tenant_network_types = vxlan,vlan,flat</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">mechanism_drivers = openvswitch,l2population,sriovnicswitch</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">extension_drivers = port_security</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">[ml2_type_vlan]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">network_vlan_ranges = physnet1,sriovnet1:1000:1009</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">[ml2_type_flat]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">flat_networks = physnet1,sriovnet1</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">[ml2_sriov]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">agent_required = False</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">supported_pci_vendor_devs = 8086:10ed</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">[ml2_type_vxlan]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">vni_ranges = 1:1000</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">----------------------------------------------</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"># grep -nr enabled_filters /etc/kolla/nova-scheduler/</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">/etc/kolla/nova-scheduler/nova.conf:13:enabled_filters = ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,SameHostFilter,DifferentHostFilter,AggregateInstanceExtraSpecsFilter,PciPassthroughFilter</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">----------------------------------------------</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"># grep passthrough /etc/kolla/nova-compute/nova.conf</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">passthrough_whitelist = [{"physical_network": "sriovnet1", "devname": "enp8s0f1"}]</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">The interfaces on VMs from normal vNICs are able to communicate inter-compute. The same is not working for the direct vNICs belonging to the sriov_network. I have tried multiple config changes, but none seem
to work.</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black"> </span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:12pt;color:black">Please help in solving this issue, or suggest a different way of achieving this if this is wrong or not optimal.</span></p>
<p style="color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);font-size:10pt;font-family:Calibri, sans-serif;margin:0px">
<span style="margin:0px;font-size:11pt"> </span></p>
<div style="margin:0px;font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif;color:rgb(32, 31, 30);background-color:rgb(255, 255, 255)">
<div style="margin:0px">
<div style="margin:0px">
<p style="font-size:10pt;font-family:Calibri, sans-serif;margin:0px"><span style="margin:0px;font-size:11pt">Thanks!</span></p>
</div>
</div>
</div>
<br>
Regards,
<div>Harshit Mishra</div>
</div>
</body>
</html>