Hello Harshit:
You need first to fix your underlying network configuration. If the interfaces created on "sriovnet1" can't talk to each other this is because your network is blocking this traffic. Check your TOR switches to enable/allow this VLAN range.
Regards.
On Thu, Jun 23, 2022 at 9:42 AM Harshit Mishra harshit@voereir.com wrote:
Hello Rodolfo,
Thanks for your support.
If the VMs are spawned without any issue, that could be a problem in the underlying network deployment. When sending traffic from a VM with an SR-IOV port, try to capture the traffic in the physical function. Please check the physical network connected to the SR-IOV network cards allow VLAN traffic in 1000:1009 (according to your configuration).
Network I have configured of VLAN type in a given range.
openstack network show 4cacd198-29c3-47b3-bf77-39f1d01c9a22 -c provider:network_type -c provider:physical_network -c provider:segmentation_id +---------------------------+-----------+ | Field | Value | +---------------------------+-----------+ | provider:network_type | vlan | | provider:physical_network | sriovnet1 | | provider:segmentation_id | 1002 | +---------------------------+-----------+
Note: I forget to add that Intra-communication between the VM's working fine deployed on same HOST but inter-communication not working. One more thing IP address is not assigning to interfaces of VM. We have to configure static IP address manually.
Please find below incoming traffic flow on PF.
# tcpdump -i enp8s0f1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp8s0f1, link-type EN10MB (Ethernet), capture size 262144 bytes 12:39:28.584000 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:88:f5:3e (oui Unknown), length 285 12:39:39.183491 IP6 fe80::a236:9fff:fe24:803a > ip6-allrouters: ICMP6, router solicitation, length 16 12:39:49.660269 IP6 fe80::a236:9fff:fe53:e598.5240 > ff02::15a.5240: UDP, length 221 12:39:49.660897 IP6 fe80::a236:9fff:fe53:e59a.5240 > ff02::15a.5240: UDP, length 221 12:39:54.105381 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from a0:36:9f:d8:0d:a8 (oui Unknown), length 293 12:39:54.897051 LLDP, length 46 12:39:56.281671 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 3c:fd:fe:cd:06:71 (oui Unknown), length 293 12:40:06.286310 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from a0:36:9f:d8:0d:aa (oui Unknown), length 293 12:40:15.039416 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 3c:fd:fe:cd:06:70 (oui Unknown), length 293 12:40:19.675902 IP6 fe80::a236:9fff:fe53:e598.5240 > ff02::15a.5240: UDP, length 221 12:40:19.676390 IP6 fe80::a236:9fff:fe53:e59a.5240 > ff02::15a.5240: UDP, length 221 12:40:25.708972 LLDP, length 46 12:40:29.288242 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:c4:3f:4f (oui Unknown), length 285 12:40:33.008653 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:88:f5:3e (oui Unknown), length 285
Another question: do the VMs with SR-IOV port receive the DHCP reply? If so, at least you know the compute to controller communication is working (assuming the DHCP agent is on the controller).
No, VMs not receiving DHCP reply, you can refer above logs. Yes, DHCP agent is on controller.
Kindly let me know anything else is required.
Thanks!
Regards, Harshit Mishra
*From:* Rodolfo Alonso Hernandez ralonsoh@redhat.com *Sent:* Wednesday, June 22, 2022 6:46 PM *To:* Harshit Mishra harshit@voereir.com *Subject:* Re: [kolla-ansible][xena] Facing network issue when providing SR-IOV support
Hello Harshit:
If the VMs are spawned without any issue, that could be a problem in the underlying network deployment. When sending traffic from a VM with an SR-IOV port, try to capture the traffic in the physical function. Please check the physical network connected to the SR-IOV network cards allow VLAN traffic in 1000:1009 (according to your configuration).
Another question: do the VMs with SR-IOV port receive the DHCP reply? If so, at least you know the compute to controller communication is working (assuming the DHCP agent is on the controller).
Regards.
On Wed, Jun 22, 2022 at 2:17 PM Harshit Mishra harshit@voereir.com wrote:
Hi!
I have deployed Openstack Xena using kolla-ansible (v13.0.1) on a multi-node setup (1 controller+network, multiple computes).
On this cluster I would like to have support for normal, as well as direct (SR-IOV) vNIC types.
I have done all the pre-requirement like configure VF on SR-IOV network interface.
In the current deployment, I have created two physnets, one for flat network (called physnet1 on br-ex), and one for SR-IOV (called sriovnet1 on Intel 10G 2P X520 card). I am creating one network of VXLAN type for normal vNIC, and one of VLAN type on sriovnet1 (called sriov_network) for direct vNIC.
Mapping of PF with provider in sriov-agent.ini:
# cat /etc/kolla/neutron-sriov-agent/sriov_agent.ini
[sriov_nic]
physical_device_mappings = sriovnet1:enp8s0f1
exclude_devices =
[securitygroup]
firewall_driver = neutron.agent.firewall.NoopFirewallDriver
ml2_conf.ini on controller node:
# cat /etc/kolla/neutron-server/ml2_conf.ini
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan,vlan,flat
mechanism_drivers = openvswitch,l2population,sriovnicswitch
extension_drivers = port_security
[ml2_type_vlan]
network_vlan_ranges = physnet1,sriovnet1:1000:1009
[ml2_type_flat]
flat_networks = physnet1,sriovnet1
[ml2_sriov]
agent_required = False
supported_pci_vendor_devs = 8086:10ed
[ml2_type_vxlan]
vni_ranges = 1:1000
# grep -nr enabled_filters /etc/kolla/nova-scheduler/
/etc/kolla/nova-scheduler/nova.conf:13:enabled_filters = ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,SameHostFilter,DifferentHostFilter,AggregateInstanceExtraSpecsFilter,PciPassthroughFilter
# grep passthrough /etc/kolla/nova-compute/nova.conf
passthrough_whitelist = [{"physical_network": "sriovnet1", "devname": "enp8s0f1"}]
The interfaces on VMs from normal vNICs are able to communicate inter-compute. The same is not working for the direct vNICs belonging to the sriov_network. I have tried multiple config changes, but none seem to work.
Please help in solving this issue, or suggest a different way of achieving this if this is wrong or not optimal.
Thanks!
Regards, Harshit Mishra