- openstack-discuss - lists.openstack.org

[nova] failing OVS offload on mellanox connectx-4 (RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7)
by Manuel Sopena Ballesteros 06 Mar '19

06 Mar '19

Dear Openstack community, first let me apologise for my ignorance in this topic, this is the first time I try to setup an environment like this and I don't know anyone who has done it before. I am trying to setup OVS offload following this document https://docs.openstack.org/neutron/rocky/admin/config-ovs-offload.html My Openstack is Rocky deployed through kolla-ansible. I have a feeling my OVS offload is not setup properly as vm creation fails and the error "RepresentorNotFound: Failed getting representor port for PF" can be see in nova-compute logs. # Nova compute logs: 2019-03-05 11:24:46.392 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Attempting claim on node zeus-54.localdomain: memory 8192 MB, disk 80 GB, vcpus 4 CPU 2019-03-05 11:24:46.393 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Total memory: 524173 MB, used: 8704.00 MB 2019-03-05 11:24:46.394 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] memory limit not specified, defaulting to unlimited 2019-03-05 11:24:46.394 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Total disk: 18630 GB, used: 80.00 GB 2019-03-05 11:24:46.395 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] disk limit not specified, defaulting to unlimited 2019-03-05 11:24:46.396 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Total vcpu: 56 VCPU, used: 4.00 VCPU 2019-03-05 11:24:46.396 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] vcpu limit not specified, defaulting to unlimited 2019-03-05 11:24:46.408 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Claim successful on node zeus-54.localdomain 2019-03-05 11:24:47.144 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Creating image 2019-03-05 11:24:48.957 7 ERROR os_vif [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Failed to plug vif VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.957 7 ERROR os_vif Traceback (most recent call last): 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 77, in plug 2019-03-05 11:24:48.957 7 ERROR os_vif plugin.plug(vif, instance_info) 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 215, in plug 2019-03-05 11:24:48.957 7 ERROR os_vif self._plug_vf_passthrough(vif, instance_info) 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 189, in _plug_vf_passthrough 2019-03-05 11:24:48.957 7 ERROR os_vif representor = linux_net.get_representor_port(pf_ifname, vf_num) 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py", line 381, in get_representor_port 2019-03-05 11:24:48.957 7 ERROR os_vif raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num) 2019-03-05 11:24:48.957 7 ERROR os_vif RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.957 7 ERROR os_vif 2019-03-05 11:24:48.960 7 ERROR nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Failed to start libvirt guest: InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.969 7 ERROR os_vif [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Failed to unplug vif VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.969 7 ERROR os_vif Traceback (most recent call last): 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 110, in unplug 2019-03-05 11:24:48.969 7 ERROR os_vif plugin.unplug(vif, instance_info) 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 277, in unplug 2019-03-05 11:24:48.969 7 ERROR os_vif self._unplug_vf_passthrough(vif, instance_info) 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 250, in _unplug_vf_passthrough 2019-03-05 11:24:48.969 7 ERROR os_vif representor = linux_net.get_representor_port(pf_ifname, vf_num) 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py", line 381, in get_representor_port 2019-03-05 11:24:48.969 7 ERROR os_vif raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num) 2019-03-05 11:24:48.969 7 ERROR os_vif RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.969 7 ERROR os_vif 2019-03-05 11:24:48.971 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Deleting instance files /var/lib/nova/instances/1b03f2e7-0c44-46c2-87f5-d68f93654c58_del 2019-03-05 11:24:48.972 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Deletion of /var/lib/nova/instances/1b03f2e7-0c44-46c2-87f5-d68f93654c58_del complete 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Instance failed to spawn: InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Traceback (most recent call last): 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2368, in _build_resources 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] yield resources 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2132, in _build_and_run_instance 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] block_device_info=block_device_info) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3080, in spawn 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] destroy_disks_on_failure=True) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5646, in _create_domain_and_network 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] destroy_disks_on_failure) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self.force_reraise() 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] six.reraise(self.type_, self.value, self.tb) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5605, in _create_domain_and_network 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self.plug_vifs(instance, network_info) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 819, in plug_vifs 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self.vif_driver.plug(instance, vif) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 807, in plug 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self._plug_os_vif(instance, vif_obj) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 787, in _plug_os_vif 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] raise exception.InternalError(msg) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] 2019-03-05 11:24:49.085 7 INFO nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Terminating instance 2019-03-05 11:24:49.091 7 INFO nova.virt.libvirt.driver [-] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Instance destroyed successfully. 2019-03-05 11:24:49.099 7 ERROR os_vif [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Failed to unplug vif VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.099 7 ERROR os_vif Traceback (most recent call last): 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 110, in unplug 2019-03-05 11:24:49.099 7 ERROR os_vif plugin.unplug(vif, instance_info) 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 277, in unplug 2019-03-05 11:24:49.099 7 ERROR os_vif self._unplug_vf_passthrough(vif, instance_info) 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 250, in _unplug_vf_passthrough 2019-03-05 11:24:49.099 7 ERROR os_vif representor = linux_net.get_representor_port(pf_ifname, vf_num) 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py", line 381, in get_representor_port 2019-03-05 11:24:49.099 7 ERROR os_vif raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num) 2019-03-05 11:24:49.099 7 ERROR os_vif RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.099 7 ERROR os_vif 2019-03-05 11:24:49.101 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Deletion of /var/lib/nova/instances/1b03f2e7-0c44-46c2-87f5-d68f93654c58_del complete 2019-03-05 11:24:49.198 7 INFO nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Took 0.11 seconds to destroy the instance on the hypervisor. 2019-03-05 11:24:50.384 7 INFO nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Took 0.77 seconds to deallocate network for instance. 2019-03-05 11:24:50.624 7 INFO nova.scheduler.client.report [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Deleted allocation for instance 1b03f2e7-0c44-46c2-87f5-d68f93654c58 2019-03-05 11:25:07.884 7 INFO nova.compute.resource_tracker [req-1b32a794-0bf0-4abe-a024-699920f791d2 - - - - -] Final resource view: name=zeus-54.localdomain phys_ram=524173MB used_ram=8704MB phys_disk=18630GB used_disk=80GB total_vcpus=56 used_vcpus=4 pci_stats=[PciDevicePool(count=15,numa_node=1,product_id='1016',tags={dev_type='type-VF',physical_network='physnet1'},vendor_id='15b3')] ### This is my environment/configuration # OVS version: docker exec -it openvswitch_vswitchd ovs-vswitchd --version ovs-vswitchd (Open vSwitch) 2.10.1 # Mellanox firmware version: [root@zeus-54 ~]# ibv_devinfo hca_id: mlx5_9 transport: InfiniBand (0) fw_ver: 14.24.1000 # Kernel version: [root@zeus-54 ~]# uname -r 3.10.0-862.el7.x86_64 # OS distro: [root@zeus-54 ~]# cat /etc/*release CentOS Linux release 7.6.1810 (Core) NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" CentOS Linux release 7.6.1810 (Core) CentOS Linux release 7.6.1810 (Core) # SR-IOV devices: [root@zeus-54 ~]# lspci -D -nn | grep -i mel 0000:88:00.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] [15b3:1015] 0000:88:00.1 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] [15b3:1015] 0000:88:00.2 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.3 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.4 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.5 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.6 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.7 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.1 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.2 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.3 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.4 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.5 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.6 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.7 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:02.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:02.1 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] [root@zeus-54 ~]# ibdev2netdev -v 0000:88:00.0 mlx5_0 (MT4117 - MT1611X09936) CX4121A - ConnectX-4 LX SFP28 fw 14.24.1000 port 1 (ACTIVE) ==> ens2f0 (Up) 0000:88:00.1 mlx5_1 (MT4117 - MT1611X09936) CX4121A - ConnectX-4 LX SFP28 fw 14.24.1000 port 1 (ACTIVE) ==> ens2f1 (Up) 0000:88:01.2 mlx5_10 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f2 (Up) 0000:88:01.3 mlx5_11 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f3 (Up) 0000:88:01.4 mlx5_12 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f4 (Up) 0000:88:01.5 mlx5_13 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f5 (Up) 0000:88:01.6 mlx5_14 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f6 (Up) 0000:88:01.7 mlx5_15 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f7 (Up) 0000:88:02.0 mlx5_16 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s2 (Up) 0000:88:02.1 mlx5_17 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s2f1 (Up) 0000:88:00.2 mlx5_2 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f2 (Up) 0000:88:00.3 mlx5_3 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f3 (Up) 0000:88:00.4 mlx5_4 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f4 (Up) 0000:88:00.5 mlx5_5 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f5 (Up) 0000:88:00.6 mlx5_6 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f6 (Up) 0000:88:00.7 mlx5_7 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f7 (Up) 0000:88:01.0 mlx5_8 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1 (Up) 0000:88:01.1 mlx5_9 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f1 (Up) # Embeded switch configuration: [root@zeus-54 ~]# devlink dev eswitch show pci/0000:88:00.0 pci/0000:88:00.0: mode legacy inline-mode none encap enable [root@zeus-54 ~]# devlink dev eswitch show pci/0000:88:00.1 pci/0000:88:00.1: mode legacy inline-mode none encap enable [root@zeus-54 ~]# sudo ethtool -k ens2f0 | grep hw-tc-offload hw-tc-offload: on [root@zeus-54 ~]# sudo ethtool -k ens2f1 | grep hw-tc-offload hw-tc-offload: on [root@zeus-54 ~]# docker exec -it openvswitch_vswitchd sudo ovs-vsctl get Open_vSwitch . other_config {hw-offload="true", max-idle="30000"} [root@zeus-54 ~]# docker exec -it openvswitch_vswitchd sudo ovs-vsctl get Open_vSwitch . other_config {hw-offload="true", max-idle="30000"} # Check ovs offload [root@zeus-54 ~]# docker exec -it openvswitch_vswitchd sudo ovs-dpctl show 2019-03-05T00:50:42Z|00001|dpif_netlink|INFO|The kernel module does not support meters. system@ovs-system: lookups: hit:0 missed:0 lost:0 flows: 0 masks: hit:0 total:0 hit/pkt:0.00 port 0: ovs-system (internal) port 1: br-ex (internal) port 2: br-int (internal) port 3: br-tun (internal) QUESTIONS: Is my kernel version right? Am I supposed to see the PFs attached to a port in the ovs-dpctl show output? What else can I check? Thank you very much NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

4 5

[ptl][placement][election] Chris Dent's Candidacy for Placement PTL
by Chris Dent 06 Mar '19

06 Mar '19

Hi, I'm announcing my candidacy [1] for Placement PTL for Train. I've been working on the Placement service from the start. Helping it become a straightforward HTTP API for managing resources and their inventories, qualities, and use. I believe the primary focus of the project should be to fulfill the promise of the service — to make any variety of workload placement more reasonable-to-manage and correct-to-use — while keeping the straightforwardness. Just as important is making sure the project, being newly independent, settles in well. We will need time, attention and patience to find our way. What we do as a group should be decided by the group. The role of the PTL should be to help the group reach decisions smoothly and make sure the information needed to make decisions is shared broadly. To help understand how I think about Placement, here are some ideas on areas we may wish to focus (if we agree): * Get the house in order. Get bug and feature tracking systems in place and properly documented. Prepare the Forum and PTG. * Help other projects use Placement. Adding features to Placement is often easier than taking advantage of those new features. A lot of features exist that are not yet used to their full potential. * Attend to maintenance. Bug fixing, documentation, performance, refactoring, gate management, and keeping clients up to date need to have equal or greater attention than feature development. * Do what matters. Only add features that have clear customers and use cases, and have committed owners responsible for long-term maintenance. * Experiment. Contrary to the above, some of our time should be spent discovering what may matter in the future and sharing those discoveries: "Placement can help with that." Edge and multi-cloud related use cases have plenty of potential. * Document. Fill in the gaps in documentation for users and contributors. There's a lot of power in Placement and using it most effectively is not as well-documented as it could be. * Be welcoming, fun, nice, and encouraging to everyone. I hope my history with the project is solid evidence for my commitment to its success. I'm extremely proud of the work we've done and optimistic about the positive impact we will have in the future. Thanks. [1] https://review.openstack.org/641339 -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

1 0

[kuryr] Daniel Mellado's Candidacy for Kuryr PTL
by Daniel Mellado 06 Mar '19

06 Mar '19

Hi Kuryrs, I'm writing this in order to announce my candidacy for Kuryr PTL during the Train cycle, althought I expect no Train horns this time at Denver! It has been my pleaseure and nonor to act as PTL for Kuryr the past two cycles and I'd like like to run for this for another release. I'd like to thank you all on helping me deliver and run this project during all this time I think we have been doing an awesome job so far within the OpenStack community. During Stein, we had an great cycle, delivering most of the goals we set for it, and movign forward, these are some of the goals that I propose for the team during the Train cycle: * Network Policies: Although this is a feature we landed in last cycle, I'd like for it to be enhanced so it gets faster, so optimizing its performance would be one of the cycle goals. * Neutron pooling resource speedupts: Tied to the last topic, it'll still need to further improve the resource handling speed. * Release model: So far we've been catching up with two release models, OpenStack and Kubernetes, as this is quite an effort and could lead to an endless testing matrix, one of the goals that I'd like to settle in the cycle would be on how to deal with such circumstances. * SRIOV/DPDK upstream gate testing. * Further improve debugging and instrospection tools usign Kubernetes plugins. * Expand the Kuryr community by adding more contributors. Also, I'd like to coordinate integrating some features that we started last cycle, such as Native OVN Layer Load balancing and Multi device/net support. Outside of this key areas, my priority is also helping the community by acting as an interface for the cross-project sessions and further improve our presence in initiatives such as Openlab, OPNFV and Edge. Besides that, I've also sent an email with etherpad links for PTG planning[1] and forum sessions[2] at the Summit. [1] https://etherpad.openstack.org/p/kuryr-denver-train-ptg-planning [2] https://etherpad.openstack.org/p/DEN-train-forum-kuryr-brainstorming Thanks a lot! Daniel Mellado (dmellado)

1 0

[horizon] Horizon PTG planning
by Ivan Kolodyazhny 06 Mar '19

06 Mar '19

Hi team, I created etherpad to collect topics for PTG. Feel free to add any topic you would like to discuss face to face. [1] https://etherpad.openstack.org/p/horizon-train-ptg Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

1 0

[ptl][docs][election] Stephen Finucane's candidacy for Documentation PTL
by Stephen Finucane 06 Mar '19

06 Mar '19

Hey, I'd like to announce my candidacy for Docs PTL for the Train cycle. I've been involved in the documentation project since the heady days of 2016, back when the documentation team centrally managed the user-facing docs of every core OpenStack project. My initial contributions mostly focused on correcting and expanding the documentation related to my primary focus area in OpenStack, NFV enablement, though this slowly expanded as I built up competence in all things docs. At some point, for many disparate reasons, *os-manuals-migration* happened [1]_, and each project began taking responsbility for their own documentation. I assisted with the migration of the nova docs in particular and have been continuously involved ever since. As a result of *os-manuals-migration*, today's documentation team is mainly intended to act as a liason to other teams, stepping in to help with various documentation-related questions and, where necessary, providing additional tooling. In my opinion, this is both a pragmatic and a sustainable approach and I would like to see the documentation project continue on this trajectory, changing course only if/when necessary. As such, my focus areas for the upcoming cycle would be continued work on making documentation easier to contribute to (or unnecessary, where docs can be generated from code), cleaning up the remaining warts in the existing documentation and documentation processes, and generally ensuring the documentation team is as helpful as possible for the projects that need us. I feel I am well equipped to handle this, and I hope you do too. Cheers, Stephen (stephenfin) .. [1] https://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migr…

1 0

[nova] address comments about the show-server-numa-topology
by yonglihe 06 Mar '19

06 Mar '19

Hi, All Thanks Sean's good comments. Cause there lots of links and discuss, I come out with a docs for collections comments, options, and investigating. please move to google docs for discuss, thanks: https://docs.google.com/document/d/1kRRZFq_ha0T9mFDOEzv0PMvXgtnjGm5ii9mSzdq… links: https://review.openstack.org/#/c/621476/ show server numa_topology https://blueprints.launchpad.net/nova/+spec/show-server-numa-topology (yonglihe) Regards Yongli He

1 0

[senlin] Senlin Monthly(ish) Newsletter Jan/Feb 2019
by Duc Truong 05 Mar '19

05 Mar '19

HTML: https://dkt26111.wordpress.com/2019/02/21/senlin-monthlyish-newsletter-janu… This is the January/February edition of the Senlin monthly(ish) newsletter. The goal of the newsletter is to highlight happenings in the Senlin project. If you have any feedback or questions regarding the contents, please feel free to reach out to me in the #senlin IRC channel. News ---- * We are almost at Stein-3 milestone which coincides with the feature freeze during the week of March 4. Please submit your changes for review before then. Blueprint Status ---------------- * Fail fast locked resource - https://blueprints.launchpad.net/senlin/+spec/fail-fast-locked-resource - Working on documentation and release notes. * Multiple detection modes - https://blueprints.launchpad.net/senlin/+spec/multiple-detection-modes - Working on documentation and release notes. Community Goal Status --------------------- * Python 3 - All patches by Python 3 goal champions for zuul migration, documentation and unit test changes have been merged. - We have set py35 functional tests as voting in gate. * Upgrade Checkers - I have added a patch set to check for unsupported health policies: https://review.openstack.org/#/c/638284/ Reviews Needed -------------- * Improve Health Manager to avoid duplicate health checks: https://review.openstack.org/#/c/634811/ * Fixing devstack tempest jobs for master and stable/rocky: https://review.openstack.org/#/c/637664/ https://review.openstack.org/#/c/635638/

2 1

Using Trove with Rocky
by Ankit Awal 05 Mar '19

05 Mar '19

I am trying to use Trove with Rocky. i am able to make a qcow image using the scripts provided in trove [ disk builder ] . Unfortunately, when i start trove, the status shows building but the instance is up and running. I feel that the problem is regarding the improper setup of the trove image. Can someone guide me with the steps to create a trove image and launch it using kolla ansible *With Regards,* *Ankit Awal* *Niveus Solutions Private Limited* *Email: ankit.awal(a)niveussolutions.com <vishnuprasad.i(a)niveussolutions.com>*

1 0

[ptl][searchlight][election] Trinh nguyen's candidacy for Searchlight PTL
by Trinh Nguyen 05 Mar '19

05 Mar '19

Hello. I would like to announce my candidacy for Searchlight PTL for Train. [4] I have been involved in OpenStack since the Queens cycle as it is needed for my research. Over the last three cycles, I learned a lot in both technical and organizational aspects. That is thank to the help of many community members, TCs, PTLs of the OpenStack projects that I am working on. That inspired me to take the role of Searchlight PTL to help survive the project. I could say I can commit my time for that responsibility with my current job. As many of you may know, at the end of the Rocky cycle, the TC intended to remove Searchlight from the Governance (OpenStack official projects). The reasons for that are because Searchlight has missed several milestones and lacked communication of the PTL. I couldn't stand the fact that a pretty cool project like Searchlight would be abandoned so I volunteered to be the PTL of Searchlight for the Stein cycle. I hoped that I could do something to revive it. With the help of some contributors, we finally could find a "light" for the project. In the first two milestones of Stein, Searchlight has identified itself as a useful tool in several use cases such as being a cloud resource lookup, a cloud resource repository, and a multi-cloud search solution [1]. We hope those use cases will attract more contributors and vendors to support and use Searchlight in the near future. Moreover, the team has developed a grand vision for Searchlight [2] to help sustain the project in alignment with the OpenStack Cloud Vision [3]. The very first thing to make that vision come true is building a strong foundation for Searchlight. I take that responsibility personally and passionately. Like many other OpenStack projects, there are always challenges in contributor participation. For that, I especially see the new contributor engagement is an important job of the PTL and always looking for opportunities to do that (e.g., conferences, meetup, training, etc.). That will not be changed whether you'll have me as the Searchlight PTL for Train or not. I will continue working on helping potential developers to come to Searchlight and build a strong community around it. I know that I knew nothing about Searchlight before Stein. I'm not trying to change things with big ideas here. I just want to help Searchlight succeed in Train and further if I can. Thank you for your consideration. Trinh Nguyen (dangtrinhnt) [1] https://docs.openstack.org/searchlight/latest/user/usecases.html [2] https://docs.openstack.org/searchlight/latest/contributor/searchlight-visio… [3] https://docs.openstack.org/searchlight/latest/contributor/vision-reflection… [4] https://review.openstack.org/#/c/641193/ -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

1 0

[ptl][nova][election] Eric Fried's Candidacy for Nova PTL
by Eric Fried 05 Mar '19

05 Mar '19

Greetings. I am hereby [1] announcing my candidacy for Nova PTL for Train. I am passionate about OpenStack, to the point where I (like many of you) changed companies so I wouldn't have to change jobs. My employer backs me as a 100% upstream developer and core in Nova and Placement, and fully supports my commitment to the role of Nova PTL if you'll have me. Nova is huge, in code as well as in history. I don't pretend to understand all (or even most) of it. It takes the cumulative skill of the whole team to make the project tick. I see the PTL as a facilitator and coordinator of that "group brain", making the whole greater than the sum of the parts. That takes organization, diplomacy, and patience: qualities I have demonstrated by running the nova-scheduler meetings, helping with the politics of extracting Placement, and engaging in tough and contentious design discussions in specs, PTGs, IRC, etc. As PTL, one of my main goals will be to keep those conversations moving and focused, avoiding unproductive tangents. We continue to face challenges in contributor (which includes reviewer!) participation. Partly this is due to the overall change in the character of the community; but I hope to break down barriers for new or part-time contributors who are willing and able to participate. In day-to-day operations this means being responsive and encouraging on IRC and gerrit. In forum and PTG sessions it means working to avoid the "echo chamber effect" and ceding the floor to these less-involved participants so they can be heard. I would also like to continue blueprint tracking including runways, which has been an effective way to ensure attention for efforts outside of our deemed main themes and priorities. Nova is on the cusp of being able to take advantage of advanced Placement features such as nested and sharing providers to solve some of the problems for which Placement was originally conceived. Nova will need to continue to work closely with Placement in its new position as an independent project. I intend to continue being an active Placement core regardless of the outcome of these elections to assist in that transition. We need to continue to foster our relationships with other teams, ensuring proper care and feeding of cross-project initiatives. This applies to the old guard (Cinder, Glance, Ironic, Keystone, Neutron, and Placement as noted) but also to nascent efforts such as Cyborg. This starts with forum and PTG logistics for Denver, and continues with constant communication during the cycle. I am aware that I am standing on the shoulders of giants. I'm not trying to do anything brilliantly new here. I want to follow the lead of my predecessors and continue to tap them for their expertise and experience to help Nova succeed in Train. Thank you for your consideration. Eric Fried (efried) (rhymes with 'deed', not with 'side') [1] https://review.openstack.org/#/c/641177/

2 1