- openstack-discuss - lists.openstack.org

about the server group GET API
by Jiatong Shen 07 Aug '25

07 Aug '25

Hello experts, I have a question about the server group GET rest API. I tried out the GET API and found it was pretty slow (around 200 servers returned in 8 seconds). By skimming through the codebase I am suspecting filtering out group members is heavy ( https://github.com/openstack/nova/blob/4100d4d8fbaa6416ea37d8cb3e64003f40ee… ). So my question is do we clear out the instance_group_member table after deleting instances? If not, why could we not do it? Thank you very much for help. . -- Best Regards, Jiatong Shen

1 0

[all][elections][ptl][tc] Combined PTL/TC 2026.1 cycle Election Season
by Slawomir Kaplonski 07 Aug '25

07 Aug '25

Election details: https://governance.openstack.org/election/ The nomination period officially begins Aug 06, 2025 23:45 UTC. Please read the stipulations and timelines for candidates and electorate contained in these governance documents[1][2][3]. Due to circumstances of timing, PTL and TC elections for the coming cycle will run concurrently; deadlines for their nomination and voting activities are synchronized but will still use separate ballots. Please note, if only one candidate is nominated as PTL for a project team during the PTL nomination period, that candidate will win by acclaim, and there will be no poll. There will only be a poll if there is more than one candidate stepping forward for a project team's PTL position. Similarly, there will only be a TC poll if there are more than 4 candidates stepping forward the available TC positions. All members of the community are encouraged to confirm their email address as described at: https://governance.openstack.org/election/#electorate All candidates and prospective voters need to make sure they have reestablished their individual memberships in the OpenInfra Foundation [4], otherwise their nominations won't be verified and ballots will not be sent. There will be further announcements posted to the mailing list as action is required from the electorate or candidates. This email is for information purposes only. If you have any questions which you feel affect others please reply to this email thread. If you have any questions that you wish to discuss in private please email any of the election officials[5] so that we may address your concerns. Thank you, [1] https://governance.openstack.org/tc/reference/charter.html [2] https://governance.openstack.org/tc/reference/role-of-the-tc.html [3] https://docs.openstack.org/project-team-guide/ptl.html [4] https://lists.openinfra.org/archives/list/foundation@lists.openinfra.org/me… [5] https://governance.openstack.org/election/#election-officials -- Best regards Slawek Kaplonski skaplons(a)redhat.com

1 0

[keystone] User Impersonation feature
by Bram Kranendonk 06 Aug '25

06 Aug '25

Hi folks, I’m looking for a way to execute actions using Admin User X for Regular User Y. I came across the OS-TRUST extension with the impersonation flag, but this does not allow for an admin user to create trusts as another trustor. Are there other ways to accomplish such feature in Keystone? Thanks in advance, . Bram Kranendonk OpenStack Engineer

2 1

[openstack-helm] Nominate Doug Goldstein (cardoe) as a core reviewer
by Vladimir Kozhukalov 06 Aug '25

06 Aug '25

Hi team, I am writing to nominate Doug Goldstein (cardoe) as a member of the Openstack-helm core reviewers team. He's been actively contributing to the project during the last few release cycles and brought many valuable ideas (both technical and organizational). I also think it is important that our project has different companies represented in this important role. Please let me know what you think. -- Best regards, Kozhukalov Vladimir

5 5

[all][ops][qa] Decommissioning RefStack and retiring related software
by Jeremy Stanley 05 Aug '25

05 Aug '25

The RefStack service at refstack.openstack.org has been unused for purposes of qualifying OpenStack trademark licenses[*] since March of last year, after a 2023 board resolution[**] relaxed the former requirement for it. Effective immediately, the service is being decommissioned along with any associated resources, and the related Git repositories in OpenDev (interop, refstack, refstack-client) will be officially retired in the near future as well. Let me know if you have any questions or concerns. [*] https://openinfra.org/legal/trademark-policy [**] https://board.openinfra.org/meetings/minutes/05122023BoardMinutes -- Jeremy Stanley on behalf of the OpenInfra Foundation

4 3

[openstack-helm] GitOps Compatibility with ArgoCD
by aravindh.murugesan＠protonmail.com 05 Aug '25

05 Aug '25

Hello everyone, Hope you're all doing well. I’ve recently started exploring OpenStack-Helm to evaluate its suitability for our long-term needs, and one of the key aspects we’re considering is GitOps integration. While testing a deployment of the Keystone chart using ArgoCD, I noticed that the init container for Keystone waits for certain jobs (like db-init and bootstrap) to complete. However, ArgoCD doesn’t seem to trigger these jobs as expected—likely because it’s still waiting for the pods to become ready. I came across earlier discussions from Doug and Vlad around this topic during a previous PTG, but I wanted to bring it up again in case there have been any updates or ideas on making OpenStack-Helm charts more compatible with GitOps tools like ArgoCD or Flux. I really appreciate the effort and time that this community puts into maintaining and improving these charts. Any guidance or suggestions on how we might adapt this workflow would be incredibly helpful. Thanks so much in advance! Best regards, Aravindh

1 0

[kolla][barbican] Secure password management in Kolla-Ansible using Castellan
by elnazcloud＠gmail.com 05 Aug '25

05 Aug '25

Hi all, I have deployed an OpenStack environment using Kolla Ansible. I'm looking for a way to avoid storing service passwords (e.g., database, keystone, etc.) in clear text inside the configuration files. Is there any recommended method to integrate Barbican and Castellan so that services like Nova, Keystone, or Cinder can fetch secrets securely during runtime using oslo.config? If not natively supported, are there best practices or known workarounds for achieving this securely in a Kolla-based deployment? Thanks in advance!

7 12

[Zed][Neutron][OVN] Severe VM Network Performance Degradation Compared to Host
by Jeff Yang 05 Aug '25

05 Aug '25

Hi all, I'm running OpenStack Zed with OVN as the Neutron backend. I've encountered a severe VM network performance degradation issue and would appreciate any insights from the community. 🔹 **Environment** - OpenStack version: Zed - Neutron backend: OVN - Two VMs (each 4 vCPUs, 8 GB RAM) launched on two different compute nodes - Multi-queue is enabled on the VM vNICs 🔹 **Issue** The network performance inside the VMs is significantly worse than on the host nodes, with high packet retransmissions observed. 🔹 **Test results** **Host-to-Host (VTEP IPs)** ``` $ iperf3 -c 192.168.152.152 Connecting to host 192.168.152.152, port 5201 [ 5] local 192.168.152.153 port 45352 connected to 192.168.152.152 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.38 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 1.00-2.00 sec 1.37 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 2.00-3.00 sec 1.42 GBytes 12.2 Gbits/sec 0 3.10 MBytes [ 5] 3.00-4.00 sec 1.39 GBytes 11.9 Gbits/sec 0 3.10 MBytes [ 5] 4.00-5.00 sec 1.38 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 5.00-6.00 sec 1.43 GBytes 12.3 Gbits/sec 0 3.10 MBytes [ 5] 6.00-7.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 7.00-8.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 8.00-9.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 9.00-10.00 sec 1.42 GBytes 12.2 Gbits/sec 0 3.10 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 14.0 GBytes 12.0 Gbits/sec 0 sender [ 5] 0.00-10.04 sec 14.0 GBytes 12.0 Gbits/sec receiver iperf Done. ``` **VM-to-VM (through overlay network)** ``` $ iperf3 -c 10.0.6.10 Connecting to host 10.0.6.10, port 5201 [ 5] local 10.0.6.37 port 56710 connected to 10.0.6.10 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 499 MBytes 4.19 Gbits/sec 263 463 KBytes [ 5] 1.00-2.00 sec 483 MBytes 4.05 Gbits/sec 467 367 KBytes [ 5] 2.00-3.00 sec 482 MBytes 4.05 Gbits/sec 491 386 KBytes [ 5] 3.00-4.00 sec 483 MBytes 4.05 Gbits/sec 661 381 KBytes [ 5] 4.00-5.00 sec 472 MBytes 3.95 Gbits/sec 430 391 KBytes [ 5] 5.00-6.00 sec 480 MBytes 4.03 Gbits/sec 474 350 KBytes [ 5] 6.00-7.00 sec 510 MBytes 4.28 Gbits/sec 567 474 KBytes [ 5] 7.00-8.00 sec 521 MBytes 4.37 Gbits/sec 565 387 KBytes [ 5] 8.00-9.00 sec 509 MBytes 4.27 Gbits/sec 632 483 KBytes [ 5] 9.00-10.00 sec 514 MBytes 4.30 Gbits/sec 555 495 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 4.84 GBytes 4.15 Gbits/sec 5105 sender [ 5] 0.00-10.05 sec 4.84 GBytes 4.14 Gbits/sec receiver iperf Done. ``` As you can see, there's a sharp drop in performance inside the VMs (from ~12 Gbps down to ~4 Gbps), with a lot of retransmissions in 10 seconds. Has anyone encountered a similar issue with OVN and VM networking performance? What could be the potential causes? Any suggestions or debugging tips are welcome. Thanks in advance! Best regards,

5 4

[openstack-ansible] Linux bridge support
by Rambo Rambo 05 Aug '25

05 Aug '25

Hi Team We are currently having openstack-ansible Antelope (2023.1) release and have linux bridge currently. We are planning SLUP upgrades from Antelope to Caracal, and than from Caracal to Epoxy. Please confirm if we can still use linux bridge and continue with the upgrades or we need to mandatory move away from linux bridge as linux bridge is now experimental feature before doing the upgrades. We are planning to move away from linux bridge after upgrading to Epoxy release. Please let me know potential issues that can arise with this plan. Regards Rambo

3 2

[neutron] Bug Deputy Report July 28 - Aug 03
by Yatin Karel 05 Aug '25

05 Aug '25

Hi Neutrinos, Below is the bug deputy report for last week:- High ==== - https://bugs.launchpad.net/neutron/+bug/2119097 - [OVN] The OVN agent "metadata" extension is registering the OVN Metadata agent tags Assigned to Rodolfo Alonso - Fix Released - https://bugs.launchpad.net/neutron/+bug/2119092 - [OVN] The ``SbGlobalUpdateEvent`` event in the OVN agent should implement LP#1991817 Assigned to Rodolfo Alonso - Fix Released - https://bugs.launchpad.net/neutron/+bug/2119453 - [FT][master] test_ovs_and_ovs_events fails randomly Unassigned Medium ====== https://bugs.launchpad.net/neutron/+bug/2119093 - [OVN] The ``OVNExtensionEvent`` class is hardcoded to work only with the "metadata" extension Assigned to Rodolfo Alonso - Fix Released Invalid ===== https://bugs.launchpad.net/neutron/+bug/2119294 - [Caracal][DNSaaS] - dns_domain is empty in port show Was related to mis configurations -- Thanks and Regards Yatin Karel

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-8)
by Goutham Pacha Ravi 05 Aug '25

05 Aug '25

Hello Stackers, We're eight weeks away from wrapping up the 2025.2 "Flamingo" release, and three weeks away from its feature freeze (August 28, 2025) [1]. Over the past week, no new governance changes were merged, but some new changes are undergoing community review. The nomination window for Project Team Leads and for four seats on the OpenStack Technical Committee begins on August 6, 2025 [2]. If you have any questions about the process, please post them on this mailing list or hop into #openstack-tc, #openstack-election, or #openstack-dev on OFTC and let's chat. This week, the TC is also looking to refresh the liaisons for projects governed by the "distributed-project-leadership" model—an opportunity for those teams to switch their governance styles, as well. === Weekly Meeting === The TC's weekly meeting focused on two major topics [3]: finalizing the definition of “affiliation” for TC election candidates and the next steps for retiring Monasca. On the affiliation front, we debated whether to reuse the OpenInfra bylaws language or simplify it for our needs. During the meeting, there was a consensus that the definition should focus on contributor intent and a visible employee-employer relationship without getting bogged down in legal technicalities. It's important to share our biases upfront for the sake of transparency and to strive toward the TC's affiliation diversity requirement. We are moving ahead with retiring the Monasca project. After earlier signals of interest in continuing maintenance, the project team now seems comfortable with retirement. The TC consulted this mailing list [4] to seek any further interest in the project. We'll work with the "unmaintained-core" group to clean up deliverables and code branches. We also touched on CI work to test OpenStack with Debian Bookworm and Trixie, which will help us make progress on the Eventlet removal community goal. Finally, a brief discussion ensued on OpenStack for AI, encouraging folks with architectural insight to join the Foundation's AI working group and its mailing list [5]. We wrapped with a shout out to the i18n interns and mentors for their recent progress [6]. The next weekly meeting of the TC is today, August 5, 2025, at 1700 UTC. This meeting will be hosted simultaneously over Meetpad and IRC. Please find the meeting's agenda and call details in the wiki [7]. I hope you'll be able to join us. === Governance Proposals === ==== Open for review ==== - Define "affiliation" within the context of the TC | https://review.opendev.org/c/openstack/governance/+/956024 - Add series names to runtimes doc | https://review.opendev.org/c/openstack/governance/+/955810 - Remove Monasca from active project | https://review.opendev.org/c/openstack/governance/+/953671 - os-test-images never releases | https://review.opendev.org/c/openstack/governance/+/954249 - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 === Upcoming Events === - 2025-08-06: Nominations open for OpenStack elections: https://governance.openstack.org/election/ - 2025-08-20: Nominations close for OpenStack elections - 2025-08-26: OpenInfra Days, Korea: https://2025.openinfradays.kr/ - 2025-08-28: Feature Freeze deadline, Milestone-3 of the Flamingo release [1] - 2025-08-29: Colombia OpenInfra User Group Meetup: https://www.meetup.com/colombia-openinfra-user-group/events/307096751/ - 2025-10-17: OpenInfra Summit, Paris-Saclay: https://summit2025.openinfra.org/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] 2026.1 OpenStack Elections: https://governance.openstack.org/election/ [3] TC Meeting IRC Log 2025-07-29: https://meetings.opendev.org/meetings/tc/2025/tc.2025-07-29-17.00.html [4] Call for maintainers for Monasca: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [5] OpenInfra AI WG List: https://lists.openinfra.org/mailman3/lists/ai-openstack-wg.lists.openinfra.… [6] i18n Internship with CMU: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [7] TC Meeting Agenda, 2025-08-05: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[qa] Proposing Sean and Tosky to Devstack core
by Ghanshyam Maan 02 Aug '25

02 Aug '25

Hello Everyone, I would like to propose adding sean-k-mooney and tosky to Devstack core team. Both have shown great knowledge and have been engaged with Devstack for a long time. Having them in the core group would help us to continue maintaining Devstack. Feel free to provide your thoughts. If there are no objections by next week, I plan to add them to the Gerrit group by Friday, August 8th. -gmaan

2 1

[Cinder] Cinder NetApp driver not working as expected on NVMe over TCP
by Vincent Godin 01 Aug '25

01 Aug '25

Hello, We have NetApp C800 arrays for our OpenStack. We have set up the Cinder configuration to use the "NVMe over TCP" protocol. We are currently seeing the following issue: - the volumes (namespaces) are correctly mounted on the hypervisors - the multipath (native) configuration is correctly handled by Cinder (this can be seen in the logs) - the volumes are only attached by a single path Looking at the NetApp driver code, and more specifically at the *nvme_library.py* file and the *initialize_connection* function, we find on line 724: portal = (target_portals[0], self.NVME_PORT, self.NVME_TRANSPORT) data = { "target_nqn": str(target_nqn), "host_nqn": host_nqn, "portals": [portal], "vol_uuid": namespace_uuid } conn_info = {"driver_volume_type": "nvmeof", "data": data} If we look at the values of the target_portals[ ], the array does indeed have all the paths available for the targeted subsystem. However, the function only returns the first path : target_portals[0] I'm not a developer, so I can't guarantee that this is the cause of the problems encountered. What is certain is that manually connecting the namespace with the nvme command does indeed yield all four paths. I'll leave the existing discussion thread that led me to this observation open. Thank you. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Vincent Godin <vince.mlist(a)gmail.com> 29 juil. 2025 15:53 (il y a 3 jours) À Rajat, Sean, openstack-discuss Hello, Here are some of the results on the host. An instance is launched by Openstack on the compute nvme list Node Generic SN Model Namespace Usage Format FW Rev --------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- -------- /dev/nvme0n1 /dev/ng0n1 81O3QJXiLzBDAAAAAAAH NetApp ONTAP Controller 0x2 16.11 GB / 16.11 GB 4 KiB + 0 B FFFFFFFF if we have a look in the subsystem nvme list-subsys nvme-subsys0 - NQN=nqn.1992-08.com.netapp:sn.ec2c63655c3d11f0a40ad039eaba99f2:subsystem.openstack-79f1de4a-6645-4b47-9377-f06db6c2e0b5 hostnqn=nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9 iopolicy=round-robin \ +- nvme0 tcp traddr=10.10.184.3,trsvcid=4420,src_addr=10.10.184.33 live I have only one path I disconnect the subsystem manually nvme disconnect -n nqn.1992-08.com.netapp:sn.ec2c63655c3d11f0a40ad039eaba99f2:subsystem.openstack-79f1de4a-6645-4b47-9377-f06db6c2e0b5 NQN:nqn.1992-08.com.netapp:sn.ec2c63655c3d11f0a40ad039eaba99f2:subsystem.openstack-79f1de4a-6645-4b47-9377-f06db6c2e0b5 disconnected 1 controller(s) I reconnect to the subsystem with a manual command nvme connect-all -t tcp -a 10.10.186.3 nvme list Node Generic SN Model Namespace Usage Format FW Rev --------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- -------- /dev/nvme0n2 /dev/ng0n2 81O3QJXiLzBDAAAAAAAH NetApp ONTAP Controller 0x2 16.11 GB / 16.11 GB 4 KiB + 0 B FFFFFFFF And if we look at the subsystem nvme list-subsys nvme-subsys0 - NQN=nqn.1992-08.com.netapp:sn.ec2c63655c3d11f0a40ad039eaba99f2:subsystem.openstack-79f1de4a-6645-4b47-9377-f06db6c2e0b5 hostnqn=nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9 iopolicy=round-robin \ +- nvme5 tcp traddr=10.10.184.3,trsvcid=4420,src_addr=10.10.184.33 live +- nvme4 tcp traddr=10.10.186.3,trsvcid=4420,src_addr=10.10.186.33 live +- nvme3 tcp traddr=10.10.184.4,trsvcid=4420,src_addr=10.10.184.33 live +- nvme2 tcp traddr=10.10.186.4,trsvcid=4420,src_addr=10.10.186.33 live As you can see, i have four paths Configuration details about multipath : - in nova.conf [libvirt] volume_use_multipath = True - in cinder.conf [DEFAULT] target_protocol = nvmet_tcp ... [netapp-backend] use_multipath_for_image_xfer = True netapp_storage_protocol = nvme ... /sys/module/nvme_core/parameters/multipath cat /sys/module/nvme_core/parameters/multipath Y nova-compute.log grep -i get_connector_properties /var/log/kolla/nova/nova-compute.log 2025-07-29 14:09:51.553 7 DEBUG os_brick.initiator.connectors.lightos [None req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 4e2ddaf17ee747f2a1f03a392943f80a cb513debb0834ec5b6588356a960bad9 - - default default] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9 dsc: get_connector_properties /var/lib/kolla/venv/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:115 2025-07-29 14:09:51.553 7 DEBUG os_brick.utils [None req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 4e2ddaf17ee747f2a1f03a392943f80a cb513debb0834ec5b6588356a960bad9 - - default default] <== get_connector_properties: return (30ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.10.52.161', 'host': 'pkc-dcp-cpt-03', *'multipath': True, 'enforce_multipath': True*, 'initiator': 'iqn.2004-10.com.ubuntu:01:d0bb7aa9bcf1', 'do_local_attach': False, 'nvme_hostid': '5ca8b6d2-aa7d-42d8-bf74-c18484fab68c', 'system uuid': '31343550-3939-5a43-4a44-305930304c48', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9', *'nvme_native_multipath': True*, 'found_dsc': '', 'host_ips': ['10.20.128.33', '10.10.184.33', '10.10.186.33', '10.10.52.161', '10.10.22.161', '10.234.2.161', '10.10.50.161', '172.17.0.1', 'fe80::7864:3eff:fe13:5e1f', 'fe80::fc16:3eff:fe7f:3430', 'fe80::4c20:48ff:fe0f:2660']} trace_logging_wrapper /var/lib/kolla/venv/lib/python3.12/site-packages/os_brick/utils.py:204 multipathd systemctl status multipathd.service ○ multipathd.service Loaded: masked (Reason: Unit multipathd.service is masked.) Active: inactive (dead) If you can see some reason to explain why openstack connect to the subsystem only with one path !!! Thanks ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Rajat Dhasmana 30 juil. 2025 12:37 (il y a 2 jours) À moi, Sean, openstack-discuss On Wed, Jul 30, 2025 at 3:15 PM Vincent Godin <vince.mlist(a)gmail.com> wrote: > Hello guys, > > Some more informations found in the nova-compute.log : > > -try iscsi > > 2025-07-29 14:09:51.523 1222 DEBUG oslo_concurrency.processutils [-] > Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi execute > /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349 > 2025-07-29 14:09:51.528 1222 DEBUG oslo_concurrency.processutils [-] CMD > "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.005s execute > /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372 > 2025-07-29 14:09:51.528 1222 DEBUG oslo.privsep.daemon [-] privsep: > reply[90a51cdb-5701-4339-b059-fefb0b79b7a5]: (4, ('## DO NOT EDIT OR REMOVE > THIS FILE!\n## If you remove this file, the iSCSI daemon will not > start.\n## If you change the InitiatorName, existing access control > lists\n## may reject this initiator. The InitiatorName must be unique\n## > for each iSCSI initiator. Do NOT duplicate iSCSI > InitiatorNames.\nInitiatorName=iqn.2004-10.com.ubuntu:01:d0bb7aa9bcf1\n', > '')) _call_back > /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_privsep/daemon.py:503 > > -try lightos ??? > > 2025-07-29 14:09:51.552 7 DEBUG os_brick.initiator.connectors.lightos > [None req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 > 4e2ddaf17ee747f2a1f03a392943f80a cb513debb0834ec5b6588356a960bad9 - - > default default] LIGHTOS: [Errno 111] ECONNREFUSED find_dsc > /var/lib/kolla/venv/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:135 > 2025-07-29 14:09:51.553 7 INFO os_brick.initiator.connectors.lightos [None > req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 4e2ddaf17ee747f2a1f03a392943f80a > cb513debb0834ec5b6588356a960bad9 - - default default] Current host hostNQN > nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9 and > IP(s) are ['10.20.128.33', '10.10.184.33', '10.10.186.33', '10.10.52.161', > '10.10.22.161', '10.234.2.161', '10.10.50.161', '172.17.0.1', > 'fe80::7864:3eff:fe13:5e1f', 'fe80::fc16:3eff:fe7f:3430', > 'fe80::4c20:48ff:fe0f:2660'] > 2025-07-29 14:09:51.553 7 DEBUG os_brick.initiator.connectors.lightos > [None req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 > 4e2ddaf17ee747f2a1f03a392943f80a cb513debb0834ec5b6588356a960bad9 - - > default default] LIGHTOS: did not find dsc, continuing anyway. > get_connector_properties > /var/lib/kolla/venv/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112 > 2025-07-29 14:09:51.553 7 DEBUG os_brick.initiator.connectors.lightos > [None req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 > 4e2ddaf17ee747f2a1f03a392943f80a cb513debb0834ec5b6588356a960bad9 - - > default default] LIGHTOS: finally hostnqn: > nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9 dsc: > get_connector_properties > /var/lib/kolla/venv/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:115 > > -then > > 2025-07-29 14:09:51.553 7 DEBUG os_brick.utils [None > req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 4e2ddaf17ee747f2a1f03a392943f80a > cb513debb0834ec5b6588356a960bad9 - - default default] <== > get_connector_properties: return (30ms) {'platform': 'x86_64', 'os_type': > 'linux', 'ip': '10.10.52.161', 'host': 'pkc-dcp-cpt-03', 'multipath': True, > 'enforce_multipath': True, 'initiator': > 'iqn.2004-10.com.ubuntu:01:d0bb7aa9bcf1', 'do_local_attach': False, > 'nvme_hostid': '5ca8b6d2-aa7d-42d8-bf74-c18484fab68c', 'system uuid': > '31343550-3939-5a43-4a44-305930304c48', 'nqn': > 'nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9', > 'nvme_native_multipath': True, 'found_dsc': '', 'host_ips': > ['10.20.128.33', '10.10.184.33', '10.10.186.33', '10.10.52.161', > '10.10.22.161', '10.234.2.161', '10.10.50.161', '172.17.0.1', > 'fe80::7864:3eff:fe13:5e1f', 'fe80::fc16:3eff:fe7f:3430', > 'fe80::4c20:48ff:fe0f:2660']} trace_logging_wrapper > /var/lib/kolla/venv/lib/python3.12/site-packages/os_brick/utils.py:204 > 'multipath': True, 'enforce_multipath': True This shows that multipath configuration is set correctly. It would be good to search for this log entry[1] in cinder-volume logs and see the *portals *field to verify how many portals does the netapp nvme driver returns *Initialize connection info:* https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/netap… > 2025-07-29 14:09:51.554 7 DEBUG nova.virt.block_device [None > req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 4e2ddaf17ee747f2a1f03a392943f80a > cb513debb0834ec5b6588356a960bad9 - - default default] [instance: > 3fcb3e36-1890-44f7-9c3c-283c05e91910] Updating existing volume attachment > record: b81aea6e-f2ae-4781-8c2e-3b7f1606ba0d _volume_attach > /var/lib/kolla/venv/lib/python3.12/site-packages/nova/virt/block_device.py:666 > 2025-07-29 14:09:53.680 7 DEBUG os_brick.initiator.connectors.nvmeof [None > req-faf2b0ca-0709-4a70-8302-fa90ad293fd3 4e2ddaf17ee747f2a1f03a392943f80a > cb513debb0834ec5b6588356a960bad9 - - default default] ==> connect_volume: > call "{'self': <os_brick.initiator.connectors.nvmeof.NVMeOFConnector object > at 0x7cf65c576090>, 'connection_properties': {'target_nqn': > 'nqn.1992-08.com.netapp:sn.ec2c63655c3d11f0a40ad039eaba99f2:subsystem.openstack-79f1de4a-6645-4b47-9377-f06db6c2e0b5', > 'host_nqn': > 'nqn.2014-08.org.nvmexpress:uuid:629788a4-04c6-547c-9121-8d7a39c17fe9', > 'portals': [['10.10.184.3', 4420, 'tcp']], 'vol_uuid': > '69da9918-7e84-4ee4-b7bb-9b50e3e6d739', 'qos_specs': None, 'access_mode': > 'rw', 'encrypted': False, 'cacheable': False, 'enforce_multipath': True}}" > trace_logging_wrapper > /var/lib/kolla/venv/lib/python3.12/site-packages/os_brick/utils.py:177 > 'portals': [['10.10.184.3', 4420, 'tcp']] Here we can see that there is only one portal returned by the netapp driver

1 0

[Neutron] - General thoughts on tenancy network MTU
by Karl Kloppenborg 01 Aug '25

01 Aug '25

Hi Neutrinos, I was discussing with a colleague about our MTU settings within our Openstack regions, generally speaking we have a physnet_global_mtu of 1550 with tenancy networks automatically provisioned as 1500 VXLAN. This ends up with VM’s receiving DHCP MTU settings of 1500 which is fine, however reading about yesterday’s discussion on OVN performance, Nguyen’s recommendation was to enable Jumboframes. This got me thinking, what do you set your tenancy and phynet MTU’s to? Very interested to hear about your experiences with running 9k MTU’s on the VM networks etc, what impact this has on OVS/OVN routers and northbound connectivity being 1500. Thanks, Karl. Karl Kloppenborg Chief Technology Officer m: +61 437 239 565 resetdata.com<https://resetdata.com/> [cid:reset_69557fc2-1d63-4932-b5fd-93bd4f39ca7b.png] ResetData supports Mandatory Client Related Financial Disclosures – Scope 3 Emissions Reporting For more information on the phasing of these requirements for business please visit; https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…<https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…> This email transmission is intended only for the addressee / person responsible for delivery of the message to such person and may contain confidential or privileged information. Confidentiality and legal privilege are not waived or lost by reason of mistaken delivery to you, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. Whilst this email has been checked for viruses, the sender does not warrant that any attachments are free from viruses or other defects. You assume all liability for any loss, damage or other consequences which may arise from opening or using the attachments. If you received this e-mail in error please delete it and any attachments and kindly notify us by immediately sending an email to contact(a)resetdata.com.au<mailto:contact@resetdata.com.au>

2 1

(Ceilometer) Openstack - Kolla-Ansible deployment - Ceilometer (Central & compute) Docker Containers are unhealthy
by TRB PREM KUMAR 01 Aug '25

01 Aug '25

Hi, We have deployed an all-in-one OpenStack server for billing. Now ceilometer containers are unhealthy. Please help us to resolve this issue. Reference Output for reference: trb1@openstack-aio-01:~$ docker ps | grep ceil c53fc69ba42b quay.io/openstack.kolla/ceilometer-compute:2024.2-rocky-9 "dumb-init --single-…" About a minute ago Up About a minute *(unhealthy) * ceilometer_compute 9337af646e0f quay.io/openstack.kolla/ceilometer-central:2024.2-rocky-9 "dumb-init --single-…" 30 minutes ago Up About a minute ( *unhealthy)* ceilometer_central 554c4b8ccd86 quay.io/openstack.kolla/ceilometer-notification:2024.2-rocky-9 "dumb-init --single-…" 4 days ago Up About a minute (healthy) ceilometer_notification *** Logs files are attached for reference. Regards, TRB.

3 6

[neutron] Drivers meeting canceled Friday, August 1st
by Brian Haley 01 Aug '25

01 Aug '25

Hello Neutrinos: I am going to cancel tomorrow's Drivers meetings as there is no agenda. Just a reminder there are still open specs that need review, list is at [0]. Have a nice weekend! -Brian [0] https://review.opendev.org/q/status:open+project:openstack/neutron-specs

1 0

[scientific-sig][help] Research about private cloud
by Winicius Allan 01 Aug '25

01 Aug '25

Hi all! I'm about to write my final paper, and I want to research something related to OpenStack and private clouds. If someone wants to take the time to discuss the problems in your environment, I'd be very glad!What I had on my mind is: - There is something that could improve utilization and performance when the user is not charged about their use on a private cloud. - There is any process scheduling algorithm that could be used on specific workloads instead of Linux default scheduler that can improve application performance? I was thinking about using eBPF here. Best regards.

2 1

[cinder] AttributeError: 'Volume' object has no attribute 'use_quota'
by Kees Meijs | Nefos 31 Jul '25

31 Jul '25

Hello Stackers, It seems we're hitting a bug within the Cinder Volume service. The problem presents itself when creating a Ceph RBD backed Volume using an originating snapshot. > 2025-07-31 19:02:54.896 3602390 WARNING cinder.volume.manager > [req-a476ac04-ee6c-4335-8fca-e606616a5959 > eccdb27249254d9d9fb73570f5de8877 d678ba8e1dee4f7484555580d0a61562 - - > -] Task > 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' > (03c06b47-7df3-4427-8ad0-61f9a97d0159) transitioned into state > 'FAILURE' from state 'RUNNING' > 5 predecessors (most recent first): > </snip (maybe the Atoms are needed and if so I could redact those)> > |__Flow 'volume_create_manager': AttributeError: 'Volume' > object has no attribute 'use_quota' > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager Traceback > (most recent call last): > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", > line 53, in _execute_task > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager result = > task.execute(**arguments) > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", > line 1158, in execute > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > model_update = self._create_from_snapshot(context, volume, > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", > line 461, in _create_from_snapshot > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > model_update = self.driver.create_volume_from_snapshot(volume, > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/drivers/rbd.py", line > 1069, in create_volume_from_snapshot > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager if > int(volume.size): > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > AttributeError: 'Volume' object has no attribute 'use_quota' > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager Yes, this reflects a non-current version, to be precise Cinder 17.4. However, does this look familiar to someone, maybe? Thanks! Cheers, Kees

1 1

[Neutron] ML2: Can Operators Safely Extend binding:profile with Custom Keys?
by cxue22＠bloomberg.net 31 Jul '25

31 Jul '25

Hi all, I'm working on a Neutron customization to expose and manage data in the `binding:profile` field stored in the `ml2_port_bindings` table, and I have a few questions around operator and third-party plugin compatibility — particularly in reference to the blueprint: https://blueprints.launchpad.net/neutron/+spec/ml2-binding-profile As I understand it, the `binding:profile` is represented as a string column containing a JSON dictionary that can be passed by the compute agent and interpreted by ML2 mechanism drivers during port binding. My intent is to: - Create a custom API endpoint that allows reading and updating specific key/value pairs in this JSON dictionary. - Enable third-party mechanism drivers and operators to add custom fields to this profile. My main question is: **Are operators and third-party ML2 drivers free to add arbitrary key/value pairs to the `binding:profile` dictionary, and can those be expected to persist across Neutron upgrades and database migrations?** Additionally: - Are there any best practices or expectations around namespacing or schema validation of custom fields in `binding:profile`? - Are there known caveats or upgrade scenarios where such data may be truncated, filtered, or overwritten? I'd appreciate any guidance from the Neutron community on how this field is intended to be extended or preserved, and whether such customization is supported in the long term. Thanks in advance! Best regards, Chang Xue Bloomberg Engineering

3 3

[openstack][magnum]Cannot upgrade clusters
by Nguyễn Hữu Khôi 31 Jul '25

31 Jul '25

Hello. I cannot upgrade clusters via magnum_cluster_api. These are log from magnum_api: https://anotepad.com/notes/c9h8bd55 OPS: 2025.1 OS: Ubuntu 24.04 magnum_cluster_api: v0.30.0 Would you give some ideas on it? Thank you! Nguyen Huu Khoi

1 1

[Neutron][VPNaaS] Multiple tunnels per IPSec Service
by Shubham Yadav 31 Jul '25

31 Jul '25

Hello OpenStack Community, I'm working with OpenStack VPNaaS (using strongSwan as the backend driver) and have a question about tunnel configuration capabilities. *Current Setup:* OpenStack VPNaaS with strongSwan driver (StrongSwanDriver) Single VPN Service (gateway) with multiple IPSec Site Connections Each connection targets different remote peers with distinct subnets *Configuration Details:* VPN Service: Connected to a single router Multiple IPSec Site Connections under the same VPN Service: Connection 1: To MY office (10.10.100.0/24) via peer 68.x.y.34 Connection 2: To Rackspace (172.24.60.0/22) via peer 184.n.p.53 Shared IKE and IPSec policies across connections Local endpoint groups covering multiple subnets (192.168.0.0/24, 10.43.0.0/16) *Question:*Is it supported/recommended to have multiple IPSec Site Connections under a single VPN Service? I understand this creates multiple tunnels from the same OpenStack router to different remote peers. *Current Issue:*One of my connections shows "PENDING_CREATE" status while the other is "ACTIVE". I'm trying to determine if this is: A configuration issue on my end A limitation of having multiple tunnels per VPN service A strongSwan-specific behavior in the OpenStack context *Specific Questions:* Does VPNaaS officially support multiple IPSec connections per VPN Service? Are there any known limitations or best practices when using multiple tunnels? Should each remote site have its own dedicated VPN Service instead? I've checked the documentation but couldn't find clear guidance on multi-tunnel scenarios. Any insights from the community would be greatly appreciated. *Environment:* OpenStack version: 2024.2 Dalmatian Neutron VPNaaS with strongSwan backend Kubernetes-deployed OpenStack (OpenStack-Helm) Thank you for your time and assistance! Best regards, Shubham

1 0

[Neutron][VPNaaS] Multiple tunnels per IPSec Service
by Shubham Yadav 31 Jul '25

31 Jul '25

Hello OpenStack Community, I'm working with OpenStack VPNaaS (using strongSwan as the backend driver) and have a question about tunnel configuration capabilities. Current Setup: OpenStack VPNaaS with strongSwan driver (StrongSwanDriver) Single VPN Service (gateway) with multiple IPSec Site Connections Each connection targets different remote peers with distinct subnets Configuration Details: VPN Service: Connected to a single router Multiple IPSec Site Connections under the same VPN Service: Connection 1: To MY office (10.10.100.0/24) via peer 68.x.y.34 Connection 2: To Rackspace (172.24.60.0/22) via peer 184.n.p.53 Shared IKE and IPSec policies across connections Local endpoint groups covering multiple subnets (192.168.0.0/24, 10.43.0.0/16) Question: Is it supported/recommended to have multiple IPSec Site Connections under a single VPN Service? I understand this creates multiple tunnels from the same OpenStack router to different remote peers. Current Issue: One of my connections shows "PENDING_CREATE" status while the other is "ACTIVE". I'm trying to determine if this is: A configuration issue on my end A limitation of having multiple tunnels per VPN service A strongSwan-specific behavior in the OpenStack context Specific Questions: Does VPNaaS officially support multiple IPSec connections per VPN Service? Are there any known limitations or best practices when using multiple tunnels? Should each remote site have its own dedicated VPN Service instead? I've checked the documentation but couldn't find clear guidance on multi-tunnel scenarios. Any insights from the community would be greatly appreciated. Environment: OpenStack version: 2024.2 Dalmatian Neutron VPNaaS with strongSwan backend Kubernetes-deployed OpenStack (OpenStack-Helm) Thank you for your time and assistance! Best regards, Shubham

1 0

[Ovs-dpdk]Issue with ovs-dpdk deployment in Kolla-Ansible (zed)
by samira_darvishnia＠yahoo.com 31 Jul '25

31 Jul '25

Hi all, I'm trying to deploy OVS-DPDK using Kolla-Ansible Zed, and I'm running into a blocking issue during deployment. I would really appreciate your insights or suggestions. Problem Summary: During the deployment of ovs-dpdk, the container ovsdpdk-vswitchd fails to start, with the following error in its logs: + echo 'Running command: '\''/usr/sbin/ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/kolla/openvswitch/ovs-vswitchd.log --pidfile'\''' + exec /usr/sbin/ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/kolla/openvswitch/ovs-vswitchd.log --pidfile Running command: '/usr/sbin/ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/kolla/openvswitch/ovs-vswitchd.log --pidfile' ovs-vswitchd: invalid option -- 'w' ovs-vswitchd: invalid option -- 'w' EAL: FATAL: Invalid 'command line' arguments. 2025-07-30T13:44:59Z|00017|dpdk|EMER|Unable to initialize DPDK: Invalid argument ovs-vswitchd: Cannot init EAL (Invalid argument) This appears to be related to an incorrect parsing of arguments for ovs-vswitchd. What I’ve tried: I already applied this patch (as referenced in Gerrit): https://review.opendev.org/c/openstack/kolla-ansible/+/913653/1/ansible/rol… However, the issue still persists. Additional context: • Deployment hangs on the task: name: Wait for dpdk tunnel ip It waits for 5 minutes and then fails. • The parameter dpdk_tunnel_interface_address is already set in my global.yml. • I’ve configured and bound NICs to vfio-pci. • Environment info: • OpenStack Zed • Kolla-Ansible Zed • dpdk_tunnel_interface_address is set • OVS-DPDK is containerized via Kolla Has anyone seen this issue before or can suggest further steps? Any insight on how to correctly pass EAL arguments or what else might be misconfigured would be very helpful. Thanks in advance, Samira

1 0

[Neutron][Tacker][Issue Investigation] Issue with Neutron in Tacker Zuul Patches �C Suspected Regression in Neutron Master Branch
by Renu Rani 31 Jul '25

31 Jul '25

Hi Neutron Team, We are currently facing an issue in Tacker Zuul patch jobs build (with master branch), where the DevStack setup fails during the initial Neutron network creation step. Problem statement: The failure occurs at this point in the stack.sh script, for devstack environment creation in build process. Below command execution is failing in stack.sh [openstack --os-cloud devstack-admin-demo --os-region RegionOne router add subnet 20b264df-5f19-4599-a83d-2e76e7cbf5dc d777d04b-93ff-4e94-b6a9-cc8de3e23921] Error Message: This results in a 500 Internal Server Error, as seen in the logs: [HttpException: 500: Server Error for url: http://10.x.x.x/networking/v2.0/routers/20b264df-5f19-4599-a83d-2e76e7cbf5d…] Analysis for the issue: Below error is coming in Neutron API logs: [ERROR neutron.plugins.ml2.ovo_rpc RuntimeError: cannot notify on un-acquired lock] 1. Tried overriding the required-projects in Zuul to use stable/2025.1 branches (instead of master) to identify project(master branch) which is causing root cause. 2. Tacker team has created DNM patch (952892) to invetsigate this in detail. https://review.opendev.org/c/openstack/tacker/+/952892 1. In DNM patch Patchset 11 (on June 29), we have taken devstack stable/2025.1. In stable branch the devstack environment created sucessfully. 2. In DNM patch Patchset 25 (on July 8), with the same devstack override (stable/2025.1), the job now fails earlier during DevStack setup. The logs show that Neutron failed to start, causing the job to fail across all nodes. As per observations and anlysis seems the recent changes in the Neutron master branch between June 29 and July 8 might be causing this issue. We’d appreciate your help in reviewing the changes in Neutron's master branch during that window that could cause this behavior and suggesting any debugging approaches or patches we can test on tacker. Please let us know if you'd like additional logs or info from the Zuul job runs. Thanks in advance for your support! Thanks & Regards, Renu Rani (レヌラニ) Mobile: +91-9311888962 E-mail: renu.rani(a)india.nec.com<mailto:renu.rani@india.nec.com> The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. It shall not attach any liability on the originator or NEC Corporation India Private Limited or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of NEC Corporation India Private Limited or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.

5 10

[neutron] Is NVIDIA ASAP2 offloading with OpenStack just a myph?
by Vasyl Saienko 31 Jul '25

31 Jul '25

Hello OpenStack community, I'm trying to set up OpenStack with Nvidia ASAP2 hardware offloading in switchdev mode according to OpenStack guidelines <https://docs.openstack.org/neutron/latest/admin/config-ovs-offload.html> and Nvidia guidelines <https://docs.nvidia.com/doca/sdk/ovs-kernel+hardware+offloads/index.html> on the ConnectX6DX. I've tried different network types VLAN and VXLAN but offloading for none of them works. From an OpenStack standpoint everything is configured according to documentation, representor device is created and plugged in the br-int bridge (eth14) physical. PF enp33s0f0np0 is set to switchdev mode ``` devlink dev eswitch show pci/0000:21:00.0 pci/0000:21:00.0: mode switchdev inline-mode none encap-mode basic ``` ``` root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl show 12e00cdb-6520-4666-908f-57f59e8b9bd4 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-ex Interface br-ex type: internal Port pr-floating Interface pr-floating Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Port int-br-tenant Interface int-br-tenant type: patch options: {peer=phy-br-tenant} Port tap58ac2b17-fb tag: 1 Interface tap58ac2b17-fb Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} Port eth14 tag: 1 Interface eth14 Bridge br-tenant Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-tenant Interface br-tenant type: internal Port phy-br-tenant Interface phy-br-tenant type: patch options: {peer=int-br-tenant} Port enp33s0f0np0 Interface enp33s0f0np0 Bridge br-tun Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} ovs_version: "3.0.0-0056-25.04-based-3.3.5" ``` But hardware offloading is not happening, tcpdump on eth14 still shows all packets, and no TC rules are created. OpenStack uses complex openvswitch rules to setup virtual networking, in this scenario vlan tagging is done by the following rule ``` cookie=0x45b91976d5c4f175, duration=233.443s, table=60, n_packets=314, n_bytes=35574, priority=100,in_port=eth14 actions=set_field:0x11->reg5,set_field:0x1->reg6,resubmit(,73) ``` If we simplify openflow rules and align them with Nvidia guidelines for openvswitch hardware offloading by plugging representor device directly into physnet bridge, and assign vlan tag as port attribute hardware offloading starts working. ``` root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl del-port eth14 root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl add-port br-tenant eth14 root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl set port eth14 tag=1443 ``` ``` root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# for i in $(ip link | awk '{print $2}' |grep -e enp -e ens |grep -v vf$); do tc filter show dev $i ingress; done filter protocol 802.1Q pref 6 flower chain 0 filter protocol 802.1Q pref 6 flower chain 0 handle 0x1 vlan_id 1443 vlan_prio 0 vlan_ethtype arp dst_mac fa:16:3e:36:1b:90 src_mac fa:16:3e:39:90:d8 eth_type arp in_hw in_hw_count 1 action order 1: vlan pop pipe index 1 ref 1 bind 1 no_percpu used_hw_stats delayed action order 2: mirred (Egress Redirect to device eth14) stolen index 1 ref 1 bind 1 cookie 4b5b5a701f4e3fee7ac1c198037522e3 no_percpu used_hw_stats delayed ``` Similar issue occurs when using the VXLAN network. OpenStack setup VXLAN tunnel like, where tunnel_id is inserted dynamically via openflow rules (in_key=flow) ``` Port vxlan-0a0c0004 Interface vxlan-0a0c0004 type: vxlan options: {df_default="true", dst_port="4790", egress_pkt_mark="0", in_key=flow, local_ip="10.12.0.9", out_key=flow, remote_ip="10.12.0.4"} ``` In this configuration hardware offloading is not happening neither. But when setting up vxlan according to nvidia guidelines (no complex rules are involved) hardware offloading starts working (tcpdump for representor device shows only first packet and tc rules are being created). ``` Port vxlan3 Interface vxlan3 type: vxlan options: {key="60987", local_ip="10.12.0.9", remote_ip="10.12.0.4"} ``` ``` Cannot find device "enp33s0f0npf0vf" Cannot find device "veth68d427d@if1" filter protocol ip pref 2 flower chain 0 filter protocol ip pref 2 flower chain 0 handle 0x1 dst_mac fa:16:3e:65:0c:6d src_mac 52:5a:d0:9e:8a:32 eth_type ipv4 enc_dst_ip 10.12.0.9 enc_src_ip 10.12.0.8 enc_key_id 60987 enc_dst_port 4789 enc_tos 0 ip_flags nofrag in_hw in_hw_count 2 action order 1: tunnel_key unset pipe index 2 ref 1 bind 1 no_percpu used_hw_stats delayed action order 2: mirred (Egress Redirect to device eth15) stolen index 2 ref 1 bind 1 cookie 5d44cdf6754fc237cc0bd688a6fe70a8 no_percpu used_hw_stats delayed ``` Am I doing something wrong? Is there anyone who has other results? -- <https://www.mirantis.com/> Vasyl Saienko Principal DevOps Engineer vsaienko(a)mirantis.com <dstoltenberg(a)mirantis.com> +(380) 66 072 07 17 <++1+(650)+564+7038>

3 3