July 2025 - openstack-discuss - lists.openstack.org

[kolla-ansible][rabbitmq] RabbitMQ cluster creation took too long
by Nguyễn Hữu Khôi 10 Feb '26

10 Feb '26

Hello. I am deploying OpenStack 2025.1 with Kolla-Ansible 2025.1 (master branch) using the latest images. However, when I created a RabbitMQ(4.0.x) cluster with 3 nodes, it took too long—around 12 minutes(it stucked at DB: virgin node -> run peer discovery). I then changed the images to RabbitMQ version 3.13.7, and it worked well. My OS: Ubuntu 24.01 Nguyen Huu Khoi

2 2

[openstack][octavia]Resize a Load Balancer
by Nguyễn Hữu Khôi 28 Jan '26

28 Jan '26

Hello. I see that patch https://review.opendev.org/c/openstack/octavia/+/890215 Could I ask why we cannot merge it? Nguyen Huu Khoi

3 6

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 08 Dec '25

08 Dec '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 16

[Octavia][Load Balancer][2023.1] Amphora Fails Health Check – “No Route to Host” & Timeout Errors
by Thamanna Farhath 02 Dec '25

02 Dec '25

Hi OpenStack Community, I am currently facing an issue with the Load Balancer (Octavia) service in my OpenStack deployment and would appreciate your guidance or suggestions on how to troubleshoot and resolve it. Environment Details: OpenStack Version: 2023.1 (Antelope) Operating System: Ubuntu 22.04 Octavia Backend: Amphora Network Type: VXLAN Deployment Tool: Kolla Ansible Issue Summary: After creating a Load Balancer, the Amphora instance is successfully launched. However, approximately 18 minutes later, the Load Balancer transitions to an ERROR state. Error Observed: octavia.common.exceptions.ComputeWaitTimeoutException: Waiting for compute id a222527f-8336-4fb1-8e9e-0e3ac6e8d580 to go active timeout. octavia.amphorae.driver_exceptions.exceptions.AmpConnectionRetry: Could not connect to amphora, exception caught: HTTPSConnectionPool(host='192.168.200.247', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError: Failed to establish a new connection: [Errno 113] No route to host) What I’ve Tried: Created a VXLAN-based internal network: openstack subnet create --network octavia-mgmt-net --subnet-range 192.168.200.0/24 octavia-mgmt-subnet openstack router create octavia-router openstack router add subnet octavia-router octavia-mgmt-subnet openstack router set octavia-router --external-gateway <public-network-id> Manually configured OVS interface: ovs-vsctl add-port br-int octavia-mgmt -- set interface octavia-mgmt type=internal ip addr add 192.168.200.1/24 dev octavia-mgmt ip link set octavia-mgmt up Security groups are open (allowing all ingress and egress). Verified OVS bridge setup – interface appears correctly on br-int. Behavior Noticed: Amphora instance gets created but fails to communicate with the controller after a while,so health checks fails. The controller cannot reach the Amphora management IP (192.168.200.x). Logs show heartbeat/connection issues (No route to host, timeouts). Request for Help: I would appreciate any of the following: Suggestions to verify if the management network is properly routed from controller to Amphora. Best practices for setting up the Octavia management network using VXLAN. Troubleshooting steps for verifying health manager connectivity to Amphora. Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

4 3

LXB Removal and OVN migration
by Ricardo Cano 10 Nov '25

10 Nov '25

Hello openstack-discuss, With the removal of the linuxbridge mechanism driver, has anyone successfully completed an in-place migrationfrom LinuxBridge to OVN? I did come across this https://www.jimmdenton.com/migrating-lxb-to-ovn/ and that looks good as a one-shoit while biting the bullet and taking a full outage. However, I was wondering if it was possible to run lxb and ovn side-by-side and cut them over, say one network at a time. This particular deployment was stood up using openstack-ansible back in the rocky days (now running antelope) and it appears that openstack ansible's default was lxb up until Zed. Any suggestions or recommendations on the best path forward? Also, will the dynamic routing agent work with ovn?

4 3

[openstack helm] [Glance] Glance showing OSError: unable to receive chunked part when uploading image
by daniel890723＠gmail.com 08 Nov '25

08 Nov '25

Hello i'm testing using Openstack-helm to install all component but after the installation complete i can't upload the OS image through both openstack CLI or Horizon it stuck at Queueing and when i checking Glance Api log it shows 2024-11-07 07:31:19.362 7 INFO glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2024-11-07 07:31:19.400 7 ERROR glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Failed to upload image data due to internal error: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Caught error: unable to receive chunked part: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Traceback (most recent call last): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1302, in __call__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1345, in dispatch 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return method(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 415, in wrapped 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 299, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 162, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 492, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 443, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/quota/__init__.py", line 322, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 596, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 487, in _upload_to_store 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 397, in add_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 479, in store_add_to_backend_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/driver.py", line 294, in add_adapter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/capabilities.py", line 176, in op_checker 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 764, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise errors.get(e.errno, e) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 746, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi for buf in utils.chunkreadable(image_file, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/common/utils.py", line 73, in chunkiter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = fp.read(chunk_size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 294, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = self.data.read(i) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 121, in readfn 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = fd.read(*args) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/format_inspector.py", line 956, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = self._source.read(size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1045, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi data = uwsgi.chunked_read() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Ceph Cluster Status: NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID ceph /var/lib/rook 3 82m Ready Cluster created successfully HEALTH_OK 5cf8b9bb-13e1-4c31-b47d-9e7d92c3c05d Ceph osd status: ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS TYPE NAME -1 0.35156 - 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 1.00 - root default -5 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 - host instance-lu-1 2 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 195 up osd.2 -3 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-2 0 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 195 up osd.0 -7 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-3 1 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 197 up osd.1 TOTAL 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 MIN/MAX VAR: 1.00/1.01 STDDEV: 0.01 now i'm able to create Empty Volume using horizon(can't create volume using default cirros image ) and i can start an Instance without volume and mount the empty volume in(Ceph also can detect when i use that volume) is there something i need to set when starting glance? because i'm following the Openstack-helm installation document to install openstack with helm

5 8

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 30 Oct '25

30 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

[openstack][nova]qemu system wont release memory
by Nguyễn Hữu Khôi 29 Sep '25

29 Sep '25

Hello. I have an OpenStack instance that was using 14GB out of 16GB of memory. Later, when it was using only 4GB, I checked the QEMU process on the compute node, and it was still consuming 14GB, this means memory isn't released when usage decreases. Do we have solutions for this? I use Ubuntu 24.04 instances. Nguyen Huu Khoi

2 4

[Neutron][operators] LinuxBridge to OVN Outreach
by Daniel Failing 29 Sep '25

29 Sep '25

Hi all, Following on the good session we had last time discussing migrating out of LinuxBridge between operators, it was proposed to meet again in June. To find a good time slot, please respond to the following survey: https://newdle.cern.ch/newdle/TtbH8Dbv For reference, the notes from last meeting: https://etherpad.opendev.org/p/neutron-lb-ovn-migration Best, Daniel Failing CERN Cloud Infrastructure

2 3

[cinder] IBM SVf driver callhome
by Vivek Pandey 10 Sep '25

10 Sep '25

Hi Joe/ Brian, Since we didn’t get a chance to discuss the patch [IBM SVf driver] Adding support for callhome plugin, we can continue the conversation here. Patch : https://review.opendev.org/c/openstack/cinder/+/951829. We need callhome enabled as default for which we have introduced a parameter “callhome” in cinder.conf. This feature is to fecth the os_version, cinder-manage version and ispbhaenabled. Note: We are not fetching any customer’s sensitive data. Please let me know if you’ve had a chance to review NetApp’s ActiveIQ implementation. Netapp’s ActiveIQ code ref : https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/netap… As we have discussed during the 9th July cinder call, Anoop from NetApp confirmed that they are enabling it by default and EMS details are being captured from the logs. Based on that, do you think we can align or extend our current IBM callhome implementation (from the patch above) to follow a similar approach? Thanks, Vivek

5 9

[openstack-magnum ui] Cannot choose an external network ID when creating a cluster template
by Nguyễn Hữu Khôi 27 Aug '25

27 Aug '25

Hello, When I create a cluster template, I cannot choose an external network ID, it disappears from horizon. My env: Ubuntu 22.04 Openstack 2024.2 Thank you. Regards Nguyen Huu Khoi

3 2

[blazar] Blazar-dashboard contains non-free files (ie: minified Javascripts)
by Thomas Goirand 22 Aug '25

22 Aug '25

Hi, I've been trying to package Blazar, all is already in Debian, but I can't get blazar-dashboard in, because upstream source contains non-free files. Namely: https://github.com/openstack/blazar-dashboard/blob/master/blazar_dashboard/… is a minifed version of a much more complex package: https://github.com/apexcharts/apexcharts.js/tree/v3.31.0 Reminder: minified Javascripts are considered by most distributions (this includes Red Hat, Ubuntu, and in my case Debian) as non-free files that cannot be shipped in the distribution without the underlying source code, and packages must build from the not minified version. Could the Blazar team work on un-vendoring apexcharts.min.js, possibly removing it completely, or replacing it with a source-full version? Cheers, Thomas Goirand (zigo)

4 6

[tc][all][monasca] Retiring the Monasca project and repositories
by Goutham Pacha Ravi 07 Aug '25

07 Aug '25

Hello Stackers, We intend to retire the Monasca project [1] and all associated repositories [2]. The retirement is being proposed with the support and consent of the current core team members and the Project Team Lead (PTL) [3]. For context, the Monasca project team was marked as being inactive during the 2024.1 (Caracal) release cycle [4]. Since then, no sustained development activity has occurred. There have been no releases tagged for any of the deliverables owned by the team either, and there are no actively maintained stable branches. So, in accordance with the procedures in the OpenStack governance documentation [5], and based on the current state of the project, we propose the following repositories ahead of the 2025.2 ("Flamingo") coordinated release: - openstack/monasca-agent - openstack/monasca-api - openstack/monasca-common - openstack/monasca-events-api - openstack/monasca-log-api - openstack/monasca-notification - openstack/monasca-persister - openstack/monasca-specs - openstack/monasca-tempest-plugin - openstack/monasca-thresh - openstack/python-monascaclient - openstack/monasca-ui - openstack/monasca-transform - openstack/monasca-ceilometer - openstack/monasca-grafana-datasource - openstack/monasca-kibana-plugin - openstack/monasca-statsd Some of the current maintainers intend to fork the project and continue development independently for their purposes. This retirement does not preclude such efforts; nor does it prevent reviving the project after the retirement. If there are any objections to retiring the project, please respond to this thread. In the event there is interest in continuing Monasca within OpenStack, we would expect the project team to meet the requirements for active status [6] — including identifying a new PTL for the upcoming release cycle and demonstrating plans for sustained maintenance. Thank you to everyone who has contributed to Monasca over the years. Thanks, On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://docs.openstack.org/monasca-api/latest/ [2] https://github.com/openstack/governance/blob/624bb5bf124452eed475c9b6e62144… [3] https://review.opendev.org/c/openstack/governance/+/953671 [4] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [5] https://docs.openstack.org/project-team-guide/repository.html#retiring-a-re… [6] https://governance.openstack.org/tc/reference/new-projects-requirements.html

3 2

[dev][designate] PSA: pkg_resources is broken with newer oslo wheels on Python 3.8
by Stephen Finucane 07 Aug '25

07 Aug '25

o/ A quick PSA for a topic has come up in a few different OpenStack forums of late. As described in a recently filed setuptools bug [1], the pkg_resources found in the most recent version of setuptools to support Python 3.8 does not correctly normalize package names passed to `pkg_resources.require`. This has become an issue of late because since earlier this year the OpenStack release tooling has started (correctly) normalizing filenames of the '.whl' files uploaded to PyPI (so e.g. 'oslo.messaging' now becomes 'oslo_messaging'). You are likely to see this manifest with weird errors like `The 'package.foo>=1.2.3' distribution was not found`, when you know that it is. I have seen it pop up because either (a) pkg_resources is being invoked via an explicit call, as in some of the release tooling used in `openstack/releases`, or (b) pkg_resources is being invoked via the auto-generated import headers added by setuptools when you use its legacy "scripts" keyword rather than newer "console scripts" replacement. The latter in particular is biting some projects indirectly due to Designate and Octavia incorrectly using this mechanism to work around the demise of wsgi_scripts, and I have fixes up waiting for review [2][3]. Just sharing in case some people have seen this, because it's quite a weird corner case to hit. Stephen [1] https://github.com/pypa/setuptools/issues/5051 [2] https://review.opendev.org/c/openstack/designate/+/950569/ [3] https://review.opendev.org/c/openstack/designate/+/955683/

3 3

[all][ops][qa] Decommissioning RefStack and retiring related software
by Jeremy Stanley 05 Aug '25

05 Aug '25

The RefStack service at refstack.openstack.org has been unused for purposes of qualifying OpenStack trademark licenses[*] since March of last year, after a 2023 board resolution[**] relaxed the former requirement for it. Effective immediately, the service is being decommissioned along with any associated resources, and the related Git repositories in OpenDev (interop, refstack, refstack-client) will be officially retired in the near future as well. Let me know if you have any questions or concerns. [*] https://openinfra.org/legal/trademark-policy [**] https://board.openinfra.org/meetings/minutes/05122023BoardMinutes -- Jeremy Stanley on behalf of the OpenInfra Foundation

4 3

[kolla][barbican] Secure password management in Kolla-Ansible using Castellan
by elnazcloud＠gmail.com 05 Aug '25

05 Aug '25

Hi all, I have deployed an OpenStack environment using Kolla Ansible. I'm looking for a way to avoid storing service passwords (e.g., database, keystone, etc.) in clear text inside the configuration files. Is there any recommended method to integrate Barbican and Castellan so that services like Nova, Keystone, or Cinder can fetch secrets securely during runtime using oslo.config? If not natively supported, are there best practices or known workarounds for achieving this securely in a Kolla-based deployment? Thanks in advance!

7 12

[Zed][Neutron][OVN] Severe VM Network Performance Degradation Compared to Host
by Jeff Yang 05 Aug '25

05 Aug '25

Hi all, I'm running OpenStack Zed with OVN as the Neutron backend. I've encountered a severe VM network performance degradation issue and would appreciate any insights from the community. 🔹 **Environment** - OpenStack version: Zed - Neutron backend: OVN - Two VMs (each 4 vCPUs, 8 GB RAM) launched on two different compute nodes - Multi-queue is enabled on the VM vNICs 🔹 **Issue** The network performance inside the VMs is significantly worse than on the host nodes, with high packet retransmissions observed. 🔹 **Test results** **Host-to-Host (VTEP IPs)** ``` $ iperf3 -c 192.168.152.152 Connecting to host 192.168.152.152, port 5201 [ 5] local 192.168.152.153 port 45352 connected to 192.168.152.152 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.38 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 1.00-2.00 sec 1.37 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 2.00-3.00 sec 1.42 GBytes 12.2 Gbits/sec 0 3.10 MBytes [ 5] 3.00-4.00 sec 1.39 GBytes 11.9 Gbits/sec 0 3.10 MBytes [ 5] 4.00-5.00 sec 1.38 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 5.00-6.00 sec 1.43 GBytes 12.3 Gbits/sec 0 3.10 MBytes [ 5] 6.00-7.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 7.00-8.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 8.00-9.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 9.00-10.00 sec 1.42 GBytes 12.2 Gbits/sec 0 3.10 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 14.0 GBytes 12.0 Gbits/sec 0 sender [ 5] 0.00-10.04 sec 14.0 GBytes 12.0 Gbits/sec receiver iperf Done. ``` **VM-to-VM (through overlay network)** ``` $ iperf3 -c 10.0.6.10 Connecting to host 10.0.6.10, port 5201 [ 5] local 10.0.6.37 port 56710 connected to 10.0.6.10 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 499 MBytes 4.19 Gbits/sec 263 463 KBytes [ 5] 1.00-2.00 sec 483 MBytes 4.05 Gbits/sec 467 367 KBytes [ 5] 2.00-3.00 sec 482 MBytes 4.05 Gbits/sec 491 386 KBytes [ 5] 3.00-4.00 sec 483 MBytes 4.05 Gbits/sec 661 381 KBytes [ 5] 4.00-5.00 sec 472 MBytes 3.95 Gbits/sec 430 391 KBytes [ 5] 5.00-6.00 sec 480 MBytes 4.03 Gbits/sec 474 350 KBytes [ 5] 6.00-7.00 sec 510 MBytes 4.28 Gbits/sec 567 474 KBytes [ 5] 7.00-8.00 sec 521 MBytes 4.37 Gbits/sec 565 387 KBytes [ 5] 8.00-9.00 sec 509 MBytes 4.27 Gbits/sec 632 483 KBytes [ 5] 9.00-10.00 sec 514 MBytes 4.30 Gbits/sec 555 495 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 4.84 GBytes 4.15 Gbits/sec 5105 sender [ 5] 0.00-10.05 sec 4.84 GBytes 4.14 Gbits/sec receiver iperf Done. ``` As you can see, there's a sharp drop in performance inside the VMs (from ~12 Gbps down to ~4 Gbps), with a lot of retransmissions in 10 seconds. Has anyone encountered a similar issue with OVN and VM networking performance? What could be the potential causes? Any suggestions or debugging tips are welcome. Thanks in advance! Best regards,

5 4

[Neutron] - General thoughts on tenancy network MTU
by Karl Kloppenborg 01 Aug '25

01 Aug '25

Hi Neutrinos, I was discussing with a colleague about our MTU settings within our Openstack regions, generally speaking we have a physnet_global_mtu of 1550 with tenancy networks automatically provisioned as 1500 VXLAN. This ends up with VM’s receiving DHCP MTU settings of 1500 which is fine, however reading about yesterday’s discussion on OVN performance, Nguyen’s recommendation was to enable Jumboframes. This got me thinking, what do you set your tenancy and phynet MTU’s to? Very interested to hear about your experiences with running 9k MTU’s on the VM networks etc, what impact this has on OVS/OVN routers and northbound connectivity being 1500. Thanks, Karl. Karl Kloppenborg Chief Technology Officer m: +61 437 239 565 resetdata.com<https://resetdata.com/> [cid:reset_69557fc2-1d63-4932-b5fd-93bd4f39ca7b.png] ResetData supports Mandatory Client Related Financial Disclosures – Scope 3 Emissions Reporting For more information on the phasing of these requirements for business please visit; https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…<https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…> This email transmission is intended only for the addressee / person responsible for delivery of the message to such person and may contain confidential or privileged information. Confidentiality and legal privilege are not waived or lost by reason of mistaken delivery to you, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. Whilst this email has been checked for viruses, the sender does not warrant that any attachments are free from viruses or other defects. You assume all liability for any loss, damage or other consequences which may arise from opening or using the attachments. If you received this e-mail in error please delete it and any attachments and kindly notify us by immediately sending an email to contact(a)resetdata.com.au<mailto:contact@resetdata.com.au>

2 1

(Ceilometer) Openstack - Kolla-Ansible deployment - Ceilometer (Central & compute) Docker Containers are unhealthy
by TRB PREM KUMAR 01 Aug '25

01 Aug '25

Hi, We have deployed an all-in-one OpenStack server for billing. Now ceilometer containers are unhealthy. Please help us to resolve this issue. Reference Output for reference: trb1@openstack-aio-01:~$ docker ps | grep ceil c53fc69ba42b quay.io/openstack.kolla/ceilometer-compute:2024.2-rocky-9 "dumb-init --single-…" About a minute ago Up About a minute *(unhealthy) * ceilometer_compute 9337af646e0f quay.io/openstack.kolla/ceilometer-central:2024.2-rocky-9 "dumb-init --single-…" 30 minutes ago Up About a minute ( *unhealthy)* ceilometer_central 554c4b8ccd86 quay.io/openstack.kolla/ceilometer-notification:2024.2-rocky-9 "dumb-init --single-…" 4 days ago Up About a minute (healthy) ceilometer_notification *** Logs files are attached for reference. Regards, TRB.

3 6

[scientific-sig][help] Research about private cloud
by Winicius Allan 01 Aug '25

01 Aug '25

Hi all! I'm about to write my final paper, and I want to research something related to OpenStack and private clouds. If someone wants to take the time to discuss the problems in your environment, I'd be very glad!What I had on my mind is: - There is something that could improve utilization and performance when the user is not charged about their use on a private cloud. - There is any process scheduling algorithm that could be used on specific workloads instead of Linux default scheduler that can improve application performance? I was thinking about using eBPF here. Best regards.

2 1

[cinder] AttributeError: 'Volume' object has no attribute 'use_quota'
by Kees Meijs | Nefos 31 Jul '25

31 Jul '25

Hello Stackers, It seems we're hitting a bug within the Cinder Volume service. The problem presents itself when creating a Ceph RBD backed Volume using an originating snapshot. > 2025-07-31 19:02:54.896 3602390 WARNING cinder.volume.manager > [req-a476ac04-ee6c-4335-8fca-e606616a5959 > eccdb27249254d9d9fb73570f5de8877 d678ba8e1dee4f7484555580d0a61562 - - > -] Task > 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' > (03c06b47-7df3-4427-8ad0-61f9a97d0159) transitioned into state > 'FAILURE' from state 'RUNNING' > 5 predecessors (most recent first): > </snip (maybe the Atoms are needed and if so I could redact those)> > |__Flow 'volume_create_manager': AttributeError: 'Volume' > object has no attribute 'use_quota' > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager Traceback > (most recent call last): > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", > line 53, in _execute_task > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager result = > task.execute(**arguments) > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", > line 1158, in execute > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > model_update = self._create_from_snapshot(context, volume, > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", > line 461, in _create_from_snapshot > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > model_update = self.driver.create_volume_from_snapshot(volume, > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/drivers/rbd.py", line > 1069, in create_volume_from_snapshot > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager if > int(volume.size): > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > AttributeError: 'Volume' object has no attribute 'use_quota' > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager Yes, this reflects a non-current version, to be precise Cinder 17.4. However, does this look familiar to someone, maybe? Thanks! Cheers, Kees

1 1

[Neutron] ML2: Can Operators Safely Extend binding:profile with Custom Keys?
by cxue22＠bloomberg.net 31 Jul '25

31 Jul '25

Hi all, I'm working on a Neutron customization to expose and manage data in the `binding:profile` field stored in the `ml2_port_bindings` table, and I have a few questions around operator and third-party plugin compatibility — particularly in reference to the blueprint: https://blueprints.launchpad.net/neutron/+spec/ml2-binding-profile As I understand it, the `binding:profile` is represented as a string column containing a JSON dictionary that can be passed by the compute agent and interpreted by ML2 mechanism drivers during port binding. My intent is to: - Create a custom API endpoint that allows reading and updating specific key/value pairs in this JSON dictionary. - Enable third-party mechanism drivers and operators to add custom fields to this profile. My main question is: **Are operators and third-party ML2 drivers free to add arbitrary key/value pairs to the `binding:profile` dictionary, and can those be expected to persist across Neutron upgrades and database migrations?** Additionally: - Are there any best practices or expectations around namespacing or schema validation of custom fields in `binding:profile`? - Are there known caveats or upgrade scenarios where such data may be truncated, filtered, or overwritten? I'd appreciate any guidance from the Neutron community on how this field is intended to be extended or preserved, and whether such customization is supported in the long term. Thanks in advance! Best regards, Chang Xue Bloomberg Engineering

3 3

[openstack][magnum]Cannot upgrade clusters
by Nguyễn Hữu Khôi 31 Jul '25

31 Jul '25

Hello. I cannot upgrade clusters via magnum_cluster_api. These are log from magnum_api: https://anotepad.com/notes/c9h8bd55 OPS: 2025.1 OS: Ubuntu 24.04 magnum_cluster_api: v0.30.0 Would you give some ideas on it? Thank you! Nguyen Huu Khoi

1 1

[Neutron][VPNaaS] Multiple tunnels per IPSec Service
by Shubham Yadav 31 Jul '25

31 Jul '25

Hello OpenStack Community, I'm working with OpenStack VPNaaS (using strongSwan as the backend driver) and have a question about tunnel configuration capabilities. *Current Setup:* OpenStack VPNaaS with strongSwan driver (StrongSwanDriver) Single VPN Service (gateway) with multiple IPSec Site Connections Each connection targets different remote peers with distinct subnets *Configuration Details:* VPN Service: Connected to a single router Multiple IPSec Site Connections under the same VPN Service: Connection 1: To MY office (10.10.100.0/24) via peer 68.x.y.34 Connection 2: To Rackspace (172.24.60.0/22) via peer 184.n.p.53 Shared IKE and IPSec policies across connections Local endpoint groups covering multiple subnets (192.168.0.0/24, 10.43.0.0/16) *Question:*Is it supported/recommended to have multiple IPSec Site Connections under a single VPN Service? I understand this creates multiple tunnels from the same OpenStack router to different remote peers. *Current Issue:*One of my connections shows "PENDING_CREATE" status while the other is "ACTIVE". I'm trying to determine if this is: A configuration issue on my end A limitation of having multiple tunnels per VPN service A strongSwan-specific behavior in the OpenStack context *Specific Questions:* Does VPNaaS officially support multiple IPSec connections per VPN Service? Are there any known limitations or best practices when using multiple tunnels? Should each remote site have its own dedicated VPN Service instead? I've checked the documentation but couldn't find clear guidance on multi-tunnel scenarios. Any insights from the community would be greatly appreciated. *Environment:* OpenStack version: 2024.2 Dalmatian Neutron VPNaaS with strongSwan backend Kubernetes-deployed OpenStack (OpenStack-Helm) Thank you for your time and assistance! Best regards, Shubham

1 0

[Neutron][VPNaaS] Multiple tunnels per IPSec Service
by Shubham Yadav 31 Jul '25

31 Jul '25

Hello OpenStack Community, I'm working with OpenStack VPNaaS (using strongSwan as the backend driver) and have a question about tunnel configuration capabilities. Current Setup: OpenStack VPNaaS with strongSwan driver (StrongSwanDriver) Single VPN Service (gateway) with multiple IPSec Site Connections Each connection targets different remote peers with distinct subnets Configuration Details: VPN Service: Connected to a single router Multiple IPSec Site Connections under the same VPN Service: Connection 1: To MY office (10.10.100.0/24) via peer 68.x.y.34 Connection 2: To Rackspace (172.24.60.0/22) via peer 184.n.p.53 Shared IKE and IPSec policies across connections Local endpoint groups covering multiple subnets (192.168.0.0/24, 10.43.0.0/16) Question: Is it supported/recommended to have multiple IPSec Site Connections under a single VPN Service? I understand this creates multiple tunnels from the same OpenStack router to different remote peers. Current Issue: One of my connections shows "PENDING_CREATE" status while the other is "ACTIVE". I'm trying to determine if this is: A configuration issue on my end A limitation of having multiple tunnels per VPN service A strongSwan-specific behavior in the OpenStack context Specific Questions: Does VPNaaS officially support multiple IPSec connections per VPN Service? Are there any known limitations or best practices when using multiple tunnels? Should each remote site have its own dedicated VPN Service instead? I've checked the documentation but couldn't find clear guidance on multi-tunnel scenarios. Any insights from the community would be greatly appreciated. Environment: OpenStack version: 2024.2 Dalmatian Neutron VPNaaS with strongSwan backend Kubernetes-deployed OpenStack (OpenStack-Helm) Thank you for your time and assistance! Best regards, Shubham

1 0

[Ovs-dpdk]Issue with ovs-dpdk deployment in Kolla-Ansible (zed)
by samira_darvishnia＠yahoo.com 31 Jul '25

31 Jul '25

Hi all, I'm trying to deploy OVS-DPDK using Kolla-Ansible Zed, and I'm running into a blocking issue during deployment. I would really appreciate your insights or suggestions. Problem Summary: During the deployment of ovs-dpdk, the container ovsdpdk-vswitchd fails to start, with the following error in its logs: + echo 'Running command: '\''/usr/sbin/ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/kolla/openvswitch/ovs-vswitchd.log --pidfile'\''' + exec /usr/sbin/ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/kolla/openvswitch/ovs-vswitchd.log --pidfile Running command: '/usr/sbin/ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/kolla/openvswitch/ovs-vswitchd.log --pidfile' ovs-vswitchd: invalid option -- 'w' ovs-vswitchd: invalid option -- 'w' EAL: FATAL: Invalid 'command line' arguments. 2025-07-30T13:44:59Z|00017|dpdk|EMER|Unable to initialize DPDK: Invalid argument ovs-vswitchd: Cannot init EAL (Invalid argument) This appears to be related to an incorrect parsing of arguments for ovs-vswitchd. What I’ve tried: I already applied this patch (as referenced in Gerrit): https://review.opendev.org/c/openstack/kolla-ansible/+/913653/1/ansible/rol… However, the issue still persists. Additional context: • Deployment hangs on the task: name: Wait for dpdk tunnel ip It waits for 5 minutes and then fails. • The parameter dpdk_tunnel_interface_address is already set in my global.yml. • I’ve configured and bound NICs to vfio-pci. • Environment info: • OpenStack Zed • Kolla-Ansible Zed • dpdk_tunnel_interface_address is set • OVS-DPDK is containerized via Kolla Has anyone seen this issue before or can suggest further steps? Any insight on how to correctly pass EAL arguments or what else might be misconfigured would be very helpful. Thanks in advance, Samira

1 0

[Neutron][Tacker][Issue Investigation] Issue with Neutron in Tacker Zuul Patches �C Suspected Regression in Neutron Master Branch
by Renu Rani 31 Jul '25

31 Jul '25

Hi Neutron Team, We are currently facing an issue in Tacker Zuul patch jobs build (with master branch), where the DevStack setup fails during the initial Neutron network creation step. Problem statement: The failure occurs at this point in the stack.sh script, for devstack environment creation in build process. Below command execution is failing in stack.sh [openstack --os-cloud devstack-admin-demo --os-region RegionOne router add subnet 20b264df-5f19-4599-a83d-2e76e7cbf5dc d777d04b-93ff-4e94-b6a9-cc8de3e23921] Error Message: This results in a 500 Internal Server Error, as seen in the logs: [HttpException: 500: Server Error for url: http://10.x.x.x/networking/v2.0/routers/20b264df-5f19-4599-a83d-2e76e7cbf5d…] Analysis for the issue: Below error is coming in Neutron API logs: [ERROR neutron.plugins.ml2.ovo_rpc RuntimeError: cannot notify on un-acquired lock] 1. Tried overriding the required-projects in Zuul to use stable/2025.1 branches (instead of master) to identify project(master branch) which is causing root cause. 2. Tacker team has created DNM patch (952892) to invetsigate this in detail. https://review.opendev.org/c/openstack/tacker/+/952892 1. In DNM patch Patchset 11 (on June 29), we have taken devstack stable/2025.1. In stable branch the devstack environment created sucessfully. 2. In DNM patch Patchset 25 (on July 8), with the same devstack override (stable/2025.1), the job now fails earlier during DevStack setup. The logs show that Neutron failed to start, causing the job to fail across all nodes. As per observations and anlysis seems the recent changes in the Neutron master branch between June 29 and July 8 might be causing this issue. We’d appreciate your help in reviewing the changes in Neutron's master branch during that window that could cause this behavior and suggesting any debugging approaches or patches we can test on tacker. Please let us know if you'd like additional logs or info from the Zuul job runs. Thanks in advance for your support! Thanks & Regards, Renu Rani (レヌラニ) Mobile: +91-9311888962 E-mail: renu.rani(a)india.nec.com<mailto:renu.rani@india.nec.com> The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. It shall not attach any liability on the originator or NEC Corporation India Private Limited or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of NEC Corporation India Private Limited or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.

5 10

[neutron] Is NVIDIA ASAP2 offloading with OpenStack just a myph?
by Vasyl Saienko 30 Jul '25

30 Jul '25

Hello OpenStack community, I'm trying to set up OpenStack with Nvidia ASAP2 hardware offloading in switchdev mode according to OpenStack guidelines <https://docs.openstack.org/neutron/latest/admin/config-ovs-offload.html> and Nvidia guidelines <https://docs.nvidia.com/doca/sdk/ovs-kernel+hardware+offloads/index.html> on the ConnectX6DX. I've tried different network types VLAN and VXLAN but offloading for none of them works. From an OpenStack standpoint everything is configured according to documentation, representor device is created and plugged in the br-int bridge (eth14) physical. PF enp33s0f0np0 is set to switchdev mode ``` devlink dev eswitch show pci/0000:21:00.0 pci/0000:21:00.0: mode switchdev inline-mode none encap-mode basic ``` ``` root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl show 12e00cdb-6520-4666-908f-57f59e8b9bd4 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-ex Interface br-ex type: internal Port pr-floating Interface pr-floating Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Port int-br-tenant Interface int-br-tenant type: patch options: {peer=phy-br-tenant} Port tap58ac2b17-fb tag: 1 Interface tap58ac2b17-fb Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} Port eth14 tag: 1 Interface eth14 Bridge br-tenant Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-tenant Interface br-tenant type: internal Port phy-br-tenant Interface phy-br-tenant type: patch options: {peer=int-br-tenant} Port enp33s0f0np0 Interface enp33s0f0np0 Bridge br-tun Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} ovs_version: "3.0.0-0056-25.04-based-3.3.5" ``` But hardware offloading is not happening, tcpdump on eth14 still shows all packets, and no TC rules are created. OpenStack uses complex openvswitch rules to setup virtual networking, in this scenario vlan tagging is done by the following rule ``` cookie=0x45b91976d5c4f175, duration=233.443s, table=60, n_packets=314, n_bytes=35574, priority=100,in_port=eth14 actions=set_field:0x11->reg5,set_field:0x1->reg6,resubmit(,73) ``` If we simplify openflow rules and align them with Nvidia guidelines for openvswitch hardware offloading by plugging representor device directly into physnet bridge, and assign vlan tag as port attribute hardware offloading starts working. ``` root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl del-port eth14 root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl add-port br-tenant eth14 root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# ovs-vsctl set port eth14 tag=1443 ``` ``` root@kaas-node-52648a8f-cbfd-4b9a-b392-1594b1d97cfa:~# for i in $(ip link | awk '{print $2}' |grep -e enp -e ens |grep -v vf$); do tc filter show dev $i ingress; done filter protocol 802.1Q pref 6 flower chain 0 filter protocol 802.1Q pref 6 flower chain 0 handle 0x1 vlan_id 1443 vlan_prio 0 vlan_ethtype arp dst_mac fa:16:3e:36:1b:90 src_mac fa:16:3e:39:90:d8 eth_type arp in_hw in_hw_count 1 action order 1: vlan pop pipe index 1 ref 1 bind 1 no_percpu used_hw_stats delayed action order 2: mirred (Egress Redirect to device eth14) stolen index 1 ref 1 bind 1 cookie 4b5b5a701f4e3fee7ac1c198037522e3 no_percpu used_hw_stats delayed ``` Similar issue occurs when using the VXLAN network. OpenStack setup VXLAN tunnel like, where tunnel_id is inserted dynamically via openflow rules (in_key=flow) ``` Port vxlan-0a0c0004 Interface vxlan-0a0c0004 type: vxlan options: {df_default="true", dst_port="4790", egress_pkt_mark="0", in_key=flow, local_ip="10.12.0.9", out_key=flow, remote_ip="10.12.0.4"} ``` In this configuration hardware offloading is not happening neither. But when setting up vxlan according to nvidia guidelines (no complex rules are involved) hardware offloading starts working (tcpdump for representor device shows only first packet and tc rules are being created). ``` Port vxlan3 Interface vxlan3 type: vxlan options: {key="60987", local_ip="10.12.0.9", remote_ip="10.12.0.4"} ``` ``` Cannot find device "enp33s0f0npf0vf" Cannot find device "veth68d427d@if1" filter protocol ip pref 2 flower chain 0 filter protocol ip pref 2 flower chain 0 handle 0x1 dst_mac fa:16:3e:65:0c:6d src_mac 52:5a:d0:9e:8a:32 eth_type ipv4 enc_dst_ip 10.12.0.9 enc_src_ip 10.12.0.8 enc_key_id 60987 enc_dst_port 4789 enc_tos 0 ip_flags nofrag in_hw in_hw_count 2 action order 1: tunnel_key unset pipe index 2 ref 1 bind 1 no_percpu used_hw_stats delayed action order 2: mirred (Egress Redirect to device eth15) stolen index 2 ref 1 bind 1 cookie 5d44cdf6754fc237cc0bd688a6fe70a8 no_percpu used_hw_stats delayed ``` Am I doing something wrong? Is there anyone who has other results? -- <https://www.mirantis.com/> Vasyl Saienko Principal DevOps Engineer vsaienko(a)mirantis.com <dstoltenberg(a)mirantis.com> +(380) 66 072 07 17 <++1+(650)+564+7038>

3 3

Openstack integration with openid as leycloak
by amine belroul 30 Jul '25

30 Jul '25

Hello, Is there any detailed docs can help me to integrate openstack with openid? Please multi domains and multi projects.

3 2

[Cinder] pbm configuring multipath on NVMeoF
by Vincent Godin 30 Jul '25

30 Jul '25

Hello guys, Openstack: 2025.1 OS: Ubuntu 24.04 We are having trouble configuring multipath on a NetApp backend using NVMe over TCP. After reading numerous articles on this issue, we concluded that it was necessary to operate in native multipath mode, which is enabled by default on Ubuntu 24.04, and that it is no longer necessary to keep the multipathd service active for this to work. We ran inconclusive tests by setting the "use_multipath_for_image_xfer" and "enforce_multipath_for_image_xfer" variables to true in cinder.conf. We also set the "volume_use_multipath variable" to true in the libvirt section of nova.conf, but without success. After creating an instance and a volume on a server with Openstack, when querying the subsystem with the nvme command, we only get a single path. But when we query the backend from the same server with "nvme discover," we do get four paths. In the os-brick code, we see that it checks a Multipath variable that must be set to true... I think this is purely a configuration issue, but the examples given by various manufacturers (NetApp, PureStorage, etc.) don't indicate anything in particular. So, does anyone know the configuration to apply for multipath to work (cinder/nova)? Thank you.

3 8

OpenStack for AI Whitepaper
by Kendall Nelson 30 Jul '25

30 Jul '25

Hello Folks! As you might have heard, the OpenInfra AI WG has been starting to put together an OpenStack for AI Whitepaper and we wanted to formally invite you all to attend the WG's meetings and working sessions, to get involved on working group's the mailing list <https://lists.openinfra.org/mailman3/lists/ai-openstack-wg.lists.openinfra.…> and to help us shape the whitepaper into a resource for those looking to understand just how OpenStack is capable of serving AI workloads. We could really benefit from your innate understanding of how the OpenStack services work and also hope to surface feedback and questions from the group for the larger OpenStack community in future meetings. Also, we have organizations volunteering to do show and tell sessions about their OpenStack clouds hosting AI workloads and discussing pain points, lessons learned etc. The next of those show and tell sessions will take place on September 29. We have a meeting coming this week on Thursday at 13 UTC in this google meet <http://meet.google.com/wzn-jgcn-cfj>. If you would like an actual calendar invite, please reach out and I will make sure you are added! -- Kendall Nelson *Senior Upstream Developer Advocate* The OpenInfra Foundation <https://openinfra.dev/>

1 1

[kolla] IRC meeting cancelled - 30-th of July 2025
by Bartosz Bezak 29 Jul '25

29 Jul '25

Hello, Due to limited availability of the Kolla core team members, tomorrow’s Kolla IRC meeting is cancelled. Best regards, Bartosz Bezak

1 0

[publiccloud-sig] Public Cloud SIG meeting d.d. 30 July
by Kees Meijs | Nefos 29 Jul '25

29 Jul '25

Hello Stackers, Tomorrow (that's 30 July) the bi-weekly Public Cloud SIG meeting will be held again. The publicly available video meeting will be conducted from 07:00 hrs UTC (09:00 hrs CEST) and on here: https://meetpad.opendev.org/publiccloud-sig Additional information to be found on https://wiki.openstack.org/wiki/PublicCloudSIG See you tomorrow! Cheers, Kees __ Kees Meijs BICT Nefos Cloud & IT <https://nefos.com/contact> Nefos IT bv Burgemeester Mollaan 34a 5582 CK Waalre - NL kvk 66494931 +31 (0)88 2088 188 <tel:+31882088188> nefos.com <https://nefos.com/contact> The information contained in this message is intended for the addressee only and may contain classified information. If you are not the addressee, please delete this message and notify the sender; you should not copy or distribute this message or disclose its contents to anyone. Any views or opinions expressed in this message are those of the individual(s) and not necessarily of the organization. No reliance may be placed on this message without written confirmation from an authorised representative of its contents. No guarantee is implied that this message or any attachment is virus free or has not been intercepted and amended. General terms and conditions ("The NLdigital Terms") apply to all our products and services.

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-9)
by Goutham Pacha Ravi 29 Jul '25

29 Jul '25

Hello Stackers, We're a month away from the feature freeze date for the 2025.2 ("Flamingo") release cycle [1], scheduled for 2025-08-28. OpenStack project teams are observing various internal deadlines prior to that important date to ensure a complete and timely release. As we look ahead, OpenStack's election officials will begin accepting self-nominations for the upcoming Project Team Lead roles and four seats on the OpenStack Technical Committee starting 2025-08-06 [2]. The nomination deadline is two weeks later, on 2025-08-20. You are welcome to submit an early nomination [3] now if you anticipate being away during the nomination window. To be a Project Team Lead candidate, you must be an Active Project Contributor—i.e., someone who has contributed to the project you wish to lead in the past 12 months (counted up to 2025-08-20). Any individual member of the OpenInfra Foundation, regardless of contributor status, may propose their candidacy for the OpenStack Technical Committee. To vote in these elections, you must be a member of the OpenInfra Foundation [4] and an active contributor—either through merged contributions to official projects or via nomination by a project’s PTL, liaisons, any Special Interest Group, or the OpenStack Technical Committee [5]. To receive your ballot, please ensure your email address is added in Gerrit and that you've opted in to receive emails from CIVS before 2025-08-20: https://civs1.civs.us/cgi-bin/opt_in.pl In the past week, the TC approved revised milestones for the cross-project goal of removing the use of Eventlet [6]. We still aim to drop Eventlet usage completely in the 2027.2 ("J") release. The "aetos" project now has a canonical service type called "metric-storage" [7]. Several other governance changes are currently under community review. === Weekly Meeting === The last weekly meeting of the OpenStack TC took place [8] on 2025-07-22, with a smaller attendance than usual. We expect this trend to continue through the summer break in the northern hemisphere. Key updates included changes to the Eventlet-removal goal timeline and a deeper look into the prolonged inactivity of the Monasca project team. While volunteers have consistently expressed interest in maintaining the project, it hasn’t met reactivation milestones since its last active cycle (2024.1). However, some improvements have been merged across the team’s repositories. The TC agreed to enforce a more formal and transparent process: projects must explicitly request extensions to inactive status [9], and future documentation will include retirement deadlines that compel TC votes on any extensions. We'll continue discussing the next steps for Monasca through governance proposals and this mailing list. Opendev administrators informed us that they completed the transition off Nodepool, and Debian Trixie images are nearly ready for testing. This is also useful for transitioning the Ceph CI job to Debian to enable faster fixes in the client software. In open discussion, the group revisited the definition of corporate affiliation in the TC charter. A proposal will be made to clarify what affiliation diversity means and how/why it should be disclosed during elections. The next meeting of the OpenStack Technical Committee is today, 2025-07-29. It will be hosted on the #openstack-tc channel on OFTC at 1700 UTC. Please find the meeting's agenda and other details on its wiki page [10]. I hope you'll be able to join us. === Governance Proposals === === Merged === - Make Eventlet removal deadlines more acceptable for operators | https://review.opendev.org/c/openstack/governance/+/952903 - Retire networking-midonet | https://review.opendev.org/c/openstack/governance/+/955364 === Open for review === - Define "affiliation" within the context of the TC | https://review.opendev.org/c/openstack/governance/+/956024 - Add series names to runtimes doc | https://review.opendev.org/c/openstack/governance/+/955810 - Remove Monasca from active project | https://review.opendev.org/c/openstack/governance/+/953671 - os-test-images never releases | https://review.opendev.org/c/openstack/governance/+/954249 - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 === Upcoming Events === - 2025-08-06: Nominations open for OpenStack elections: https://governance.openstack.org/election/ - 2025-08-20: Nominations close for OpenStack elections - 2025-08-26: OpenInfra Days, Korea: https://2025.openinfradays.kr/ - 2025-08-28: Feature Freeze deadline, Milestone-3 of the Flamingo release [1] - 2025-08-29: Colombia OpenInfra User Group Meetup: https://www.meetup.com/colombia-openinfra-user-group/events/307096751/ - 2025-10-17: OpenInfra Summit, Paris-Saclay: https://summit2025.openinfra.org/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] 2026.1 OpenStack Elections: https://governance.openstack.org/election/ [3] Submitting your candidacy: https://governance.openstack.org/election/#how-to-submit-a-candidacy [4] Join the OpenInfra Foundation: https://openinfra.org/join/individual/ [5] Who is an Active Contributor: https://governance.openstack.org/tc/reference/charter.html#voters-for-tc-se… [6] Eventlet removal timeline: https://governance.openstack.org/tc/goals/selected/remove-eventlet.html#com… [7] OpenStack service types: https://specs.openstack.org/openstack/service-types-authority/#service-data [8] TC Meeting IRC Log 2025-07-22: https://meetings.opendev.org/meetings/tc/2025/tc.2025-07-22-17.00.html [9] Emerging and inactive projects: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [10] TC Meeting Agenda, 2025-07-29: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[manila] Flamingo Mid Cycle: Aug 6 - 7th
by Carlos Silva 28 Jul '25

28 Jul '25

Hello, Zorillas and interested stackers, The OpenStack Manila's mid cycle event [1] will take place on Aug 6th and 7th, from 14 to 16 UTC. More details on the .ics file [1]. At the event, we will focus on checking on the progress of the deliverables we planned for this cycle, as well as do collaborative review sessions for the features being proposed to speed up the review process. If you have a topic you would like to discuss, please add them to the mid cycle etherpad [2]. [1] https://gist.github.com/silvacarloss/598ad6d311cfe0a588b6b47ea979dff5 [2] https://etherpad.opendev.org/p/manila-flamingo-mid-cycle Looking forward to it, carloss

1 0

[i18n] Sharing CMU Students’ Contributions: Progress on Zanata-to-Weblate Migration
by Ian Y. Choi 28 Jul '25

28 Jul '25

Dear OpenStack Community, Over the past six weeks, Seongsoo and I had the privilege of mentoring a group of four students from Carnegie Mellon University (CMU) [1] as part of our ongoing effort to migrate the OpenStack Internationalization (I18n) platform from Zanata to Weblate. I would like to take this opportunity to highlight the significant and meaningful contributions made by three students, Furqan Saeed, Simon Yu, and Jana Hussien, who recently completed video demos showcasing their work [2] [3]. Project Context As many may know, the I18n SIG has been working to modernize our translation infrastructure by transitioning from Zanata (now unmaintained) to Weblate — an actively developed, open source platform. This project not only ensures the sustainability of our translations, but also offers a valuable learning path for new contributors. Below are the key outcomes achieved by Furqan, Simon and Jana: 1. i18n tools script test codes for Weblate (by Furqan Saeed & Simon Yu) Furqan and Simon focused on bringing robust, production-quality unit testing to the collection of Python scripts and tools responsible for Weblate integration. Furqan’s video [2] demonstrates these efforts. Together, they: Developed a comprehensive suite of ~34 unit tests covering the main Weblate tool files. Used Python’s unittest and mocking strategies to ensure functions behave as expected, even simulating server responses. Noted and addressed gaps in requirements, such as missing library dependencies. Provided clear documentation (README.rst) explaining test infrastructure and file structure. Gained and demonstrated familiarity with OpenStack’s development guidelines, Gerrit workflows, and CI requirements. Their efforts establish a much needed test baseline for future improvements and bring our I18n tooling closer to OpenStack’s best practices. This contribution not only enhances software reliability but also creates a safer, more efficient environment for future contributors. 2. Migration of Zuul infrastructure scripts to Weblate (by Jana Hussien) Jana concentrated on migrating and refactoring the automation scripts that enable daily translation updates in Zuul, shifting from Zanata to Weblate. Her work involved: Analyzing and translating the logic in current scripts to ensure seamless migration, including first-time migration routines for new projects. Developing new scripts to automate creation of Weblate components and upload of translation templates (POT) and PO files; handling language code transformations between platforms. Troubleshooting integration challenges between Weblate’s API/CLI and OpenStack workflows, identifying and overcoming tricky configuration issues related to version control structure. Successfully demonstrating the migration workflow for projects such as trove-dashboard and horizon, with detailed documentation of results. Improving operational resilience by implementing enhanced error handling and summary reporting in the scripts. Discovering and addressing several issues during live testing, and documenting workarounds and plans for future improvements. Jana has also outlined next steps, including integrating these scripts further into the Zuul pipeline, modularizing code, and enhancing public documentation. These contributions represent valuable progress toward a full, sustainable migration to Weblate - benefiting the entire OpenStack community by advancing our translation workflows and broadening global accessibility. On behalf of all mentors (myself, Seongsoo Cho, Goutham Pacha Ravi), I’d like to thank Furqan, Simon, and Jana for their hard work, dedication, and collaborative spirit, and encourage other students and new contributors to get involved. If anyone would like to review their code, documentation, or get involved in this effort, please feel free to reach out or interact directly with their contributions [6-9]! Best regards, Ian Y. Choi (ianychoi) & Seongsoo Cho (seongsoocho) OpenStack I18n SIG & Student Mentors [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [2] Video - OpenStack I18n Zanata-to-Weblate Migration: Weblate Tools Unit Test Development (CMU Project Demo) (Furqan), link: https://youtu.be/w966IpBCqeE [3] Video - OpenStack I18n Zanata-to-Weblate Migration: Translation Scripts working on Zuul (CMU Project Demo) (Jana), link: https://youtu.be/RcQweOe-vfo [4] Photo of our kick off meeting: https://photos.app.goo.gl/QhNwBusFZjwvTBzi7 [5] Photos of our final mentoring meeting: Final Mentoring Meeting - https://1drv.ms/f/c/6cc85a64cb418204/EljY3AfrnypGrF6EyDZq83MBDZpLTPt-8VPymj… [6] https://review.opendev.org/c/openstack/i18n/+/953745 (by Furqan, review is welcome) [7] https://review.opendev.org/c/openstack/i18n/+/955982 (by Simon, review is welcome) [8] https://review.opendev.org/c/openstack/i18n/+/955439 (by Jana, working in progress) [9] https://review.opendev.org/c/openstack/trove-dashboard/+/952787 (by Jana, merged)

1 0

[neutron] Bug Deputy Jul 14 - Jul 28
by Jakub Libosvar 28 Jul '25

28 Jul '25

Hi, as I didn’t find the bug deputy report from the last week and confirmed by looking at the IRC meeting logs, I’ve skimmed through the bugs of the last 2 weeks to make sure nothing falls through the cracks. There are a few incomplete bugs that require more info from the reporters in order to find the culprit, seems Brian and Rodolof are on top of it. There is one Critical from today and seems it’s clear what is wrong with it and it seems Yatin is looking at it. Thanks Kuba Critical -------- * tobiko jobs failing while download fedora qcow2 image Edit https://bugs.launchpad.net/neutron/+bug/2118902 Yatin seems to be looking into it Fedora 40 image URL has moved, we need to update the URL to fetch Fedora 41 High —--- * test_subport_delete random failure https://bugs.launchpad.net/neutron/+bug/2117405 Functional test that fails sporadically Assigned to Rodolfo * [tobiko] OVN Metadata agent class failing when using OVN agent https://bugs.launchpad.net/neutron/+bug/2118398 Assigned to Rodolfo Patch proposed: https://review.opendev.org/c/x/tobiko/+/955769 * OVN Neutron agent is being registered as OVN Metadata agent https://bugs.launchpad.net/neutron/+bug/2118876 Rodolfo and Yatin are working on it Patch proposed: https://review.opendev.org/c/openstack/neutron/+/955987 Medium ------- * Horizon shows all prefix lengths when creating subnet from subnet pool https://bugs.launchpad.net/neutron/+bug/2116927 Lajos looked at it as it might be related to the SDK migration, unclear if any fix in Neutron is required * [eventlet-removal] Add N535 hacking check to ban eventlet imports https://bugs.launchpad.net/neutron/+bug/2117373 This is to add the check into Neutron tree but later the check itself was questioned as it might be not working as intended * [neutron-tempest-plugin] Test ``test_create_router_update_external_gateways`` failing https://bugs.launchpad.net/neutron/+bug/2117383 Assigned to Miguel * [unmaintained-only] unknown keyword flow_group_id https://bugs.launchpad.net/neutron/+bug/2117477 Assigned to Bence * [OVN] OVN agents should clean up "Chassis_Private" previous tags https://bugs.launchpad.net/neutron/+bug/2118893 Assigned to Rodolfo Patch proposed: https://review.opendev.org/c/openstack/neutron/+/955998 Low —-- * NotFoundException: 404 error when associating FIP with an unreachable network https://bugs.launchpad.net/neutron/+bug/2118360 Incomplete ---------- * VPN Perfect Forward Secrecy (PFS) is deactivated although configured as active https://bugs.launchpad.net/neutron/+bug/2116723 Rodolfo asked the reporter for more details * Performance Tuning for Nested VXLAN Tunnel in OpenStack (OVS Backend) https://bugs.launchpad.net/neutron/+bug/2116837 Brian asked the reporter for more details * OVN: Intermittent metadata failures for SR-IOV VMs https://bugs.launchpad.net/neutron/+bug/2117078 Rodolfo asked the reporter for more details, the root cause is still unknown * neutron-openvswitch-agent crashes on start https://bugs.launchpad.net/neutron/+bug/2117153 Brian asked the reporter for more details, the root cause is still unknown, happened just once

1 0

Ovn ext ports
by engineer2024 26 Jul '25

26 Jul '25

I have marked a chassis as extport host. But when a sriov port is created, it is being claimed by the node from the controller logs, but it is showing up and down in southbound alternatively,. But when the same node is marked as chassis-gw, the sriov port is showing as up in southbound and stable. What's could be the reason for this? Thanks elinux

1 0

[ops][largescale-sig][scientific-sig] Ops Radio Hour on July 25, 2025
by Ildiko Vancsa 25 Jul '25

25 Jul '25

Hi OpenStack community, I'm reaching out to you with a friendly reminder that the next OpenStack Ops Radio Hour[1] is scheduled to happen on __July 25th, 2025 at 1300UTC.__ The call series is a casual setting for operators, engineers, and contributors to come together, share knowledge, and collaborate on the complexities of managing and deploying OpenStack environments. Topics the next gathering will dive into: - OVN Migration +Documentation proposal: https://github.com/mariojmdavid/migrate-to-ovn - (SWIFT) Compliance processes - GPUs - VMware migration - vTPM (live migrations or not?): reboot, start What other topics should we add to the agenda? A few reminders: - The group on the call agreed to try out Matrix as a chat platform for more synchronous communication outside calls + You can access the Matrix room here: https://matrix.to/#/#openstack-ops:opendev.org <https://matrix.to/#/%23openstack-ops:opendev.org> + Further details are here: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… - If you would like to read and catch up on notes from previous calls, you can find them in this etherpad: https://etherpad.opendev.org/p/openstack_ops_radiohour See you on the call on July 25th! Bets Regards, Ildikó [1] https://wiki.openstack.org/wiki/Operations/Meetups

1 1

[PTLs][all] 2026.1 Gazpacho's proposed cycle/release schedule
by Elõd Illés 25 Jul '25

25 Jul '25

Hi, This is just to advertise next cycle's proposed release schedule [1][2]. Feel free to comment on the patch [1] about the timing of milestones, etc. The generated page can also be checked here [2] (though be careful, as this belongs to PS4, so if that is not the latest patch set at the time of looking, then please search for the latest one). [1] https://review.opendev.org/c/openstack/releases/+/954614 [2] https://6b6a75d201a9c42f31a1-68f6595b812925233ee703f28dff66ba.ssl.cf5.rackc… Cheers, Elod Illes irc: elodilles @ #openstack-release

1 0

[Kolla-ansible][epoxy]nova problems at restart
by Franck VEDEL (UGA) 25 Jul '25

25 Jul '25

Hello everyone, I need your help. II will try to explain my problem by providing a lot of information. I have been trying to solve it for 3 days, without success. In May, my old cluster became too small for the 200 students using it, so I started installing a new cluster. Six nodes, three in one data center, three in another. After testing with Bobcat and Caracal (unsatisfactory, with numerous issues, especially with authentication and display in Horizon), I tried Epoxy. I installed everything I needed and tested it. It was a real success. It was very fast in its new configuration, and I thought everything would be ready by September. In May, my old cluster was becoming too small for the 200 students using it, so I started installing a new cluster. 6 nodes, 3 in one data center, 3 in another. After testing with Bobcat and Caracal (unsatisfactory, with numerous issues, especially with authentication and display in Horizon), I tried Epoxy. I installed everything I needed and tested it. It was a real success. It was very fast in its new configuration; I thought everything would be ready by September. At the beginning of July, I was told to shut down everything, due to electrical maintenance in a data center. I shut everything down. 3 days later, I restarted it, and nothing worked anymore. However, it was simply a shutdown/restart of the servers. The problems are with Nova. Here's some information and logs, if you can help me restart the cluster. So Epoxy on Ubuntu 2404. (venv2) user1@ordi1:~/OPENSTACK/INSTALL$ kolla-ansible --version kolla-ansible 20.0.1.dev4 Since nova didn't want to work I did this: (venv2) user1@ordi1:~/OPENSTACK/INSTALL$ kolla-ansible reconfigure -I multinode -t nova ERROR with a task: TASK [nova-cell : Waiting for nova-compute services to register themselves] **************** skipping: [ordi2] skipping: [ordi3] skipping: [ordi4] skipping: [ordi5] skipping: [ordi6] An exception occurred during task execution. To see the full traceback, use -vvv. The error was: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) fatal: [ordi1]: FAILED! => {"msg": "Unexpected failure during module execution: Expecting value: line 1 column 1 (char 0)", "stdout": ""} I search with Google and find this (BUG2 in the link): https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… But I am in 20.0.1.dev4 There is a lot of logs (with nova)….. total 533340 -rw-r--r-- 1 42436 42436 0 juil. 17 06:42 nova-api.log -rw-r--r-- 1 42436 42436 105512247 juil. 17 06:42 nova-api.log.1 -rw-r--r-- 1 42436 42436 2460643 juil. 6 06:40 nova-api.log.2.gz -rw-r--r-- 1 42436 42436 7305818 juil. 5 06:31 nova-api.log.3.gz -rw-r--r-- 1 42436 42436 7234505 juil. 2 06:48 nova-api.log.4.gz -rw-r--r-- 1 42436 42436 2356307 juin 29 06:35 nova-api.log.5.gz -rw-r--r-- 1 42436 42436 7181854 juin 28 06:49 nova-api.log.6.gz -rw-r--r-- 1 42436 42436 56065234 juil. 18 09:06 nova-api-uwsgi.log -rw-r--r-- 1 42436 42436 113689390 juil. 17 06:42 nova-api-uwsgi.log.1 -rw-r--r-- 1 42436 42436 2150056 juil. 6 06:40 nova-api-uwsgi.log.2.gz -rw-r--r-- 1 42436 42436 6393602 juil. 5 06:31 nova-api-uwsgi.log.3.gz -rw-r--r-- 1 42436 42436 6383183 juil. 2 06:48 nova-api-uwsgi.log.4.gz -rw-r--r-- 1 42436 42436 2083376 juin 29 06:35 nova-api-uwsgi.log.5.gz -rw-r--r-- 1 42436 42436 6351824 juin 28 06:49 nova-api-uwsgi.log.6.gz -rw-r--r-- 1 42436 42436 1467596 juil. 18 09:06 nova-compute.log -rw-r--r-- 1 42436 42436 2626598 juil. 18 09:06 nova-conductor.log -rw-r--r-- 1 42436 42436 44883046 juil. 18 06:45 nova-conductor.log.1 -rw-r--r-- 1 42436 42436 7154 juil. 18 08:58 nova-manage.log -rw-r--r-- 1 42436 42436 221050 juil. 16 15:08 nova-metadata.log -rw-r--r-- 1 42436 42436 28094556 juil. 18 09:06 nova-metadata-uwsgi.log -rw-r--r-- 1 42436 42436 74174166 juil. 17 06:42 nova-metadata-uwsgi.log.1 -rw-r--r-- 1 42436 42436 5650339 juil. 5 06:31 nova-metadata-uwsgi.log.2.gz -rw-r--r-- 1 42436 42436 1861983 juin 29 06:35 nova-metadata-uwsgi.log.3.gz -rw-r--r-- 1 42436 42436 5601477 juin 27 06:53 nova-metadata-uwsgi.log.4.gz -rw-r--r-- 1 42436 42436 5640648 juin 21 06:36 nova-metadata-uwsgi.log.5.gz -rw-r--r-- 1 42436 42436 1871248 juin 15 06:43 nova-metadata-uwsgi.log.6.gz -rw-r--r-- 1 42436 42436 111880 juil. 16 15:08 nova-novncproxy.log -rw-r--r-- 1 42436 42436 2668833 juil. 18 09:06 nova-scheduler.log -rw-r--r-- 1 42436 42436 45997165 juil. 18 06:45 nova-scheduler.log.1 -rw-r--r-- 1 42436 42436 0 mai 15 08:47 privsep-helper.log 2 containers have problems: docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6231ee0f3c97 quay.io/openstack.kolla/nova-scheduler:2025.1-ubuntu-noble "dumb-init --single-…" 10 minutes ago Exited (1) 5 seconds ago nova_scheduler 57707a9d136e quay.io/openstack.kolla/nova-conductor:2025.1-ubuntu-noble "dumb-init --single-…" 39 hours ago Exited (1) 4 seconds ago nova_conductor Have a look to nova_conductor.log (same thing with nova_scheduler): 2025-07-18 09:08:25.170 7 ERROR nova.scheduler.client.report [-] Failed to initialize placement client (is keystone available?): openstack.exceptions.NotSupported: The placement service for 192.168.100.100:RegionOne exists but does not have any supported versions. 2025-07-18 09:08:25.171 7 ERROR nova.conductor.manager [-] Fatal error initializing placement client: The placement service for 192.168.100.100:RegionOne exists but does not have any supported versions.: openstack.exceptions.NotSupported: The placement service for 192.168.100.100:RegionOne exists but does not have any supported versions. 2025-07-18 09:08:25.172 7 CRITICAL nova [-] Unhandled error: openstack.exceptions.NotSupported: The placement service for 192.168.100.100:RegionOne exists but does not have any supported versions. 2025-07-18 09:08:25.172 7 ERROR nova Traceback (most recent call last): 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/bin/nova-conductor", line 8, in <module> 2025-07-18 09:08:25.172 7 ERROR nova sys.exit(main()) 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/cmd/conductor.py", line 45, in main 2025-07-18 09:08:25.172 7 ERROR nova server = service.Service.create(binary='nova-conductor', 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/service.py", line 256, in create 2025-07-18 09:08:25.172 7 ERROR nova service_obj = cls(host, binary, topic, manager, 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/service.py", line 120, in __init__ 2025-07-18 09:08:25.172 7 ERROR nova self.manager = manager_class(host=self.host, *args, **kwargs) 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/conductor/manager.py", line 124, in __init__ 2025-07-18 09:08:25.172 7 ERROR nova self.compute_task_mgr = ComputeTaskManager() 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/conductor/manager.py", line 255, in __init__ 2025-07-18 09:08:25.172 7 ERROR nova self.report_client 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/conductor/manager.py", line 283, in report_client 2025-07-18 09:08:25.172 7 ERROR nova return report.report_client_singleton() 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/scheduler/client/report.py", line 91, in report_client_singleton 2025-07-18 09:08:25.172 7 ERROR nova PLACEMENTCLIENT = SchedulerReportClient() 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/scheduler/client/report.py", line 237, in __init__ 2025-07-18 09:08:25.172 7 ERROR nova self._client = self._create_client() 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/scheduler/client/report.py", line 280, in _create_client 2025-07-18 09:08:25.172 7 ERROR nova client = self._adapter or utils.get_sdk_adapter( 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/utils.py", line 1035, in get_sdk_adapter 2025-07-18 09:08:25.172 7 ERROR nova return getattr(conn, service_type.replace('-', '_')) 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/openstack/service_description.py", line 88, in __get__ 2025-07-18 09:08:25.172 7 ERROR nova proxy = self._make_proxy(instance) 2025-07-18 09:08:25.172 7 ERROR nova ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-07-18 09:08:25.172 7 ERROR nova File "/var/lib/kolla/venv/lib/python3.12/site-packages/openstack/service_description.py", line 290, in _make_proxy 2025-07-18 09:08:25.172 7 ERROR nova raise exceptions.NotSupported( 2025-07-18 09:08:25.172 7 ERROR nova openstack.exceptions.NotSupported: The placement service for 192.168.100.100:RegionOne exists but does not have any supported versions. 2025-07-18 09:08:25.172 7 ERROR nova Another thing is strange: Autre point bizarre: tail -f /var/log/kolla/placement/placement-api-uwsgi.log --- no python application found, check your startup logs for errors --- [pid: 1092|app: -1|req: -1/307186] 192.168.100.4 () {22 vars in 215 bytes} [Fri Jul 18 09:11:28 2025] GET / => generated 21 bytes in 0 msecs (HTTP/1.0 500) 3 headers in 102 bytes (0 switches on core 0) What can I do ? Can you help me to find a solution to restart my Openstack ? Thanks a lot if you can. Franck

3 8

[neutron]instances can see traffic of others
by Nguyễn Hữu Khôi 25 Jul '25

25 Jul '25

Hello, I have some instances in the same vlan and compute node. They can see traffic from another instance which looks like port mirror. I can workaround via anti affinity rules but is there any solution for this problem? On vmware we can turn off or on promiscuous mode on the network port group. Thank you. Regards Nguyen Huu Khoi

2 6

[kolla][barbican] How to securely manage OpenStack service passwords with Castellan?
by elnaz cl 24 Jul '25

24 Jul '25

Hi all, I have deployed OpenStack using Kolla Ansible. By default, service passwords (e.g., for MySQL, Keystone, etc.) are stored in clear text inside configuration files rendered by Kolla (such as nova.conf, keystone.conf, etc.). I'm looking for a more secure approach to avoid exposing these secrets. Specifically, is there a way to integrate Barbican and Castellan with Kolla-based deployments, so that secrets can be securely loaded at runtime using oslo.config (e.g., using {{ secret:UUID }})? If this is not natively supported, are there any best practices or workarounds for handling this securely? Thanks in advance for any guidance. Best, [Your Name]

1 0

[neutron] metadata-agents work but are not alive
by Eugen Block 23 Jul '25

23 Jul '25

2 4

Neutron dhcp
by engineer2024 23 Jul '25

23 Jul '25

Hi Folks, In a neutron DHCP network, the route to the link local metadata ip from a VM, is thru the ip set by the neutron dhcp agent's static route option for that subnet. This is the same IP as that of the qdhcp namespace's tap interface. Can we choose a specific ip from this subnet and pass it to neutron to use the same when assigning the ip in qdhcp ns and as a static route option to link local ?

2 1

[Ironic][Plugins] networking-baremetal vs networking-generic-switch
by Joao Antonio 22 Jul '25

22 Jul '25

Hello, team! What is the difference between *networking-generic-switch* and *networking-baremetal* in terms of l*ocal_link_information*? The networking-generic-switch README file is explicit about local_link_information. Does networking-baremetal also handle local_link_information? -- Joao

2 1

[cinder] Disable Availability Zone
by Kees Meijs | Nefos 22 Jul '25

22 Jul '25

Hello Stackers, For legacy purposes, we've got an Availability Zone running that's not to be used for new workload any more. Within Nova one could simply disable Compute Nodes (and that's what we did) ensuring no Compute Instances get scheduled. But, how about Cinder? How can we keep current workload running within a certain AZ, but ensure no new workload gets accepted? Thanks! Cheers, Kees

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-10)
by Goutham Pacha Ravi 22 Jul '25

22 Jul '25

Hello Stackers, We're ten weeks away from the coordinated release of OpenStack 2025.2 ("Flamingo") [1]. The feature freeze date for this release is in five weeks (2025-08-28). In the past week, the OpenStack Technical Committee did not merge any new governance changes, though a few are currently under community review. === Weekly Meeting === The last weekly meeting of the OpenStack Technical Committee took place on 2025-07-15 over IRC [2]. The TC caught up on a backlog of action items, starting with progress on the community goal to drop the use of eventlet [3]. We discussed the activity within the Monasca project team and have reached out to the PTL and core reviewers to talk about its future. The rest of the meeting focused on the maintenance of the "service-types-authority" repository. While some suggested involving the openstacksdk-core team in its maintenance, others opined that the repo should remain under TC oversight due to its governance significance—since changes affect how services are named and recognized across the ecosystem. For now, we have renewed our commitment to reviewing and merging change proposals more actively. A late tangent about the First Contact SIG’s dormancy prompted an idea to review SIG activity at the upcoming PTG in October. The next meeting of the OpenStack TC is today, 2025-07-22, at 1700 UTC. This meeting will be hosted over IRC on the #openstack-tc channel on OFTC. The agenda has been added to the meeting's wiki page [4]. I hope you'll be able to join us there. Any community member is welcome to edit the wiki page and propose new topics. === Governance Proposals === ==== Open for review ==== - os-test-images never releases | https://review.opendev.org/c/openstack/governance/+/954249 - Retire networking-midonet | https://review.opendev.org/c/openstack/governance/+/955364 - Remove Monasca from active project | https://review.opendev.org/c/openstack/governance/+/953671 - Make Eventlet removal deadlines more acceptable for operators | https://review.opendev.org/c/openstack/governance/+/952903 - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 === Upcoming Events === - 2025-07-26: OpenInfra Days, Vietnam: https://www.vietopeninfra.org/void2025 - 2025-08-06: Nominations open for OpenStack elections: https://governance.openstack.org/election/ - 2025-08-26: OpenInfra Days, Korea: https://2025.openinfradays.kr/ - 2025-08-29: Colombia OpenInfra User Group Meetup: https://www.meetup.com/colombia-openinfra-user-group/events/307096751/ - 2025-10-17: OpenInfra Summit, Paris-Saclay: https://summit2025.openinfra.org/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] TC Meeting IRC Log 2025-07-15: https://meetings.opendev.org/meetings/tc/2025/tc.2025-07-15-17.00.html [3] Eventlet removal goal timeline changes: https://review.opendev.org/c/openstack/governance/+/952903 [4] TC Meeting Agenda, 2025-07-22: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

Instance Freeze During Snapshot in OpenStack 2024.1 with Ceph Octopus
by mohammad kokabi 21 Jul '25

21 Jul '25

Hello, I hope this message finds you well. I am currently using OpenStack version 2024.1, with Ceph Octopus as the backend storage. The configuration is as follows: Glance is configured to use the "images" pool on Ceph. The virtual machine disks are stored on the "vms" pool. When I attempt to create a snapshot of a running instance, the instance appears to freeze for a few seconds, during which it loses network connectivity (e.g., I cannot ping it). This behavior is unexpected, as I understand that snapshot operations, especially with Ceph backend, should not significantly interrupt the instance's runtime. Could you please advise what might be causing this issue? Are there any known limitations, configurations, or best practices I should be aware of regarding snapshot behavior with Ceph in this version of OpenStack? Looking forward to your insights. Best regards,

2 1

RE: Create image error
by berndbausch＠gmail.com 21 Jul '25

21 Jul '25

Violating limits imposed by the operator should lead to a useful error message rather than an HTTP 5xx error. In any case, there should have been a message in the Glance API log. Perhaps you are using a prerelease version of OpenStack? When you get Devstack from the Git, you may want to specify the version; otherwise, you obtain software that is currently under development. For example, the following gets you the first version released this year (Epoxy): git clone https://opendev.org/openstack/devstack -b stable/2025.1 ------------------------------------------------------------------------------------------------------------------- From: BEDDA Fadhel <fadhel.bedda(a)gmail.com> Sent: Saturday, May 31, 2025 6:00 PM To: berndbausch(a)gmail.com Subject: Re: Create image error Hello, I was able to work around the problem by disabling the limits in the file: /etc/glance/glance-api.conf False instead of True Thanks Le sam. 31 mai 2025 à 09:55, <mailto:berndbausch@gmail.com> a écrit : My guess is that the image service is down. To check its status: systemctl status devstack@g-api Also check the image service log: journalctl -e -u devstack@g-api ------------------------------------------------------------------------ From: BEDDA Fadhel <mailto:fadhel.bedda@gmail.com> Sent: Saturday, May 31, 2025 3:08 PM To: mailto:openstack-discuss@lists.openstack.org Subject: Create image error Hi everyone, I installed OpenStack (DevStack). At first, I was able to create three images (Ubuntu, Debian, and Windows) in addition to the Cirros image. When I tried to add another image, everything got stuck. I couldn't add any images, and I got the infamous error 550. HttpException: 502: Server Error for url:

1 0