July 2025 - openstack-discuss - lists.openstack.org

[kolla-ansible][rabbitmq] RabbitMQ cluster creation took too long
by Nguyễn Hữu Khôi 10 Feb '26

10 Feb '26

Hello. I am deploying OpenStack 2025.1 with Kolla-Ansible 2025.1 (master branch) using the latest images. However, when I created a RabbitMQ(4.0.x) cluster with 3 nodes, it took too long—around 12 minutes(it stucked at DB: virgin node -> run peer discovery). I then changed the images to RabbitMQ version 3.13.7, and it worked well. My OS: Ubuntu 24.01 Nguyen Huu Khoi

2 2

[openstack][octavia]Resize a Load Balancer
by Nguyễn Hữu Khôi 27 Jan '26

27 Jan '26

Hello. I see that patch https://review.opendev.org/c/openstack/octavia/+/890215 Could I ask why we cannot merge it? Nguyen Huu Khoi

3 6

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 08 Dec '25

08 Dec '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 16

[Octavia][Load Balancer][2023.1] Amphora Fails Health Check – “No Route to Host” & Timeout Errors
by Thamanna Farhath 02 Dec '25

02 Dec '25

Hi OpenStack Community, I am currently facing an issue with the Load Balancer (Octavia) service in my OpenStack deployment and would appreciate your guidance or suggestions on how to troubleshoot and resolve it. Environment Details: OpenStack Version: 2023.1 (Antelope) Operating System: Ubuntu 22.04 Octavia Backend: Amphora Network Type: VXLAN Deployment Tool: Kolla Ansible Issue Summary: After creating a Load Balancer, the Amphora instance is successfully launched. However, approximately 18 minutes later, the Load Balancer transitions to an ERROR state. Error Observed: octavia.common.exceptions.ComputeWaitTimeoutException: Waiting for compute id a222527f-8336-4fb1-8e9e-0e3ac6e8d580 to go active timeout. octavia.amphorae.driver_exceptions.exceptions.AmpConnectionRetry: Could not connect to amphora, exception caught: HTTPSConnectionPool(host='192.168.200.247', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError: Failed to establish a new connection: [Errno 113] No route to host) What I’ve Tried: Created a VXLAN-based internal network: openstack subnet create --network octavia-mgmt-net --subnet-range 192.168.200.0/24 octavia-mgmt-subnet openstack router create octavia-router openstack router add subnet octavia-router octavia-mgmt-subnet openstack router set octavia-router --external-gateway <public-network-id> Manually configured OVS interface: ovs-vsctl add-port br-int octavia-mgmt -- set interface octavia-mgmt type=internal ip addr add 192.168.200.1/24 dev octavia-mgmt ip link set octavia-mgmt up Security groups are open (allowing all ingress and egress). Verified OVS bridge setup – interface appears correctly on br-int. Behavior Noticed: Amphora instance gets created but fails to communicate with the controller after a while,so health checks fails. The controller cannot reach the Amphora management IP (192.168.200.x). Logs show heartbeat/connection issues (No route to host, timeouts). Request for Help: I would appreciate any of the following: Suggestions to verify if the management network is properly routed from controller to Amphora. Best practices for setting up the Octavia management network using VXLAN. Troubleshooting steps for verifying health manager connectivity to Amphora. Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

4 3

LXB Removal and OVN migration
by Ricardo Cano 09 Nov '25

09 Nov '25

Hello openstack-discuss, With the removal of the linuxbridge mechanism driver, has anyone successfully completed an in-place migrationfrom LinuxBridge to OVN? I did come across this https://www.jimmdenton.com/migrating-lxb-to-ovn/ and that looks good as a one-shoit while biting the bullet and taking a full outage. However, I was wondering if it was possible to run lxb and ovn side-by-side and cut them over, say one network at a time. This particular deployment was stood up using openstack-ansible back in the rocky days (now running antelope) and it appears that openstack ansible's default was lxb up until Zed. Any suggestions or recommendations on the best path forward? Also, will the dynamic routing agent work with ovn?

4 3

[openstack helm] [Glance] Glance showing OSError: unable to receive chunked part when uploading image
by daniel890723＠gmail.com 08 Nov '25

08 Nov '25

Hello i'm testing using Openstack-helm to install all component but after the installation complete i can't upload the OS image through both openstack CLI or Horizon it stuck at Queueing and when i checking Glance Api log it shows 2024-11-07 07:31:19.362 7 INFO glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2024-11-07 07:31:19.400 7 ERROR glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Failed to upload image data due to internal error: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Caught error: unable to receive chunked part: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Traceback (most recent call last): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1302, in __call__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1345, in dispatch 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return method(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 415, in wrapped 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 299, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 162, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 492, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 443, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/quota/__init__.py", line 322, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 596, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 487, in _upload_to_store 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 397, in add_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 479, in store_add_to_backend_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/driver.py", line 294, in add_adapter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/capabilities.py", line 176, in op_checker 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 764, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise errors.get(e.errno, e) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 746, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi for buf in utils.chunkreadable(image_file, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/common/utils.py", line 73, in chunkiter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = fp.read(chunk_size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 294, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = self.data.read(i) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 121, in readfn 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = fd.read(*args) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/format_inspector.py", line 956, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = self._source.read(size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1045, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi data = uwsgi.chunked_read() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Ceph Cluster Status: NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID ceph /var/lib/rook 3 82m Ready Cluster created successfully HEALTH_OK 5cf8b9bb-13e1-4c31-b47d-9e7d92c3c05d Ceph osd status: ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS TYPE NAME -1 0.35156 - 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 1.00 - root default -5 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 - host instance-lu-1 2 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 195 up osd.2 -3 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-2 0 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 195 up osd.0 -7 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-3 1 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 197 up osd.1 TOTAL 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 MIN/MAX VAR: 1.00/1.01 STDDEV: 0.01 now i'm able to create Empty Volume using horizon(can't create volume using default cirros image ) and i can start an Instance without volume and mount the empty volume in(Ceph also can detect when i use that volume) is there something i need to set when starting glance? because i'm following the Openstack-helm installation document to install openstack with helm

5 8

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 29 Oct '25

29 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

[openstack][nova]qemu system wont release memory
by Nguyễn Hữu Khôi 29 Sep '25

29 Sep '25

Hello. I have an OpenStack instance that was using 14GB out of 16GB of memory. Later, when it was using only 4GB, I checked the QEMU process on the compute node, and it was still consuming 14GB, this means memory isn't released when usage decreases. Do we have solutions for this? I use Ubuntu 24.04 instances. Nguyen Huu Khoi

2 4

[Neutron][operators] LinuxBridge to OVN Outreach
by Daniel Failing 29 Sep '25

29 Sep '25

Hi all, Following on the good session we had last time discussing migrating out of LinuxBridge between operators, it was proposed to meet again in June. To find a good time slot, please respond to the following survey: https://newdle.cern.ch/newdle/TtbH8Dbv For reference, the notes from last meeting: https://etherpad.opendev.org/p/neutron-lb-ovn-migration Best, Daniel Failing CERN Cloud Infrastructure

2 3

[cinder] IBM SVf driver callhome
by Vivek Pandey 10 Sep '25

10 Sep '25

Hi Joe/ Brian, Since we didn’t get a chance to discuss the patch [IBM SVf driver] Adding support for callhome plugin, we can continue the conversation here. Patch : https://review.opendev.org/c/openstack/cinder/+/951829. We need callhome enabled as default for which we have introduced a parameter “callhome” in cinder.conf. This feature is to fecth the os_version, cinder-manage version and ispbhaenabled. Note: We are not fetching any customer’s sensitive data. Please let me know if you’ve had a chance to review NetApp’s ActiveIQ implementation. Netapp’s ActiveIQ code ref : https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/netap… As we have discussed during the 9th July cinder call, Anoop from NetApp confirmed that they are enabling it by default and EMS details are being captured from the logs. Based on that, do you think we can align or extend our current IBM callhome implementation (from the patch above) to follow a similar approach? Thanks, Vivek

5 9

[openstack-magnum ui] Cannot choose an external network ID when creating a cluster template
by Nguyễn Hữu Khôi 27 Aug '25

27 Aug '25

Hello, When I create a cluster template, I cannot choose an external network ID, it disappears from horizon. My env: Ubuntu 22.04 Openstack 2024.2 Thank you. Regards Nguyen Huu Khoi

3 2

[blazar] Blazar-dashboard contains non-free files (ie: minified Javascripts)
by Thomas Goirand 22 Aug '25

22 Aug '25

Hi, I've been trying to package Blazar, all is already in Debian, but I can't get blazar-dashboard in, because upstream source contains non-free files. Namely: https://github.com/openstack/blazar-dashboard/blob/master/blazar_dashboard/… is a minifed version of a much more complex package: https://github.com/apexcharts/apexcharts.js/tree/v3.31.0 Reminder: minified Javascripts are considered by most distributions (this includes Red Hat, Ubuntu, and in my case Debian) as non-free files that cannot be shipped in the distribution without the underlying source code, and packages must build from the not minified version. Could the Blazar team work on un-vendoring apexcharts.min.js, possibly removing it completely, or replacing it with a source-full version? Cheers, Thomas Goirand (zigo)

4 6

[tc][all][monasca] Retiring the Monasca project and repositories
by Goutham Pacha Ravi 07 Aug '25

07 Aug '25

Hello Stackers, We intend to retire the Monasca project [1] and all associated repositories [2]. The retirement is being proposed with the support and consent of the current core team members and the Project Team Lead (PTL) [3]. For context, the Monasca project team was marked as being inactive during the 2024.1 (Caracal) release cycle [4]. Since then, no sustained development activity has occurred. There have been no releases tagged for any of the deliverables owned by the team either, and there are no actively maintained stable branches. So, in accordance with the procedures in the OpenStack governance documentation [5], and based on the current state of the project, we propose the following repositories ahead of the 2025.2 ("Flamingo") coordinated release: - openstack/monasca-agent - openstack/monasca-api - openstack/monasca-common - openstack/monasca-events-api - openstack/monasca-log-api - openstack/monasca-notification - openstack/monasca-persister - openstack/monasca-specs - openstack/monasca-tempest-plugin - openstack/monasca-thresh - openstack/python-monascaclient - openstack/monasca-ui - openstack/monasca-transform - openstack/monasca-ceilometer - openstack/monasca-grafana-datasource - openstack/monasca-kibana-plugin - openstack/monasca-statsd Some of the current maintainers intend to fork the project and continue development independently for their purposes. This retirement does not preclude such efforts; nor does it prevent reviving the project after the retirement. If there are any objections to retiring the project, please respond to this thread. In the event there is interest in continuing Monasca within OpenStack, we would expect the project team to meet the requirements for active status [6] — including identifying a new PTL for the upcoming release cycle and demonstrating plans for sustained maintenance. Thank you to everyone who has contributed to Monasca over the years. Thanks, On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://docs.openstack.org/monasca-api/latest/ [2] https://github.com/openstack/governance/blob/624bb5bf124452eed475c9b6e62144… [3] https://review.opendev.org/c/openstack/governance/+/953671 [4] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [5] https://docs.openstack.org/project-team-guide/repository.html#retiring-a-re… [6] https://governance.openstack.org/tc/reference/new-projects-requirements.html

3 2

[dev][designate] PSA: pkg_resources is broken with newer oslo wheels on Python 3.8
by Stephen Finucane 07 Aug '25

07 Aug '25

o/ A quick PSA for a topic has come up in a few different OpenStack forums of late. As described in a recently filed setuptools bug [1], the pkg_resources found in the most recent version of setuptools to support Python 3.8 does not correctly normalize package names passed to `pkg_resources.require`. This has become an issue of late because since earlier this year the OpenStack release tooling has started (correctly) normalizing filenames of the '.whl' files uploaded to PyPI (so e.g. 'oslo.messaging' now becomes 'oslo_messaging'). You are likely to see this manifest with weird errors like `The 'package.foo>=1.2.3' distribution was not found`, when you know that it is. I have seen it pop up because either (a) pkg_resources is being invoked via an explicit call, as in some of the release tooling used in `openstack/releases`, or (b) pkg_resources is being invoked via the auto-generated import headers added by setuptools when you use its legacy "scripts" keyword rather than newer "console scripts" replacement. The latter in particular is biting some projects indirectly due to Designate and Octavia incorrectly using this mechanism to work around the demise of wsgi_scripts, and I have fixes up waiting for review [2][3]. Just sharing in case some people have seen this, because it's quite a weird corner case to hit. Stephen [1] https://github.com/pypa/setuptools/issues/5051 [2] https://review.opendev.org/c/openstack/designate/+/950569/ [3] https://review.opendev.org/c/openstack/designate/+/955683/

3 3

[all][ops][qa] Decommissioning RefStack and retiring related software
by Jeremy Stanley 05 Aug '25

05 Aug '25

The RefStack service at refstack.openstack.org has been unused for purposes of qualifying OpenStack trademark licenses[*] since March of last year, after a 2023 board resolution[**] relaxed the former requirement for it. Effective immediately, the service is being decommissioned along with any associated resources, and the related Git repositories in OpenDev (interop, refstack, refstack-client) will be officially retired in the near future as well. Let me know if you have any questions or concerns. [*] https://openinfra.org/legal/trademark-policy [**] https://board.openinfra.org/meetings/minutes/05122023BoardMinutes -- Jeremy Stanley on behalf of the OpenInfra Foundation

4 3

[kolla][barbican] Secure password management in Kolla-Ansible using Castellan
by elnazcloud＠gmail.com 05 Aug '25

05 Aug '25

Hi all, I have deployed an OpenStack environment using Kolla Ansible. I'm looking for a way to avoid storing service passwords (e.g., database, keystone, etc.) in clear text inside the configuration files. Is there any recommended method to integrate Barbican and Castellan so that services like Nova, Keystone, or Cinder can fetch secrets securely during runtime using oslo.config? If not natively supported, are there best practices or known workarounds for achieving this securely in a Kolla-based deployment? Thanks in advance!

7 12

[Zed][Neutron][OVN] Severe VM Network Performance Degradation Compared to Host
by Jeff Yang 05 Aug '25

05 Aug '25

Hi all, I'm running OpenStack Zed with OVN as the Neutron backend. I've encountered a severe VM network performance degradation issue and would appreciate any insights from the community. 🔹 **Environment** - OpenStack version: Zed - Neutron backend: OVN - Two VMs (each 4 vCPUs, 8 GB RAM) launched on two different compute nodes - Multi-queue is enabled on the VM vNICs 🔹 **Issue** The network performance inside the VMs is significantly worse than on the host nodes, with high packet retransmissions observed. 🔹 **Test results** **Host-to-Host (VTEP IPs)** ``` $ iperf3 -c 192.168.152.152 Connecting to host 192.168.152.152, port 5201 [ 5] local 192.168.152.153 port 45352 connected to 192.168.152.152 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.38 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 1.00-2.00 sec 1.37 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 2.00-3.00 sec 1.42 GBytes 12.2 Gbits/sec 0 3.10 MBytes [ 5] 3.00-4.00 sec 1.39 GBytes 11.9 Gbits/sec 0 3.10 MBytes [ 5] 4.00-5.00 sec 1.38 GBytes 11.8 Gbits/sec 0 3.10 MBytes [ 5] 5.00-6.00 sec 1.43 GBytes 12.3 Gbits/sec 0 3.10 MBytes [ 5] 6.00-7.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 7.00-8.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 8.00-9.00 sec 1.41 GBytes 12.1 Gbits/sec 0 3.10 MBytes [ 5] 9.00-10.00 sec 1.42 GBytes 12.2 Gbits/sec 0 3.10 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 14.0 GBytes 12.0 Gbits/sec 0 sender [ 5] 0.00-10.04 sec 14.0 GBytes 12.0 Gbits/sec receiver iperf Done. ``` **VM-to-VM (through overlay network)** ``` $ iperf3 -c 10.0.6.10 Connecting to host 10.0.6.10, port 5201 [ 5] local 10.0.6.37 port 56710 connected to 10.0.6.10 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 499 MBytes 4.19 Gbits/sec 263 463 KBytes [ 5] 1.00-2.00 sec 483 MBytes 4.05 Gbits/sec 467 367 KBytes [ 5] 2.00-3.00 sec 482 MBytes 4.05 Gbits/sec 491 386 KBytes [ 5] 3.00-4.00 sec 483 MBytes 4.05 Gbits/sec 661 381 KBytes [ 5] 4.00-5.00 sec 472 MBytes 3.95 Gbits/sec 430 391 KBytes [ 5] 5.00-6.00 sec 480 MBytes 4.03 Gbits/sec 474 350 KBytes [ 5] 6.00-7.00 sec 510 MBytes 4.28 Gbits/sec 567 474 KBytes [ 5] 7.00-8.00 sec 521 MBytes 4.37 Gbits/sec 565 387 KBytes [ 5] 8.00-9.00 sec 509 MBytes 4.27 Gbits/sec 632 483 KBytes [ 5] 9.00-10.00 sec 514 MBytes 4.30 Gbits/sec 555 495 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 4.84 GBytes 4.15 Gbits/sec 5105 sender [ 5] 0.00-10.05 sec 4.84 GBytes 4.14 Gbits/sec receiver iperf Done. ``` As you can see, there's a sharp drop in performance inside the VMs (from ~12 Gbps down to ~4 Gbps), with a lot of retransmissions in 10 seconds. Has anyone encountered a similar issue with OVN and VM networking performance? What could be the potential causes? Any suggestions or debugging tips are welcome. Thanks in advance! Best regards,

5 4

[Neutron] - General thoughts on tenancy network MTU
by Karl Kloppenborg 01 Aug '25

01 Aug '25

Hi Neutrinos, I was discussing with a colleague about our MTU settings within our Openstack regions, generally speaking we have a physnet_global_mtu of 1550 with tenancy networks automatically provisioned as 1500 VXLAN. This ends up with VM’s receiving DHCP MTU settings of 1500 which is fine, however reading about yesterday’s discussion on OVN performance, Nguyen’s recommendation was to enable Jumboframes. This got me thinking, what do you set your tenancy and phynet MTU’s to? Very interested to hear about your experiences with running 9k MTU’s on the VM networks etc, what impact this has on OVS/OVN routers and northbound connectivity being 1500. Thanks, Karl. Karl Kloppenborg Chief Technology Officer m: +61 437 239 565 resetdata.com<https://resetdata.com/> [cid:reset_69557fc2-1d63-4932-b5fd-93bd4f39ca7b.png] ResetData supports Mandatory Client Related Financial Disclosures – Scope 3 Emissions Reporting For more information on the phasing of these requirements for business please visit; https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…<https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…> This email transmission is intended only for the addressee / person responsible for delivery of the message to such person and may contain confidential or privileged information. Confidentiality and legal privilege are not waived or lost by reason of mistaken delivery to you, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. Whilst this email has been checked for viruses, the sender does not warrant that any attachments are free from viruses or other defects. You assume all liability for any loss, damage or other consequences which may arise from opening or using the attachments. If you received this e-mail in error please delete it and any attachments and kindly notify us by immediately sending an email to contact(a)resetdata.com.au<mailto:contact@resetdata.com.au>

2 1

(Ceilometer) Openstack - Kolla-Ansible deployment - Ceilometer (Central & compute) Docker Containers are unhealthy
by TRB PREM KUMAR 01 Aug '25

01 Aug '25

Hi, We have deployed an all-in-one OpenStack server for billing. Now ceilometer containers are unhealthy. Please help us to resolve this issue. Reference Output for reference: trb1@openstack-aio-01:~$ docker ps | grep ceil c53fc69ba42b quay.io/openstack.kolla/ceilometer-compute:2024.2-rocky-9 "dumb-init --single-…" About a minute ago Up About a minute *(unhealthy) * ceilometer_compute 9337af646e0f quay.io/openstack.kolla/ceilometer-central:2024.2-rocky-9 "dumb-init --single-…" 30 minutes ago Up About a minute ( *unhealthy)* ceilometer_central 554c4b8ccd86 quay.io/openstack.kolla/ceilometer-notification:2024.2-rocky-9 "dumb-init --single-…" 4 days ago Up About a minute (healthy) ceilometer_notification *** Logs files are attached for reference. Regards, TRB.

3 6

[scientific-sig][help] Research about private cloud
by Winicius Allan 31 Jul '25

31 Jul '25

Hi all! I'm about to write my final paper, and I want to research something related to OpenStack and private clouds. If someone wants to take the time to discuss the problems in your environment, I'd be very glad!What I had on my mind is: - There is something that could improve utilization and performance when the user is not charged about their use on a private cloud. - There is any process scheduling algorithm that could be used on specific workloads instead of Linux default scheduler that can improve application performance? I was thinking about using eBPF here. Best regards.

2 1

[cinder] AttributeError: 'Volume' object has no attribute 'use_quota'
by Kees Meijs | Nefos 31 Jul '25

31 Jul '25

Hello Stackers, It seems we're hitting a bug within the Cinder Volume service. The problem presents itself when creating a Ceph RBD backed Volume using an originating snapshot. > 2025-07-31 19:02:54.896 3602390 WARNING cinder.volume.manager > [req-a476ac04-ee6c-4335-8fca-e606616a5959 > eccdb27249254d9d9fb73570f5de8877 d678ba8e1dee4f7484555580d0a61562 - - > -] Task > 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' > (03c06b47-7df3-4427-8ad0-61f9a97d0159) transitioned into state > 'FAILURE' from state 'RUNNING' > 5 predecessors (most recent first): > </snip (maybe the Atoms are needed and if so I could redact those)> > |__Flow 'volume_create_manager': AttributeError: 'Volume' > object has no attribute 'use_quota' > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager Traceback > (most recent call last): > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", > line 53, in _execute_task > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager result = > task.execute(**arguments) > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", > line 1158, in execute > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > model_update = self._create_from_snapshot(context, volume, > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", > line 461, in _create_from_snapshot > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > model_update = self.driver.create_volume_from_snapshot(volume, > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager File > "/usr/lib/python3/dist-packages/cinder/volume/drivers/rbd.py", line > 1069, in create_volume_from_snapshot > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager if > int(volume.size): > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager > AttributeError: 'Volume' object has no attribute 'use_quota' > 2025-07-31 19:02:54.896 3602390 ERROR cinder.volume.manager Yes, this reflects a non-current version, to be precise Cinder 17.4. However, does this look familiar to someone, maybe? Thanks! Cheers, Kees

1 1

[Neutron] ML2: Can Operators Safely Extend binding:profile with Custom Keys?
by cxue22＠bloomberg.net 31 Jul '25

31 Jul '25

Hi all, I'm working on a Neutron customization to expose and manage data in the `binding:profile` field stored in the `ml2_port_bindings` table, and I have a few questions around operator and third-party plugin compatibility — particularly in reference to the blueprint: https://blueprints.launchpad.net/neutron/+spec/ml2-binding-profile As I understand it, the `binding:profile` is represented as a string column containing a JSON dictionary that can be passed by the compute agent and interpreted by ML2 mechanism drivers during port binding. My intent is to: - Create a custom API endpoint that allows reading and updating specific key/value pairs in this JSON dictionary. - Enable third-party mechanism drivers and operators to add custom fields to this profile. My main question is: **Are operators and third-party ML2 drivers free to add arbitrary key/value pairs to the `binding:profile` dictionary, and can those be expected to persist across Neutron upgrades and database migrations?** Additionally: - Are there any best practices or expectations around namespacing or schema validation of custom fields in `binding:profile`? - Are there known caveats or upgrade scenarios where such data may be truncated, filtered, or overwritten? I'd appreciate any guidance from the Neutron community on how this field is intended to be extended or preserved, and whether such customization is supported in the long term. Thanks in advance! Best regards, Chang Xue Bloomberg Engineering

3 3

[openstack][magnum]Cannot upgrade clusters
by Nguyễn Hữu Khôi 31 Jul '25

31 Jul '25

Hello. I cannot upgrade clusters via magnum_cluster_api. These are log from magnum_api: https://anotepad.com/notes/c9h8bd55 OPS: 2025.1 OS: Ubuntu 24.04 magnum_cluster_api: v0.30.0 Would you give some ideas on it? Thank you! Nguyen Huu Khoi

1 1

[Neutron][VPNaaS] Multiple tunnels per IPSec Service
by Shubham Yadav 31 Jul '25

31 Jul '25

Hello OpenStack Community, I'm working with OpenStack VPNaaS (using strongSwan as the backend driver) and have a question about tunnel configuration capabilities. *Current Setup:* OpenStack VPNaaS with strongSwan driver (StrongSwanDriver) Single VPN Service (gateway) with multiple IPSec Site Connections Each connection targets different remote peers with distinct subnets *Configuration Details:* VPN Service: Connected to a single router Multiple IPSec Site Connections under the same VPN Service: Connection 1: To MY office (10.10.100.0/24) via peer 68.x.y.34 Connection 2: To Rackspace (172.24.60.0/22) via peer 184.n.p.53 Shared IKE and IPSec policies across connections Local endpoint groups covering multiple subnets (192.168.0.0/24, 10.43.0.0/16) *Question:*Is it supported/recommended to have multiple IPSec Site Connections under a single VPN Service? I understand this creates multiple tunnels from the same OpenStack router to different remote peers. *Current Issue:*One of my connections shows "PENDING_CREATE" status while the other is "ACTIVE". I'm trying to determine if this is: A configuration issue on my end A limitation of having multiple tunnels per VPN service A strongSwan-specific behavior in the OpenStack context *Specific Questions:* Does VPNaaS officially support multiple IPSec connections per VPN Service? Are there any known limitations or best practices when using multiple tunnels? Should each remote site have its own dedicated VPN Service instead? I've checked the documentation but couldn't find clear guidance on multi-tunnel scenarios. Any insights from the community would be greatly appreciated. *Environment:* OpenStack version: 2024.2 Dalmatian Neutron VPNaaS with strongSwan backend Kubernetes-deployed OpenStack (OpenStack-Helm) Thank you for your time and assistance! Best regards, Shubham

1 0

[Neutron][VPNaaS] Multiple tunnels per IPSec Service
by Shubham Yadav 31 Jul '25

31 Jul '25

Hello OpenStack Community, I'm working with OpenStack VPNaaS (using strongSwan as the backend driver) and have a question about tunnel configuration capabilities. Current Setup: OpenStack VPNaaS with strongSwan driver (StrongSwanDriver) Single VPN Service (gateway) with multiple IPSec Site Connections Each connection targets different remote peers with distinct subnets Configuration Details: VPN Service: Connected to a single router Multiple IPSec Site Connections under the same VPN Service: Connection 1: To MY office (10.10.100.0/24) via peer 68.x.y.34 Connection 2: To Rackspace (172.24.60.0/22) via peer 184.n.p.53 Shared IKE and IPSec policies across connections Local endpoint groups covering multiple subnets (192.168.0.0/24, 10.43.0.0/16) Question: Is it supported/recommended to have multiple IPSec Site Connections under a single VPN Service? I understand this creates multiple tunnels from the same OpenStack router to different remote peers. Current Issue: One of my connections shows "PENDING_CREATE" status while the other is "ACTIVE". I'm trying to determine if this is: A configuration issue on my end A limitation of having multiple tunnels per VPN service A strongSwan-specific behavior in the OpenStack context Specific Questions: Does VPNaaS officially support multiple IPSec connections per VPN Service? Are there any known limitations or best practices when using multiple tunnels? Should each remote site have its own dedicated VPN Service instead? I've checked the documentation but couldn't find clear guidance on multi-tunnel scenarios. Any insights from the community would be greatly appreciated. Environment: OpenStack version: 2024.2 Dalmatian Neutron VPNaaS with strongSwan backend Kubernetes-deployed OpenStack (OpenStack-Helm) Thank you for your time and assistance! Best regards, Shubham

1 0