- openstack-discuss - lists.openstack.org

List Limits returns Null in Nova Unified Limits
by Tyler Wilson 11 Nov '25

11 Nov '25

Hello All, I'm looking into testing out the Nova unified limits, however I am experiencing an odd issue where the 'limit list' returns no data. The 'limit create' and 'limit show' (with the UUID from the create command) seem to work fine. Paste data for the output is here: https://pastebin.com/sPZgZDEK I've even tried sending a GET to http://KEYSTONE_IP:5000/v3/limits with the output of [] ({"limits": [], "links": {"next": null, "self": "http://KEYSTONE_IP:5000/v3/limits", "previous": null}}) The log in keystone WSGI just shows a 200 for the GET request, not much else I could find. Should this work in the way I am expecting here? Or am I missing something? Thanks for any and all assistance!

2 5

[cinder][PTG] 2026.1 Gazpacho PTG summary
by Jon Bernard 11 Nov '25

11 Nov '25

Hello everyone, thanks for a very productive PTG for the Gazpacho cycle. Below is a brief summary of the highlights. The raw etherpad notes are here [1] and both the summary and transcribed etherpad notes are captured in out wiki here [2]. # Retrospective We spent time discussing the good and not-so-good of the flamingo cycle. Unsurprisingly, code reviews continue to be our most significant opportunity for improvement. We reviewed what a core reviewer is and how to work towards becoming one, including the metrics that we use to determine eligibility and what we look for in a potential core reviewer. I encourage anyone interested in contributing the project to reach out, in addition to our guideline document on the review process, we're happy to answer questions and help guide folks through the process. # Festival of Reviews We are reworking our 'festival of reviews' meeting to better accommodate a larger set of potential patches. Previously it focused on very small patches that were easier to review, but feedback suggests that time spent as a team on larger reviews would yield a more desirable result. Last Friday we ran the modified format and it seemed to work quite well. # 3rd Party CI This cycle 3rd party CI is required for patches to be reviewed. In the flamingo cycle we saw some challenges with deploying and configuring the software factory setup required to operate a CI. We remain committed to helping vendors sort out any remaining problems, and want to avoid any further slipping on our requirements, as one-off exceptions do not scale. # Eventlet Removal There will be significant focus this cycle on the removal of eventlet from our codebase. Our goal is to have this work completed before M2. The remaining issues and technical challenges were discussed, including some struggles with our multi-backend and it's use of sqlalchemy. We learned that the oslo team is working on some code that may help with this effort. We may have finally found a use for the rally job, unfortunately it has been broken for some time, we will try to get this fixed and reporting useful results this cycle as well. # Driver Block Geometry An original patch to the RBD driver was proposed to make configuring block geometry an option for the operator. Upon inspection, we decided make this option available for all drivers. Discussion about exact location given out driver class model, we have a way forward. https://review.opendev.org/c/openstack/cinder/+/658283 # Per-Tenant SVM Creation Doug Goldstein showed up and blew our minds with a proof-of-concept feature he added to the NetApp driver. This change allows the driver to create per-tentant SVMs and map them to dynamically created backends. There will be some technical hurdles to overcome, but the idea in theory could work. Doug is going to work to propose a patch upstream and with the NetApp team to see if it can take shape. # OpenAPI Stephen has had patches up for openAPI compliance for far too long. We don't have any technical objections to the effort. Brian and I committed to working through the proposed patches at the very start of the cycle. Significant progress already, should be completed soon. # Reverting To Any Snapshot Currently you can only roll a volume back to the last available snapshot. We would like to be able to select any snapshot as a rollback target. Technical requirements were discussed, including the need for a spec. Hoping to see progress on this effort in this cycle. # Clone Across Pools We are interested in improving our implementation for cloning a volume across different pools. The code itself has some distinct shortcomings and Rajat identified several opportunities for improvement. Feedback on current usage was requested, we discussed volume caching as part of these improvements. A path forward was defined and spec should follow. # NFS Driver Feedback Fernando has been working on several improvements to the generic NFS driver and requested feedback. NetApp offered valuable suggestions, some improvements to our tempest testing patches are forthcoming, and a direction forward is understood. # Replication Feedback We solicited feedback on replication, its current usage, and the features it may require to see wider adoption. Questions and answers were discussed, hoping to see a spec capturing the most valuable changes in this cycle. [1] https://etherpad.opendev.org/p/cinder-gazpacho-ptg [2] https://wiki.openstack.org/wiki/CinderGazpachoPTGSummary -- Jon

1 0

[ironic] [all] Ironic PTG Summary
by Jay Faulkner 11 Nov '25

11 Nov '25

Hi all, The Ironic team had another successful PTG planning our 2026.1 release. As usual, we've published a summary of the discussions into the work items document we'll be referencing throughout the year. https://specs.openstack.org/openstack/ironic-specs/priorities/2026-1-workit… If you'd like more in-depth detail as to discussions, please review the etherpad for the PTG at https://etherpad.opendev.org/p/ironic-ptg-2026.1 -- everyone is also welcome to respond here or come join us in OFTC #openstack-ironic to discuss any of these items if needed. Thanks, Jay Faulkner G-Research OSS

2 2

[PTG] Call for Summaries, Attendance Numbers, and Team Photos
by Kendall Nelson 10 Nov '25

10 Nov '25

Hello Folks! I am working on a summary post for our OpenInfra Blog as usual and I am missing A LOT of attendance numbers from OpenStack teams. Specifically, I am missing attendance numbers from: - Barbican - Blazar - Cinder - Cloudkitty - Cross Deploy - Designate - Eventlet Removal - Glance - Heat - Horizon - i18n - Ironic - Keystone - Kolla - Magnum - Neutron - OpenStack Ansible - OpenStack Helm - Sunbeam - Swift - Tacker - Telemetry - Watcher If you were a moderator for this session please send me your approximate attendance to ptg(a)openinfra.org ASAP. If you simply attended one of these sessions, and you would be so kind as to remind the moderator to send me the info I would be very grateful! Additionally, I will happily accept any team photos that were taken and links to blog posts or other summaries that I can include in OpenInfra Blog post as well. There are a lot of open source communities outside of the OpenInfra Foundation that are curious about how we conduct our PTGs and these posts are incredibly helpful at educating others on how to follow the four opens and build a stronger community. Thank you for all of your support! Looking forward to the next PTG! -Kendall

2 3

[nova]Migrating instances with anti-affinity rules causes them to enter ERROR state
by Nguyễn Hữu Khôi 10 Nov '25

10 Nov '25

Hello, I have two instances of an anti-affinity rule. When I move instance a to the host which contains instance b then instance a enters error state. It will be nice if we keep instances at source compute and notify. My env: Openstack 2025.1 Ubuntu 24.04 Nguyen Huu Khoi

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2026.1/R-21)
by Goutham Pacha Ravi 10 Nov '25

10 Nov '25

Hello Stackers, We are twenty-one weeks away from the date of the coordinated release of OpenStack 2025.2 "Gazpacho" [1]. Next week is Milestone-1 of the release. This is typically a bug and feature tracking milestone for project teams. Hope you have been catching up on all the post-PTG action from the past week on this mailing list and on IRC. The OpenStack Technical Committee's PTG summary was shared as well [2]. We are hoping to check on the pending items through our weekly meeting. In the past week, the Telemetry team welcomed back Juan Larriba (jlarriba) as its PTL for the current release cycle. There have been no other governance updates from the OpenStack Technical Committee to report, however, we are looking to close out a long list of pending changes to fully retire the Monasca project [3]. === Weekly Meeting === The last weekly meeting of the OpenStack TC occurred on November 4th 2025 in the #openstack-tc channel on OFTC [4]. It was quite brief since we were still recovering from all the sync meetings from the Summit and the PTG week. We covered the follow-up from the awesome APAC/EU-friendly TC meeting and realized we need to nail down some owners for the user survey action items that we took. We noted that the deadline for getting new questions in for the next survey is December 1st 2025. We had a good discussion about tracking and preserving project PTL appointment history on the governance repository. On the CI front, we hit a brief Gerrit outage, likely caused by an OOM issue that one of our generous providers, Vexxhost, was looking into. OpenDev administrators were working through a Zuul bug that is causing multinode jobs to jump between providers. We also noted the transition of the "stable/2024.1" branch to "unmaintained/2024.1" - this transition affected all repositories that contained the stable/2024.1 branch. A link to a discussion on the CRA (Cyber Resilience Act) was shared [5], with a prevailing interpretation being that individual open source contributors/maintainers have no legal obligations under it. Overall, this was a light meeting, but things are about to get heavy once we digest all the PTG takeaways. The next meeting of the OpenStack TC is on November 11th 2025 at 1700 UTC. It will be hosted on the #openstack-tc channel on OFTC. The agenda for this meeting is on the meeting's wiki page [6]. I hope you will be able to join us. === Governance Proposals === ==== Merged ==== - Propose jlarriba to Telemetry PTL | https://review.opendev.org/c/openstack/governance/+/964516 ==== Open For Review ==== - Retire Monasca project | https://review.opendev.org/c/openstack/governance/+/953671 === Upcoming Events === - 2025-11-10: KubeCon NA: https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america - 2025-11-15: Celebrating 15 years of OpenStack, Montréal: https://www.meetup.com/montreal-openstack/events/310374679/ - 2025-11-15: KCD + OpenInfra Days, Hangzhou: https://sessionize.com/kcd-hangzhou-and-oicd-2025/ Thank you very much for reading On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2026.1 "Gazpacho" Release Schedule: https://releases.openstack.org/gazpacho/schedule.html [2] OpenStack TC PTG Summary: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] Monasca Retirement: https://review.opendev.org/q/hashtag:%22monasca-retirement%22+(status:open%… [4] TC Meeting IRC Log, Nov 4th 2025: https://meetings.opendev.org/meetings/tc/2025/tc.2025-11-04-17.00.html [5] CRA update: https://lists.openinfra.org/archives/list/foundation-board@lists.openinfra.… [6] TC Meeting Agenda, Nov 11 2025: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[ironic] Proposed: Reviewer & Approver team updates
by Jay Faulkner 10 Nov '25

10 Nov '25

Ironic approvers had a discussion about the current state of the core team, and we propose the following changes. As some context for those not familiar, Ironic has two review groups: - Ironic-Approvers -- this group has the ability to +2 changes and Workflow +1 changes - Ironic-Reviewers -- this group has the ability to +2 changes, but cannot Workflow +1 changes I am proposing the promotion of our two inaugural members of ironic-reviewers into approvers. Both of these contributors have used their +2 judiciously and have earned the trust of the community to use that same judgement about which changes to approve: * Afonne-CID (cid) from GR-OSS (move from ironic-reviewer to ironic-approver) * Doug Goldstein (cardoe) from Rackspace (move from ironic-reviewer to ironic-approver ) Additionally, we'd like to add an existing community member to ironic-reviewers. Jakub Jelinek has become the community expert on RAID support in IPA and has deepened his knowledge across the codebase. We'd like to recognize that by asking him to join the ironic-reviewers team. * Jakub Jelinek (kubajj) from CERN (add to ironic-reviewer) Finally, we would like to move an existing ironic-approver and longtime contributor to emeritus status. Mark Goddard is no longer active in the OpenStack community, and I propose he be moved to emeritus status. This just means that while he may be removed from the team, if at some future point he returns to the Ironic community we may expedite his return to the approvers group. * Mark Goddard (remove from ironic-approvers) Thanks everyone for your contributions to Ironic, current and future. It's nice to see the growth of the team, and hope this opens the door for CID, cardoe, and kubajj to grow themselves and the team further. Thanks, Jay Faulkner G-Research OSS

4 4

[Nova] Spec Soft Freeze Approaching November 20!
by René Ribaud 10 Nov '25

10 Nov '25

Hi everyone, Time really flies, we’re already approaching the Nova Spec Soft Freeze for the Gazpacho cycle! Just a quick reminder that after November 20, 2025 (23:59 UTC), no new specs will be accepted. If you’re currently working on a proposal or reviewing one, now’s the time to propose it. Let’s make the most of the remaining days, and thank you all for keeping the momentum going! Cheers, René

1 0

[openstack-ansible] Bug fighting day - Save the date
by Dmitriy Rabotyagov 10 Nov '25

10 Nov '25

Hi everyone, This week, on Thursday, *November 13*, we are going to hold our yearly bug-fighting day. We plan to start active work around *1:00 PM UTC*. The activity aims to gather all Core Reviewers along with interested community members to join forces to review, triage and propose patches to as many bug reports as possible. This should contribute towards project stability and reduce the amount of issues in the project codebase. We invite everyone interested to participate to join our #openstack-ansible IRC channel on OFTC network. Alternatively, you can join the room #_oftc_#openstack-ansible:matrix.org with your Matrix (Element) account. We are going to use an Etherpad [1] to work in an asynchronous way and to ensure that different people will not be working on the same bug reports. Please sign in in the Etherpad [1] participants list before starting the work, so we could estimate participation and scope we can pull during the day. [1] https://etherpad.opendev.org/p/osa-nov-2025-bug-triage

1 0

[openstack-ansible][ptg] 2026.1 Gazpacho Summary
by Dmitriy Rabotyagov 10 Nov '25

10 Nov '25

Hi everyone. OpenStack-Ansible team has held a PTG session, where discussed left scope before 2025.2 (Flamingo) Release as well as development scope for 2026.1 releases. For 2025.2 we agreed to finalize these topics: - Debian 13 (Trixie) support - Fix playbooks to address ANSIBLE_GATHER_SUBSET removal in ansible-core 2.18 - Finalize OpenBao backend for the PKI role As I am writing a summary late this time, 2 out of 3 topics (Debian 13 support and ANSIBLE_GATHER_SUBSET replacement) have already been concluded. With that being said, we move couple of topics to 2026.1 (Gazpacho) cycle, specifically: - Mainstreaming Magnum CAPI drivers (promoting from ops repo to the integrated one) - Molecule coverage for the HAProxy role For the 2026.1 (Gazpacho) release cycle, the following topics were raised: - Improving MariaDB proxying and introducing read/write balancing. Originally we agreed to adopt ProxySQL, as we had plans for that since the Xena release, but it was never prioritized. Now we've realized that there is a possibility to leverage oslo.db's `slave_connection` parameter, which should be quite trivial to add, while providing similar behavior with just HAProxy balancing. - Big topic was adoption of ansible-core 2.19 and related challenges with role patching as well as our custom modules adoption to it. And we agreed to prioritize this work, as ansible-core 2.18 goes EOL in May 2026. - Once HAProxy role will be covered with molecule tests, we agreed to proceed with potential role refactoring, and removing obscure variables and parameters. Specifically, frontend definition is a focus, as it has some legacy, which prevents it being flexible enough for 3rd-party usage of the role. - With that we highlighted existing complexity regarding non-TLS -> TLS migrations, but uWSGI remains a blocker to move to TLS-only deployments, as HTTP 1/1 remains to be a bottleneck in terms of throughput. - We agreed to proceed to refactoring a standalone method for PKI role, and remove the necessity of storing individual certificates and private keys on the deploy host. - Attempt to add Ubuntu 26.04 support for 2026.1 Gazpacho - We touched basis on post-quantum encryption and figure out dependencies and blockers for that. With Debian 13 and Ubuntu 26.04 we will have OpenSSL 3.5, which supports already ML-KEM. However, a bunch of software, like uWSGI are still gonna be blockers to implement PQC deployment-wise. Support for things like RabbitMQ, MariaDB and etc remains unknown.

1 0

[ops][largescale-sig] Refactoring Ops-related activities - a discussion at the PTG
by Thierry Carrez 10 Nov '25

10 Nov '25

Hi everyone, A quick reminder that we'll have a session at 13:00UTC today in the PTG on how we can regroup the operators-facing activities between our various groups: the Ops meetup group, the Large Scale SIG, the Public Cloud SIG, the Scientific SIG... Join us if you can! -- Thierry Carrez (ttx)

2 2

LXB Removal and OVN migration
by Ricardo Cano 09 Nov '25

09 Nov '25

Hello openstack-discuss, With the removal of the linuxbridge mechanism driver, has anyone successfully completed an in-place migrationfrom LinuxBridge to OVN? I did come across this https://www.jimmdenton.com/migrating-lxb-to-ovn/ and that looks good as a one-shoit while biting the bullet and taking a full outage. However, I was wondering if it was possible to run lxb and ovn side-by-side and cut them over, say one network at a time. This particular deployment was stood up using openstack-ansible back in the rocky days (now running antelope) and it appears that openstack ansible's default was lxb up until Zed. Any suggestions or recommendations on the best path forward? Also, will the dynamic routing agent work with ovn?

4 3

[openstack helm] [Glance] Glance showing OSError: unable to receive chunked part when uploading image
by daniel890723＠gmail.com 08 Nov '25

08 Nov '25

Hello i'm testing using Openstack-helm to install all component but after the installation complete i can't upload the OS image through both openstack CLI or Horizon it stuck at Queueing and when i checking Glance Api log it shows 2024-11-07 07:31:19.362 7 INFO glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2024-11-07 07:31:19.400 7 ERROR glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Failed to upload image data due to internal error: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Caught error: unable to receive chunked part: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Traceback (most recent call last): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1302, in __call__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1345, in dispatch 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return method(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 415, in wrapped 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 299, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 162, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 492, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 443, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/quota/__init__.py", line 322, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 596, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 487, in _upload_to_store 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 397, in add_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 479, in store_add_to_backend_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/driver.py", line 294, in add_adapter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/capabilities.py", line 176, in op_checker 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 764, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise errors.get(e.errno, e) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 746, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi for buf in utils.chunkreadable(image_file, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/common/utils.py", line 73, in chunkiter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = fp.read(chunk_size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 294, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = self.data.read(i) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 121, in readfn 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = fd.read(*args) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/format_inspector.py", line 956, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = self._source.read(size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1045, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi data = uwsgi.chunked_read() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Ceph Cluster Status: NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID ceph /var/lib/rook 3 82m Ready Cluster created successfully HEALTH_OK 5cf8b9bb-13e1-4c31-b47d-9e7d92c3c05d Ceph osd status: ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS TYPE NAME -1 0.35156 - 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 1.00 - root default -5 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 - host instance-lu-1 2 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 195 up osd.2 -3 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-2 0 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 195 up osd.0 -7 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-3 1 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 197 up osd.1 TOTAL 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 MIN/MAX VAR: 1.00/1.01 STDDEV: 0.01 now i'm able to create Empty Volume using horizon(can't create volume using default cirros image ) and i can start an Instance without volume and mount the empty volume in(Ceph also can detect when i use that volume) is there something i need to set when starting glance? because i'm following the Openstack-helm installation document to install openstack with helm

5 8

[all][tc][ptg] OpenStack Technical Committee Gazpacho PTG Summary
by Goutham Pacha Ravi 07 Nov '25

07 Nov '25

Hello Stackers, The community had a very productive "design summit" at the Project Teams Gathering last week. As you're reading the various summaries posted here, and participating in new initiatives that have rolled out since, I'd like to add a brief summary of the OpenStack Technical Committee's meetings [0] at the PTG. We recorded all of these sessions and have posted the recordings to the TC's YouTube channel [1]. Alongside this summary, I'd recommend reading the Eventlet Removal Goal updates and discussion from the session that Hervé Beraud (hberaud) hosted [2]. Remember, Etherpads are true to their name—you can lose them :D So we made a handy reference to the PTG Etherpads on the wiki [3]. Community Leaders' Forum ====================== We discussed active Community Goals [4], and took questions regarding the status from various project maintainers. As it stands, the FIPS compliance goal may no longer fit the criteria for community goals. It's been challenging due to the lack of a stable OS to test FIPS compliance in the CI. The community may need to explore CentOS Stream alternatives such as Rocky Linux 10 and Alma Linux 10. We need someone to pick up the work and replace existing test jobs. Secure RBAC has got an active Goal Champion; however, work has slowed down in several project teams. We're hoping to make another push through the Gazpacho cycle. We agreed that the current cross project liaisons information (on a wiki) is severely outdated and needs to be migrated to the openstack/governance repository, a centralized, programmatic location, to avoid duplication and ensure accuracy. We also discussed the OpenInfra AI contribution policy. Some project teams have recently seen an uptick in AI-assisted contributions, but there has been some confusion, and potential stigmatization of such contributions. We identified a need for clarity on how to apply the AI policy, specifically regarding using AI-generated code and verifying copyrights. We could use some documentation and guidelines for contributors and reviewers. Action Items: - Goutham (gouthamr) will move the FIPS goal from "selected" to "proposed" and start a separate ML thread. Project teams can remove any failing FIPS jobs when this change merges. - Tony Breeds (tonyb) will enhance the project schema in the governance repository to support adding cross-project liaisons. - (needs volunteers) The FIPS goal needs rework, especially around testing, and any objective refinements. - (needs volunteers) Clarifications and instructions pertaining to AI Contributions in the Project Team Guide. Testing and Runtimes ================= We discussed testing matrix updates for MariaDB/MySQL versions and Linux distributions. The goal is to drop older, unmaintained versions and move to modern platforms (like newer Python versions, CentOS Stream, AlmaLinux, and Rocky Linux) to ensure compatibility and support. AlmaLinux images are now available, but no projects are currently testing with them. Kolla and OpenStack-Ansible project teams are currently interested. The biggest advantage to project teams with this would be that the wide gamut of CI providers are compatible with this Red Hat-like distribution because of its support for x86_64v2 and x86_64v3 CPU architectures (as opposed to CentOS Stream or Rocky Linux). The CentOS Stream community has offered to run a third-party CI to test CentOS Stream with OpenStack, and the team noted a good meeting between maintainers in both communities at the OpenInfra Summit in Paris that Amy Marrich (spotz) facilitated. A precursor to this discussion was the concern that the latest LTS versions of MariaDB/MySQL have changed the default behavior with respect to character sets and collations, and several OpenStack services are non-compliant with these changes. We agreed that OpenStack services need to specify the character set on tables explicitly, and so creating a community goal to mandate the use of a specific character set (and to handle character set migrations) was seen as a good path forward. We briefly discussed OpenStack Special Interest Groups. We noted that we could use fresh updates to the SIGs website, including refreshing the SIG Chairs of several SIGs. There was a suggestion that, during this exercise, we could suggest merging some SIGs, such as the Scientific SIG and other operationally related SIGs, under a more general OpenStack operators umbrella. Action Items: - Dmitriy Rabotyagov (noonedeadpunk) will propose a community goal for project teams to support character set changes in newer LTS releases of MariaDB/MySQL. - tonyb will refresh the data for all the SIGs. - gouthamr will merge the governance and governance-sigs repositories for ease of management. University Partnerships and Collaborations ================================== Kendall Nelson (diablo_rojo) would like us to maintain an ongoing list of projects and mentors for OpenStack's storied university interns. This would alleviate bottlenecks and increase student engagement in OpenStack. There's now an Etherpad for this purpose [5]. If you'd like to mentor, please do sign up. We also entertained the idea of combining efforts with the First Contact SIG, as it has not been active in a while, to create a running list of mentors and participants for various onboarding events and activities throughout the community. Action Items: - Community members sign up and propose projects [5] - Revive First Contact SIG by collaborating with the University Partnership Projects. Post-Quantum Cryptography ======================= We discussed the potential breaking of traditional asymmetric cryptography by quantum computers around 2030. JP Jung (JP) joined us to detail that there are 17 cryptographic modules in OpenStack, and the goal is to understand the problem in each project, catalog the cryptographic algorithms used, and make the entire chain well-known to achieve quantum safety. A significant concern was that OpenStack has security threats today, and the energy should be directed towards securing it for current use cases as well, not just focusing on potential future threats like quantum computing. It was noted that the Python ecosystem has minimal support for post-quantum cryptography, and Python's cryptography project has not committed to a timeline for adding support, so building a foundation with cryptography or alternative libraries might be necessary before focusing on OpenStack. JP proposed a document for the community to review [6]. Much of it probably has to be less verbose and focus on specific gaps/changes. JP's willing to be a champion of the effort to secure OpenStack in a post-quantum world. He'd like project teams to audit the use of encryption algorithms, key exchange libraries/mechanisms in their deliverables. The Security SIG is a good venue to continue this discussion, and, if necessary, a sub-team or a separate pop-up team can be formed as well. Action Items: - JP will fix up the current analysis [6] to make it specific, and highlight actionable problems. OpenStack Universe a.k.a Affiliated Projects ================================== Artem Goncherov (gtema) wanted to address the issue of signaling to users that certain tools work well with OpenStack, even if they are not official OpenStack projects. The idea is to create a connection between OpenStack and these affiliated projects, giving them recognition and a voice, and potentially providing some protection from competing projects. Thierry Carrez (ttx) suggested that the OpenInfra Universe [7] is an existing platform that lists projects in a similar category, with criteria including OSI-recognized licenses and integration with Open Infra projects. We brainstormed how to evolve the current project intake form to better represent affiliated projects, with a potential meeting to include OpenInfra Marketing folks and explore options. Action Items: - ttx will set up a meeting with interested parties to improve the OpenInfra Universe, and will share updates in a future OpenStack TC meeting. Bridging the gap between community and contributing orgs ============================================= Ildiko Vancsa (ildikov), Jeremy Stanley (fungi) and Clark Boylan (clarkb) provided an update on the effort to bridge the gap between the community and contributing organizations. There are two surveys that are currently open to contributors [8] and maintainers [9]. These surveys focus on the Flamingo release cycle and will end on November 16th, 2025. (Please take them!) Once these surveys end, we're expecting them to inform us of where things are going well and where they are not, and to measure the effectiveness of changes we've slowly been making across the community, with the ultimate goal of improving the contributor experience. There is a perception that getting a feature into OpenStack requires a personal relationship with a maintainer, which can be a barrier to entry for new contributors. The complexity of OpenStack's CI and software can also be a significant technical element that makes it difficult for external contributors to get started. Collectively, we're having trouble finding new core maintainers and building knowledge in parts of our code base to sustain them. Extending trust and lowering barriers to entry can help address these issues, and projects should consider reevaluating their processes to make it easier for new contributors to get involved. It was also highlighted that #openstack and #openstack-dev IRC channels on OFTC have become less active over time. We explored the idea of revitalizing these channels as preferred venues for cross-project discussions, and having maintainers parked there to engage with folks that aren't familiar with project-specific channels or have cross-project concerns. Action Items: - Please take the contributor [8] and maintainer [9] surveys for the 2025.2 "Flamingo" release. - (ildikov/fungi) Analyze survey results when surveys close and meet with project teams and the OpenStack TC to discuss them. Project Cascade ============= Professor Corey Leong (profcorey) introduced the Cascade project, an open-source unified communications project. He asked us about the steps to make it an official OpenStack project, including potential voting and governance requirements. The project aims to integrate unified communications services, such as voice, phone, and video using open-source projects, including OpenStack components. It'd allow OpenStack users to download and install the service on their machines. Participants drew a similarity to Trove in that Cascade would provide a managed service to OpenStack users akin to Trove providing database-as-a-service. We opined that the project is still in the early stages, and the team is researching how to integrate with OpenStack components, including identifying dependencies and interactions between services. In this stage, all of the community's infrastructure: hosting on OpenDev, publishing on wiki.openstack.org, using official IRC channels on OFTC, Meetpad, Gerrit, and Zuul, are available to Project Cascade's maintainers. We tried to understand the benefits of switching to the OpenStack namespace, i.e., bringing the project under OpenStack governance and what it would provide that is not currently available. We hope to continue this discussion in a future TC meeting or on the openstack-discuss mailing list. Action Items: - tonyb is working on fixing OpenID auth issues with wiki.openstack.org and will coordinate with profcorey on problems with other maintainer infrastructure. Now that we have that out of the way, we can focus on some other important things [10]. Again, thank you very much for participating and for the great discussions over the week! Sincerely, Goutham Pacha Ravi (gouthamr) Chair, OpenStack Technical Committee [0] OpenStack TC PTG Etherpad: https://etherpad.opendev.org/p/r.5dfdc57d7bc31731ccf9d51cb80de6c5 [1] OpenStack TC Gazpacho PTG playlist: https://www.youtube.com/playlist?list=PLhwOhbQKWT7WkK5VGgipjt4XQQlgLLci9 [2] Hervé's summary of eventlet-removal: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] "Gazpacho" Oct 2025 PTG Etherpads: https://wiki.openstack.org/wiki/PTG/Oct2025PTG/Etherpads [4] OpenStack Wide Goals: https://governance.openstack.org/tc/goals/ [5] University Mentors Sign Up: https://etherpad.opendev.org/p/UPP-Projects%26Mentors [6] Post Quantum OpenStack: https://wiki.openstack.org/wiki/Post_quantum_openstack [7] OpenInfra Universe: https://openinfra.org/universe [8] Flamingo Contributor Survey: https://openinfrafoundation.formstack.com/forms/openstack_contributor_exper… [9] Flamingo Maintainer Survey: https://openinfrafoundation.formstack.com/forms/openstack_maintainer_experi… [10] https://cooking.nytimes.com/recipes/1017577-best-gazpacho

1 0

[i18n] Summary of 2026.1 Gazpacho PTG Discussions
by Seongsoo Cho 07 Nov '25

07 Nov '25

Hello all, I would like to share with you a summary of the recent discussions held during the 2026.1 Gazpacho virtual PTG regarding the i18n SIG. We had several productive sessions focused on our two main goals for the upcoming cycles: the Weblate Migration, and a forward-looking discussion on AI Translation. Special thanks to the OpenDev Infra Team for the infrastructure support, the OpenSource Contribution Academy mentees in Korea who worked on the Weblate migration, and the KNU students who brought great energy and ideas to our AI discussion. Please refer to the full details from our Etherpad: https://etherpad.opendev.org/p/oct2025-ptg-i18n 1. Weblate Migration Status & Challenges A significant portion of our PTG was dedicated to the ongoing migration from Zanata to Weblate. - Migration Status: DaGyeong Kim presented the current status. 15 dashboard projects were initially migrated, but we identified issues that require a retry. A detailed status page is tracking all projects [1]. - Tooling Development: We are actively developing and refining automation scripts to manage the migration, including language creation/deletion and handling plural forms (patches [2], [3]). This work is being tracked in a blueprint [4]. - Key Challenge - Language Definitions: We are addressing mismatches between Zanata's language codes (e.g., ko_KR) and Weblate's CLDR standards (e.g., ko), especially concerning plural form definitions. - Verification of Migration Completion: It was discussed that a clear method needs to be defined to verify whether all translated PO files in Zanata have been successfully migrated to Weblate. In subsequent discussions, it was agreed to establish specific criteria for accurate verification and to proceed with the work based on those criteria. 2. Zuul Integration & Infrastructure The second major topic was integrating our new Weblate instance with the OpenDev (Zuul) CI/CD pipeline. - Core Patches: We reviewed the main patches for openstack-zuul-jobs [5] and project-config [6] that will enable Weblate to propose translation updates. - New Zuul Job Testing Strategy: It was discussed to create a completely new job to minimize the impact and to apply it to specific projects first, gradually expanding its scope over time. 3. AI Translation PoC We held an exciting session with students from Kyungpook National University (KNU) to discuss a Proof-of-Concept (PoC) for using AI in our translation workflow. - Goals: The discussion focused on two potential tools: - An automated, CPU-based LLM tool (e.g., using Ollama) to catch new strings from merged patches and provide initial translations. - A local AI-assisted tool for translators to help with the initial translation of large .po files. - Next Steps: We are exploring different instance flavors for performance and comparing the pros/cons of self-hosting a model versus using external APIs (similar to the Firefox translation model). 4. I18n Repository Packaging We are working to package the openstack i18n repository [7] and publish an updated package to PyPI. This initiative aims to enhance productivity by independently managing the tools related to i18n. Best Regards Seongsoo Cho [1] https://wiki.openstack.org/wiki/I18nTeam/Migration-to-weblate#List_of_proje… [2] https://review.opendev.org/c/openstack/i18n/+/961371 [3] https://review.opendev.org/c/openstack/i18n/+/963758 [4] https://blueprints.launchpad.net/openstack-i18n/+spec/weblate-migration [5] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/921878 [6] https://review.opendev.org/c/openstack/project-config/+/961499 [7] https://review.opendev.org/c/openstack/i18n/+/955037

3 2

[manila] 2026.1 Gazpacho PTG summary
by Carlos Silva 07 Nov '25

07 Nov '25

Hello, Zorillas and interested stackers. Here's a summary of all the discussions, decisions, action items and plans we came up with during last week's PTG. All of the recordings are available in this playlist [10] from the OpenStack Manila Youtube channel. In case you missed something and would like to get back to the discussions, please refer to it. Flamingo Retrospective ================== We discussed what went well and what needs some attention in the upcoming cycles. - Bugsquash events and collaborative review sessions are having a positive impact on bugs being closed and features being reviewed faster. - The mid cycle helped with the reviews of the changes prior to the feature freeze. - We had several interns working on Manila UI, OpenAPI patches and fixing other known issues in Manila, providing good opportunities for community engagement. - We had enhancements on the review bandwidth but we should keep looking on adding more core reviewers to Manila and help share the load on reviews. We didn't manage to cover all topics, so the retrospective will continue in the next Manila upstream weekly meeting, November 13th. All Things CephFS: Status and Gazpacho Plans ===================================== Status updates ------------------------ - Allocated Capacity GiB fixes and thin provisioning improvements are being backported to older releases. - fs_name metadata is now also used in Manila CSI [2] - Patches to introduce CI testing for CephFS manage/unmanage operations will be included in the Gazpacho cycle. Plans for 2026.1 Gazpacho ------------------------------------ - We started testing with Ceph Tentacle release candidate images in the devstack jobs and will merge these jobs once the official Ceph Tentacle release is out. *QoS:* NFS Ganesha v7 supports QoS, i.e., throughput throttling (iops and bandwidth) Manila team intends to start the CephFS NFS driver implementation during this cycle. The implementation is expected to support the new proposed QoS types alongside driver-scoped share type extra specs. *Share encryption (BYOK):* NFS-Ganesha and Ceph communities are working on Bring-Your-Own-Key subvolume encryption. The team will work on a Proof of Concept (PoC) to extend the existing BYOK Encryption feature in Manila to support shares encryption with CephFS. *OVN extension targeting NFS-Ganesha* Discussion is ongoing with the Neutron project team for adding an extension to the OVN agent [3], which aims to improve usability and security in a cloud that relies on NFS-Ganesha to provide RWX storage volumes to end-user workloads. [NetApp] Sync snapmirror support and Async snapmirror enhancements ======================================================= NetApp is adding support for zero RPO (i.e., synchonous mirroring) with Share Replication - a feature that's been supported by this driver since the Mitaka release. Doing this will involve some more tenant-visible directives. The Manila team considered it as a good enhancement, but suggested implementing share replicas metadata as the vehicle for this change, as this will make it more extensible and reusable in the future. The NetApp team agreed with this approach and will work on implementing share replicas metadata in the Gazpacho cycle. Graduating Migration and Backup APIs ============================== Share backup APIs -------------------------- The Share Backup APIs are mostly feature complete. We're missing a way to force backup deletion and the ability to set backup quota via the CLI and UI. Some improvements are planned for the Share Backup driver abstraction as well. We are also evaluating the possibility of adding testing with CERN's CBACK utility and Backup driver in the CI, and Zac is working on further enhancements for targeted restores. Share migration APIs status ------------------------------------- The share migration APIs have been stable for a while and well tested. We believe that this API is ready to be graduated and we need a contributor to help drive this to completion. Share Encryption follow up ==================== The Manila team discussed the known issues and gaps noted while reviewing and merging this feature in the flamingo cycle: - Inability to unmanage/manage share servers with encrypted shares: the NetApp team will investigate how to implement the changes to the manage share servers workflow. - We agreed that this should also follow steps similar to the creation workflow, as, for example, checking if the barbican key exists, if the user can manipulate it, and if it matches the one from the share server. - Inability to migrate share servers and setup share replication: the NetApp team will look into the required next steps. NetApp - NAE (NetApp Aggregate Encryption) Support ========================================== NetApp is planning to add support for encrypted pools (aggregates) to Manila when DHSS=True is in use [4]. It will allow customers to encrypt the shares (volumes) data by assigning the encryption key to aggregate level. The Manila team is okay with this proposal and the NetApp team will start this implementation in the Gazpacho release cycle. Adding new/failure notification ======================= We currently have notifications for share operations (create, shrink, delete). eunkyung, a contributor from Samsung would like to add notifications for other operations in Manila (creating snapshots, reverting to snapshots and others). The team agreed that this is a great addition, as we started with notifications for shares but didn't have the cycles for adding notifications to other resources. This change will be proposed to Manila and the Manila team will share feedback. Add mount_point_name option to managing share ====================================== It is currently possible to define a custom mount point name to a share export location, but while managing a share, it is not possible. We discussed the possibility of extending this feature to the manage workflow and agreed that this is will be a great improvement for the operators. This feature will be proposed during the Gazpacho cycle and the Manila team will review. QoS enhancements =============== Kiran shared a proposal for introducing QoS types in Manila, with the possibility of having multiple QoS types being linked to a share type. The QoS types will be added to the share servers and shares created. The team agrees that this is a nice addition, and we asked more details in the specification [5] seeking clarifications on spec conflicts and to elaborate on the need to tie the QoS types to the share type itself, instead of using an approach to tie them directly to shares or share servers. Eventlet Removal ============= The manila team is actively working on removing eventlet. We used this topic to talk about the progress from the team, share our plans, roadmap and discuss an issue shared by Eric from the Cinder team, where sqlalchemy objects shared by multiple threads were affected when dropping eventlet from the cinder-volume service. damani joined us for the discussion and has started on changes that are going to address this behavior [6][7]. The Manila team intends to support running all manila components without eventlet by the end of the Gazpacho release. Dropping the v1 API =============== The Manila v1 API was deprecated ~10 years ago and we intend to remove it during the Gazpacho Cycle. Removing it will clean up our code and make enhancements such as the JSON Schema Validation work easier. This work will help us with the OpenAPI schema and documenatation enhancements that we've been slowly closing the gap on for the past few releases. AI: carloss will send an email to the mailing list announcing the removal. JSON Schema Validation =================== We shared the current status of the JSON Schema validation changes in Manila and also talked about the lack of reviews in the open changes. We decided to split changes up into smaller chunks, making it more friendly to new contributors, and to reviewers. AI: carloss will organize a Review Jam to help making progress, as well as unify the tracking of the current issues being worked on in a Taiga board [8]. Manila client removal + merging manilaclient's OSC functionality into OSC ======================================================== The Manila CLI utility was deprecated several release cycles ago and we intend to remove it in the Gazpacho cycle. We also discussed moving Manila's OSC functionality from python-manilaclient into openstackclient itself. Doing this will reduce the burden to discover and import command modules at runtime, significantly improving performance of OSC. We discussed the work that would be needed and how it would affect the release process. NetApp REST/ZAPI situation ====================== Retirement of NetApp's ONTAPI is currently on hold [9]. Many OpenStack users are still heavily relying on ZAPI because of how the NetApp drivers in Cinder and Manila integrate with the product. All new ONTAP features will be added only to REST, with no fallback approach to ZAPI. So, the drivers in OpenStack have started seeing implementation changes over several releases to support NetApp's REST API where available. The NetApp team is also putting efforts into identifying the gaps between REST and ZAPI in Manila. As for CI Testing: Stabilizing CI runs with REST is a priority for the NetApp team, with an engineer actively working on the necessary fixes. Improvements for managing shares/snapshots =================================== The proposal is to update the snapshot's created_at field to reflect the creating time on the storage backend during the manage operation, rather than the time recorded by Manila. We agreed with the proposal but also think this should be applied for the shares. A blueprint will be registered and inyonghong will work on the fix. Dell's PowerScale driver roadmap ========================== We heard requests for a couple of features in the Dell PowerScale drivers: - manage/unmanage for shares and snapshots - revert to snashot - custom mount point names - QoS. We noted that the revert to snapshot feature is already being worked on by the Dell team and the Manila team will review the changes and target them for the gazpacho cycle. [1] https://etherpad.opendev.org/p/gazpacho-ptg-manila [2] https://github.com/kubernetes/cloud-provider-openstack/pull/2994 [3] https://review.opendev.org/c/openstack/neutron-specs/+/936063/3/specs/2025.… [4] https://etherpad.opendev.org/p/nteapp_nae_support [5] https://review.opendev.org/c/openstack/manila-specs/+/962706 [6] https://review.opendev.org/c/openstack/oslo.service/+/963742 [7] https://review.opendev.org/c/openstack/oslo.service/+/965208 [8] https://tree.taiga.io/project/gouthampacha-manila-jsonschema-openapi/kanban [9] https://kb.netapp.com/on-prem/ontap/DM/REST-API/REST_API_KBs/Deferral_of_ON… [10] https://www.youtube.com/watch?v=3O3ByYU6gHc&list=PLnpzT0InFrqAf4kRsXDoVpv0V… Lots of positives to take from this PTG and definitely a productive cycle ahead. Thank you for your participation! Regards, carloss

1 0

[release] Release countdown for week R-20, Nov 10 - 14
by Elõd Illés 07 Nov '25

07 Nov '25

Development Focus ----------------- The Gazpacho-1 milestone is next week, on November 13th, 2025! Project team plans for the 2026.1 Gazpacho cycle should now be solidified. General Information ------------------- Libraries need to be released at least once per milestone period. Next week, the release team will propose releases for any library which had changes but has not been otherwise released since the 2025.2 Flamingo release. PTL's or release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well, by posting a -1. If we do not hear anything at all by the end of the week, we will assume things are OK to proceed. NB: If one of your libraries is still releasing 0.x versions, start thinking about when it will be appropriate to do a 1.0 version. The version number does signal the state, real or perceived, of the library, so we strongly encourage going to a full major version once things are in a good and usable state. Upcoming Deadlines & Dates -------------------------- Gazpacho-1 milestone: November 13th, 2025 Előd Illés irc: elodilles @ #openstack-release

1 0

[Action required] Glanceclient freeze in I cycle
by Cyril Roelandt 07 Nov '25

07 Nov '25

Hello, The Glance team is planning on freezing development on python-glancelient. Starting in the I cycle, the client will go into maintenance mode, and new features will no longer be implemented. Users are encouraged to use the OpenStack client instead and report bugs or missing features to the SDK or OSC projects. The following projects currently use glanceclient as a library: - ceilometer[1] - cinder - cyborg - freezer - heat - heat-translator - horizon - magnum - manila[2] - mistral-extra - nova - octavia - os-client-config - rally-openstack - skyline-apiserver[3] - sunbeam-charms - trove - venus[4] - vitrage-tempest-plugin - zun The Glance team has proposed some migration patches already to some of the affected projects. We would appreciate your team's quick attention to reviewing these patches during this cycle. Additionally, we will be happy to help review migration patches proposed by your team. Use the topic 'remove-glanceclient' so we can easily find them. We're starting the migration process now in Gazpacho so that we can discover and address any feature gaps well in advance of the glanceclient going dark in the I release. On behalf of the Glance team, Cyril Roelandt [1] https://review.opendev.org/c/openstack/ceilometer/+/963632 [2] https://review.opendev.org/c/openstack/manila/+/935835 [3] https://review.opendev.org/c/openstack/skyline-apiserver/+/963862 [4] https://review.opendev.org/c/openstack/venus/+/931291

1 0

[tacker][ptg] 2026.1 Gazpacho PTG summary
by Yasufumi Ogawa 07 Nov '25

07 Nov '25

Hi all, I'd like to share a brief summary of Tacker vPTG. Please refer to our etherpad [1] for whole the duscussion. [1] https://etherpad.opendev.org/p/oct2025-ptg-tacker ## Enhancing Tacker Testing Coverage in the OpenStack Helm Check Pipeline - It's a recap from the previous cycle. To expand the test coverage, improve Tacker testing within the OpenStack Helm (OSH) pipeline by adding more test cases. - Use the OpenStack CLI for Tacker testing, ensuring that no modifications to the OSH test code will be needed even if Tacker's functional tests change. ## Tosca-parser 2.13.0 Validation Failures – Tacker Handling Strategy for Specification Gaps in ETSI SOL001 v2.6.1 - Address the validation failures in tosca-parser happened after it's updated to v2.13.0. - Some VNFDs in tacker's repo are still old and invalid for the stricter validations, but required to be not changed for keeping compatibility with ETSI std version v2.6.1. - We introduced temporary fix just for passing the tests as a remedy, but haven't conclude our policy how we keep the compatibility. As the result, current tacker impl isn't fully compatible with ETSI std. - Continue the discussion for the policy while adding a note for describing the current status of the issue on docs. ## Approach to Prevent Broken Reference URLs in Tacker Documentation - For better maintenance, introduce a periodic job for detecting invalid links. ## Proposal for New Compliance Testcase Implementation for ETSI TST010 Release 3 - Implement new ETSI TST010 Release 3 (v3.3.1) testcases to continue alignment with the ETSI std and strengthen its compliance verification capabilities for VNFM APIs. ## Enabling VNF Support in Prometheus Plugin for Fault Management - Extend Tacker's Prometheus Plugin to enable Fault Management support for VNFs which suppports only CNFs. ## Fix VNF-Package Multi-tenancy issues - Enable multi-tenancy support in VNF Package management by changing the uniqueness scope of vnfd_id from global to per-project. - Due to the current NFV does not support VNF level multi-tenant and it might be a gap between ETSI std, continue to discuss how we fill the gap, for example, making a feedback for multi-tenancy usecase or so. ## Confirm support versions for k8s/Helm/Prometheus on Tacker Gazpacho - The current support versions of k8s/Helm/Prometheus aren't the latest since we've had a failure of updating patch and cannot be fixed yet. - Fix the issue and update to the latest. ## Eventlet removal (Community goal) - Tacker depends on eventlet and need to be modified for the removal. - NEC will discuss if we can proceed with this work. ## Bug triage - Revise Bug maintenance on launchpad for closing too old bugs labeled as critical but can be discarded anymore or so. - Go through the bug list and give comments about their related bugs Thanks, Yasufumi

1 0

[qa] Tempest gate blocked and dropping python 3.9
by Ghanshyam Maan 06 Nov '25

06 Nov '25

Hi All, Requirement repo has dropped the Python 3.9 constraints[1], and due to that, the Tempest gate is blocked because the Python 3.9 unit test job is failing. We discussed this type of issue in the past[2], that if the requirement repo drops older Python version constraints, then Tempest can drop the support of that Pythion version. Tempest master can still be used to test the Python 3.9 env; for that, you need to install tempest in a Python 3.10 (or higher) env. I have proposed the change to drop py3.9[3], and Yatin's change will unblock the stable branch testing[4]. Hold your recheck until those fixes are merged. [1] https://review.opendev.org/c/openstack/requirements/+/957587 [2] https://lists.openstack.org/pipermail/openstack-discuss/2022-May/028711.html [3] https://review.opendev.org/c/openstack/tempest/+/966101 [4] https://review.opendev.org/c/openstack/tempest/+/965903 -gmaan

1 3

[watcher] Unmaintained branch cleanup
by Joan Gilabert 06 Nov '25

06 Nov '25

Hi everyone, Watcher currently has two unmaintained branches[1], unmaintained/2024.1 and unmaintained/2023.1. The latter has not had any patch in 2 years [2] so we plan to move it to eol. We will discuss it in the next Watcher IRC meeting, but I'm writing this message early to give everyone a chance to raise any objection. Please respond to this message if you have any objection, if there are none, I will propose a patch to move the branch next Thursday, after the Watcher IRC meeting and share the link here. Regards, Joan [1] https://opendev.org/openstack/watcher/branches [2] https://opendev.org/openstack/watcher/commits/branch/unmaintained/2023.1

2 2

[kolla-ansible] nova ceph backend
by Jonathan Proulx 06 Nov '25

06 Nov '25

Hi All, looking at https://docs.openstack.org/kolla-ansible/2025.1/reference/storage/external-… To use ceph with my particular setup it seems I need to set this in globals.yml: nova_backend_ceph: "yes" ceph_nova_user: "nova" ceph_nova_pool_name: "ov-vms" And provide the keyring file in config/nova/ceph.client.nova.keyring I'm not configuring cells explicitly but when I run `kolla-ansible reconfigure --tags nova` I get: ASK [nova-cell : Check cinder keyring file] ************************************************************************** fatal: [kvmgen6-0.csail.mit.edu -> localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable.. ['{{ node_custom_config }}/nova/{{ inventory_hostname }}/{{ keyring }}', '{{ node_custom_config }}/nova/{{ keyring }}']: {{ nova_cell_ceph_backend['cluster'] }}.client.{{ nova_cell_ceph_backend['volumes']['user'] }}.keyring: 'dict object' has no attribute 'volumes'\n\nThe error appears to be in '/opt/kolla/venv/share/kolla-ansible/ansible/roles/nova-cell/tasks/external_ceph.yml': line 17, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Check cinder keyring file\n ^ here\n"} Digging through the code in roles/nova-cell/tasks/external_ceph.yml I kind of make it go by adding: nova_cell_ceph_backend: cluster: "ceph" vms: user: "{{ ceph_nova_user }}" pool: "{{ ceph_nova_pool_name }}" volumes: user: "{{ ceph_nova_user }}" pool: "{{ ceph_nova_pool_name }}" but I can find no reference in the docs to this and don't understand what in needs "volumes" for (And suspect it should be using some cinder stuff for that?) What have I done here? The nova backend "works" but I worry about how I got here since nova+cinder is very common and I shoudl not have needed to leave the documented path so I worry there's a deeper error in my config that caused this. Thanks, -Jon -- Jonathan Proulx (he/him) Sr. Technical Architect The Infrastructure Group MIT CSAIL

2 2

[nova] NVLink passthrough
by Pavlo Shchelokovskyy 05 Nov '25

05 Nov '25

Hi all, This is a continuing thread from 3 years ago here: https://lists.openstack.org/pipermail/openstack-discuss/2022-June/029273.ht… We've recently faced the same issue re Nvidia NVLink / NVSwitch, and I wanted to share how we are solving it. Given: - server with 8x Nvidia H200 pGPUs aimed to be used as PCI passthrough - separate pGPUs can be connected with NVLink but not any with any - there are 4 specific combinations to use when pairing 2 gpus - gpu1+gpu3, gpu2+gpu4, gpu5+gpu7, gpu6+gpu8 - and there are only 2 possible combinations of pairing 4 gpus - gpu{1..4} and gpu{5..8} The requirement is to have a single flavor that would ask for 2 GPUs, and the scheduling would pick only appropriate pair where NVLink between GPUs is possible, and compute will enable/disable NVLink for those GPUs as appropriate (same for 4 GPUs flavor). The compute part is solved by a libvirt hook provided by Nvidia here https://github.com/NVIDIA/libvirt-hooks/. Given the correct devices are chosen by Nova to be passed into the VM, the hook will enable NVLink between them. However, that scheduling part is currently rather impossible with Nova as-is. The best one can do is assign separate trait to appropriate pairs and quartets of GPUs, and then have 4 flavors for 2-GPU configuration and 2 flavors for the 4-GPU one. This if course is very inefficient and brittle, and can not work in a self-service cloud. What we are trying to do is the following: - add a new flavor extra spec `placement:same_subtree`. The value will be passed directly to placement's allocation candidates API. - in order for this to be usable in a flavor, we need predictable resource group names generated for PCI devices. Right now they are auto-generated from request-id, which won't work. - hence we extended `pci_passthrough:alias` spec to have an optional `:group_prefix` part. Then, what we do is reshape the PCI resource provider tree a bit. We introduce two extra layers of providers, let's call them `group4` and `group2`, each group has a specific trait, for example `CUSTOM_PCI_GROUP_OF_2` and `CUSTOM_PCI_GROUP_OF_4`. Two `group4` providers have compute nodes as a parent, and are parent to two `group2` providers each. Then we re-parent PCI resource providers to `group2` providers, so that each `group2` and `group4` tree corresponds to a set of GPUs that can be connected via NVLink. Luckily enough, resource tracker of nova-compute does not care about the parent provider, as long as the name and ID of PCI resource providers are not changed, so the reshape has no effect on the resource tracker. We also need to use all this with Aggregate Extra Spec filter active, so we've duplicated the `group_policy` flavor extra spec to `placement:group_policy` one. And of course, for all of this to work, one has to enable "PCI in placement" feature too https://docs.openstack.org/nova/2025.2/admin/pci-passthrough.html#pci-track… As a result, assuming the PCI alias for the GPUs is `h200` and its resource class is `PGPU`, we can create the following flavor extra spec: pci_passthrough:alias=h200:2:_nvlink2 trait_groupof2:CUSTOM_PCI_GROUP_OF_2=required placement:group_policy=none placement:same_subtree=_nvlink2-0,_nvlink2-1,_groupof2 that will result in the following placement query: GET /allocation_candidates ?resources_nvlink2-0=PGPU:1 &resources_nvlink2-1=PGPU:1 &required_groupof2=CUSTOM_PCI_GROUP_OF_2 &group_policy=none &same_subtree=_nvlink2-0,_nvlink2-1,_groupof2 That will select only a pair of GPUs that are children of a single provider with `CUSTOM_PCI_GROUP_OF_2` trait - which is exactly what we need! You can find this work proposed as a patch at https://review.opendev.org/966205. This is definitely just a beginning of discussion, and most certainly would require a spec, so I am eager to hear your thoughts and comments :-) Cheers, pas-ha -- Dr. Pavlo Shchelokovskyy Principal Software Engineer Mirantis Inc www.mirantis.com

2 1

[horizon][PTG] 2026.1 Gazpacho PTG summary
by Tatiana Ovchinnikova 05 Nov '25

05 Nov '25

Hello everyone, A huge thank you to everyone who joined the OpenStack Horizon team for the Gazpacho PTG, who organized sessions, presented topics, and actively contributed to the discussions! Thank you for making the PTG a success! The full list of sessions and topics can be found on our Etherpad [1] Here is the summary of the topics: # Flamingo Retrospective A review of the previous Flamingo cycle, highlighting positive developments like switching to SDK for certain services, improving TOTP setup, and improving region/domain selection. # Gazpacho Cycle Planning Planning for the current Gazpacho cycle, establishing AngularJS removal as the top priority. Other key goals include investigating the removal of old integration tests and planning for Django 5.2 support. # Open Discussion A session for discussing various in-progress code reviews. Topics included a discussion on a feature to add changing instance password support. # Django 5.2 Support Discussion on the plan to adopt Django 5.2, the latest LTS release. # Tests Review of current test stability and plans for test code cleanup. The team noted that both horizon-integration-pytest and horizon-ui-pytest are currently stable and plans to remove old integration tests from the repository are moving forward. We also discussed progress on implementing Manila UI integration tests as a template for other plugin teams. # Towards New OpenStack contributors: An OpenInfra User Group Kenya Story A presentation by Agnes Nduta to share the experiences and strategies of the OpenInfra User Group Kenya in successfully onboarding new contributors into the OpenStack ecosystem. # Contributor/Maintainer Surveys and Metrics Analysis A review of current project health using metrics and analysis from contributor and maintainer surveys. # How to create a Cursor MCP for analyzing Horizon code A walk-through demonstrating the steps required to set up a Cursor Maintainer Candidate Program (MCP) for analyzing Horizon code. # Multi-Realm Support and Multi-Region Support discussion A dedicated cross project session to discuss the implementation and implications of providing Multi-Realm and Multi-Region support within the Horizon dashboard. More details can be found on the separate Etherpad [2]. [1] https://etherpad.opendev.org/p/horizon-gazpacho-ptg [2] https://etherpad.opendev.org/p/multi-realm With kind regards, Tatiana

1 0

Nova error for fetch or server create after upgrade
by Ahmad Milad Pour 05 Nov '25

05 Nov '25

Hello there, I did upgrade the packages and repo, and something were upgraded: BEFORE: root@gregion83:~# apt-cache policy nova-api | head -n 10 nova-api: Installed: 3:29.2.0-0ubuntu1.1 AFTER: root@gregion83:~# apt-cache policy nova-api | head nova-api: Installed: 3:31.0.0-0ubuntu1.2~cloud0 I also did add "DEBUG=True" under "default" section of /etc/nova/nova.conf in my region and tried to create server, but I cannot find any important logs or details to see what's wrong: ==> /var/log/nova/nova-api.log <== 2025-11-05 14:42:44.524 2960 INFO nova.osapi_compute.wsgi.server [-] 185.137.25.83 "GET /v2.1 HTTP/1.1" status: 200 len: 790 time: 0.0164449 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi [None req-892920aa-c4be-4aea-abab-aa8e9fe8c7e0 64c74c80ec9a4a31ae27b82af846b36a 906db1c682bc4e3db 822c2e6dbc9cb6a - - default default] Unexpected exception in API method: neutronclient.common.exceptions.InternalServerError: Request Failed: internal serve r error while processing your request. Neutron server returns request_ids: ['req-80f3ccda-11ee-4417-a27c-2437540e6ee0'] 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.InternalServerError: Request Failed: internal server error while processing your request. 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-80f3ccda-11ee-4417-a27c-2437540e6ee0'] 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi 2025-11-05 14:42:45.628 2960 INFO nova.api.openstack.wsgi [None req-892920aa-c4be-4aea-abab-aa8e9fe8c7e0 64c74c80ec9a4a31ae27b82af846b36a 906db1c682bc4e3db822c2e6dbc9cb6a - - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'neutronclient.common.exceptions.InternalServerError'> 2025-11-05 14:42:45.632 2960 INFO nova.osapi_compute.wsgi.server [None req-892920aa-c4be-4aea-abab-aa8e9fe8c7e0 64c74c80ec9a4a31ae27b82af846b36a 906db1c682bc4e3db822c2e6dbc9cb6a - - default default] 185.137.25.83 "GET /v2.1/servers/c18f669d-1338-42dd-947f-c40217238450 HTTP/1.1" status: 500 len: 660 time: 1.1018338 My last openstack version was 6.6.0, and I did upgrade after another email I sent "How to install Flamingo openstack in Ubuntu 24". Before upgrade, everything was working fine, but now I have issue. Would you please help me for this? Regards, Ahmad

2 3

How to install Flamingo openstack in Ubuntu 24
by Ahmad Milad Pour 05 Nov '25

05 Nov '25

Hello there, I use Ubuntu 24 and I did try to install openstack, but it seems it's not the latest one. root@gregion83:~# cat /etc/os-release PRETTY_NAME="Ubuntu 24.04.3 LTS" NAME="Ubuntu" VERSION_ID="24.04" VERSION="24.04.3 LTS (Noble Numbat)" VERSION_CODENAME=noble ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL=" https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=noble LOGO=ubuntu-logo root@gregion83:~# openstack --version openstack 6.6.0 root@gregion83:~# apt-cache policy nova-api | head -n 10 nova-api: Installed: 3:29.2.0-0ubuntu1.1 Candidate: 3:29.2.0-0ubuntu1.2 Version table: 3:29.2.0-0ubuntu1.2 500 500 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages *** 3:29.2.0-0ubuntu1.1 100 100 /var/lib/dpkg/status 3:29.0.1-0ubuntu1.4 500 500 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages root@gregion83:~# apt-cache policy openstack-dashboard | head -n 10 openstack-dashboard: Installed: 4:24.0.1-0ubuntu0.24.04.1 Candidate: 4:24.0.1-0ubuntu0.24.04.1 Version table: *** 4:24.0.1-0ubuntu0.24.04.1 500 500 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages 100 /var/lib/dpkg/status 4:24.0.0-0ubuntu1 500 500 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages root@gregion83:~# curl -s -H "X-Auth-Token: $(openstack token issue -c id -f value)" http://$HOSTNAME:8774 | jq { "versions": [ { "id": "v2.0", "status": "SUPPORTED", "version": "", "min_version": "", "updated": "2011-01-21T11:33:21Z", "links": [ { "rel": "self", "href": "http://gregion83.domain.com:8774/v2/" } ] }, { "id": "v2.1", "status": "CURRENT", "version": "2.96", "min_version": "2.1", "updated": "2013-07-23T11:33:21Z", "links": [ { "rel": "self", "href": "http://gregion83.com:8774/v2.1/" } ] } ] } That's the information I thought is good for the help. And that's the error I have: root@gregion83:~# openstack server rebuild --image 608349cf-8743-4339-8e2c-e707f89be64f --no-user-data c18f669d-1338-42dd-947f-c40217238450 --os-compute-api-version 2.54 or greater is required to support the --no-user-data option root@gregion83:~# openstack server rebuild --image 608349cf-8743-4339-8e2c-e707f89be64f --user-data /dev/null c18f669d-1338-42dd-947f-c40217238450 --os-compute-api-version 2.54 or greater is required to support the --user-data option I see this link from which I can install: https://docs.openstack.org/horizon/2025.2/install/install-ubuntu.html How to resolve this issue? Regards, Ahmad

3 4

[kolla-ansible] Upgrade from 2024.1 to 2024.1 neutron-l3-agent unhealthy
by hamid.lotfi＠gmail.com 04 Nov '25

04 Nov '25

Hi everyone, I’m hitting a weird issue after a kolla-ansible upgrade and would appreciate any pointers. Environment kolla-ansible 18.8.1 rabbitmq 4.1.3 (deployed by Kolla) upgrade path: 2024.1 → 2025.1 What happened Ran kolla-ansible -i … upgrade from 2024.1 to 2025.1. The upgrade finished without obvious errors in kolla logs. Right after the upgrade every neutron-l3-agent container on all control/compute nodes is marked unhealthy (no ERROR/FATAL lines in the container logs). All tenant instances lose external connectivity. Simply docker restart neutron_l3_agent on each node is enough: containers become healthy and external access is restored instantly. No other OpenStack service shows the same behaviour. Any idea why the agents start in “unhealthy” state until manually restarted? Happy to collect more data if you tell me what to look for.

1 0

[ops][largescale-sig][scientific-sig] Ops Radio Hour on October 31, 2025 at the PTG
by Ildiko Vancsa 04 Nov '25

04 Nov '25

Hi OpenStack community, I'm reaching out to you with a friendly reminder that the next OpenStack Ops Radio Hour[1] is scheduled to happen on __October 31st, 2025 at 1300UTC,__ which is during the PTG! The call series is a casual setting for operators, engineers, and contributors to come together, share knowledge, and collaborate on the complexities of managing and deploying OpenStack environments. The main topic of the next gathering will be to discuss how to refactor Ops-related activities. See Thierry's email for more details: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… If you have further topics you would like to dive into during the call, please add them to this etherpad: https://etherpad.opendev.org/p/openstack_ops_radiohour A few reminders: - The group on the call agreed to try out Matrix as a chat platform for more synchronous communication outside calls + You can access the Matrix room here: https://matrix.to/#/#openstack-ops:opendev.org<https://matrix.to/#/%23openstack-ops:opendev.org> + Further details are here: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… - If you would like to read and catch up on notes from previous calls, you can find them in this etherpad: https://etherpad.opendev.org/p/openstack_ops_radiohour See you on the call on October 31st! Bets Regards, Ildikó [1] https://wiki.openstack.org/wiki/Operations/Meetups

1 1

[horizon] Cancelling weekly meeting tomorrow
by Tatiana Ovchinnikova 04 Nov '25

04 Nov '25

Hello team, I don't have anything to share after the PTG, so I'm cancelling the IRC meeting tomorrow, November 5th. Let's save some time for the reviews and see you next week! With kind regards, Tatiana

1 0

[ironic][all] Making core teams more accessible
by Jay Faulkner 04 Nov '25

04 Nov '25

I'm going to preface this with a comment that I'm not completely sold on this idea yet, but I think it's worth a larger discussion and might be a seed to help with discussion. I've tagged Ironic as this proposal is for Ironic specifically, but I'm very, very interested in the larger community opinion for this kind of approach so I also tagged all. One of the toughest parts about walking the path to becoming core is how fuzzy the targets are. With Ironic recently having added a two-tier review system, with reviewers and approvers separate, we have an opportunity to allow a much, much bigger tent for the ironic-reviewers group without risking code getting merged without approval by one of our folks with more experience and context in OpenStack and Ironic as a whole. So here's the suggestion -- we come up with a static criteria and period with a clear bar to "if you do this, you are an ironic-reviewer". For instance, we could say (I'm not sure about these numbers): if you do an average of Y (5?) reviews/week and have merged a major Ironic feature or multiple bugfix patches, you would automatically be a reviewer. Obviously, we'd have to keep safeguards to prevent abuse; in our community in the past we had a rule that changes required review from two separate corporate entities -- in this case, I'd suggest a waiting period to prevent quick-merging by someone with a single perspective. Exceptions would be, as in the current system, for emergency CI fixes and documentation improvements. To be clear: the safeguards shouldn't be seen as a lack of trust, but maybe instead as an acknowledgement that structural controls hold value even if there's no reason to assume mistrust. Full proposal: * ironic-approvers (+2 and Workflow +1) continues to operate as usual, with admission being based on technical merit, trust, and review frequency. * ironic-reviewers (+2 only) becomes a data-driven group, where reviewers are added/removed based on individual performance metrics TBD (likely reviews/commits/bugs fixed). -- I'll note that today, there's not really any set of reasonable metrics that would add additional reviewers to this group. * Code changes would either need a review from two separate company-representatives, or to have sat for a short waiting period to ensure other folks had a chance to look at it. * Urgent CI fixes, trivial changes, and documentation updates would still only need one review to land. What do folks think? Would this be a good carrot for existing non-cores to up their review and upstream contribution commitments? - Jay Faulkner G-Research OSS

6 6

[nova] Patch for Calico with libvirt >= 9.5.0
by Nell Jerram 04 Nov '25

04 Nov '25

I apologise if this is the wrong place to ask, but is there something I can do to encourage review and approval for https://review.opendev.org/c/openstack/nova/+/960284 ? Many thanks - Nell

2 2

[manila] Cancelling Nov 6th weekly meeting
by Carlos Silva 04 Nov '25

04 Nov '25

Hello Zorillas and interested stackers, As discussed in the previous upstream weekly meeting, we're cancelling this week's meeting, so we have some extra time to go through PTG action items before we regroup. The next meeting will be on Nov 13th. Thank you, carloss

1 0

Kolla-Ansible deployment results in broken dashboard
by Ricardo Matheus 04 Nov '25

04 Nov '25

Hi all I just deployed a multinode OpenStack using Kolla-Ansible, but it seems that the Horizon image (ubuntu / master) is somewhat broken. Horizon runs and the dashboard is accessible, but every file in the "static" part of the web server is not available (no logo images, no css, nothing... just plain unformatted html). Is this a known bug? (btw, there was a guy asking about the same problem in irc a few days ago, it is in the logs for 2025-10-23 (Thursday) <https://meetings.opendev.org/irclogs/%23openstack/%23openstack.2025-10-23.l…> ) Cheers! RDM

2 2

[ptg] #eventlet-removal 2026.1 Gazpacho PTG summary
by Herve Beraud 04 Nov '25

04 Nov '25

Hi everyone, Following our last week PTG meeting, I’d like to share a brief summary of the key outcomes, achievements, and next steps we agreed upon for the Eventlet removal initiative. ## Flamingo Retrospective We have crossed an important milestone: over 50% of impacted projects have shown significant migration progress. 19 deliverables are fully migrated, while 32 others are currently undergoing migration work. Here are the main accomplishments delivered during Flamingo: - Eventlet now includes support for Python 3.14, ensuring compatibility with upcoming Python releases; - Oslo.service introduced a new threading backend to provide a fully threading-based execution model; - Ironic, Heat, Barbican, Mistral, and several other projects are now fully migrated; - the Neutron migration is just one step away from completion; - Nova-api, nova-metadata-api, nova-conductor nova-scheduler can now run in threading mode. ## Gazpacho Call to Actions The following actions are expected to structure our efforts for the Gazpacho development cycle: - we must enhance oslo.service to prevent fork-inherited lock deadlocks by defaulting to the spawn multiprocessing start method, which will address known Manila and Cinder bugs; WIP https://review.opendev.org/c/openstack/oslo.service/+/963742 - we are preparing to merge the new opt-in H905 hacking rule to ensure consistent Eventlet usage detection across all OpenStack projects (see: https://review.opendev.org/c/openstack/hacking/+/959699) - we should clean up the bug tracker of migrated projects to reflect the current status of the eventlet removal efforts. Many of those projects surely have historical eventlet bugs that could be closed; - we need to maintain momentum on ongoing migrations and accelerate progress on the most delayed projects; - we should prioritize the “low-hanging-fruit” migrations, such as Cyborg and Magnum. ## Project Needing Attention Several projects have not shown any notable update during this cycle and require focus in the next series. These projects still contain Eventlet occurrences that must be addressed urgently: Blazar, Venus, Tacker, Masakari, and Zun. The TC has marked Venus and Vitrage as inactive, and these teams may be retired in 2026.2 if activity does not resume. ## Open Discussion Nova plans to focus on nova-compute, nova-novncproxy, testing improvements, and Rally-based benchmarking to fine-tune thread pool defaults, making Gazpacho the likely cycle where Nova fully switches from Eventlet to threading. Dan Smith presented a strategy idea for Glance that we could name the “Optional Eventlet Shim Strategy.” The goal of this approach is to make the use of Eventlet fully optional, allowing Glance to run without this dependency by default. To achieve this, all Eventlet-related code would be isolated outside of Glance’s core and moved into a separate module acting as a bridge. The core of Glance would remain unique and free of any references to Eventlet. If users still wish to rely on Eventlet, they would be able to install an additional module that provides the required implementation. Without this module, Glance would naturally run using the native threads concurrency mode. This approach offers a smooth transition path while preserving the ability to revert if needed, ultimately moving Glance toward a fully Eventlet-free architecture. Thank you to everyone who contributed during the PTG discussions and for the continuous efforts across teams. Let’s keep the momentum going as we enter the Gazpacho cycle and move closer to a fully Eventlet-free OpenStack. As always, you’re welcome to join us on IRC at #openstack-eventlet-removal, and you can find guidance, resources, and updates at https://removal.eventlet.org/ Thanks for your attention. -- Hervé Beraud Principal Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[watcher] 2026.1 Gazpacho PTG summary
by Douglas Viroel 04 Nov '25

04 Nov '25

Hi all, Last week's PTG had a lot of great discussions, covering many different aspects of the Watcher project. If you want to get mode details about a topic, you can find the PTG etherpad with all notes here https://etherpad.opendev.org/p/watcher-2026.1-ptg Here is a summary of the discussions that we had: Future of datasources backends and untested integrations *Discussion*: The discussion revolved around the removal timeline for Monasca, the status of Gnocchi, and the deprecation of Prometheus datasource in favor of Aetos. There was also a proposal to deprecate MAAS support due to lack of testing and documentation, and to re-evaluate experimental integrations in 2026.2. *Agreed*: - AI(jgilaber): Monasca datasource will be removed this cycle. - More research is needed on current Gnocchi usage, and there will be no change in its support this cycle. Suggestion: include a question about gnocchi in the next openstack user survey. - AI(jgilaber): Prometheus will be deprecated in 2026.1 in favor of Aetos. Suggestion: include documentation on how to upgrade between datasources. - AI(dviroel): MAAS will be marked as deprecated this cycle, since it has Eventlet dependent code that is planned to be removed in a near future. We will send another email to the ML to call for maintainers. - AI(sean-k-mooney): Update Watcher documentation about service integrations that are now deprecated. Openstack SDK *Discussion*: It was focused on integrating Watcher with the OpenStack SDK to deprecate and remove the python-bindings in other places. The goal for watcher is to replace the current usage of the project's client with the SDK instead (if there is support for it). The goal for python-watcherclient is to only provide the *openstackclient* plugin and replace python binding by adopting the usage of the SDK (when available). And for the watcher-dashboard is to use the SDK exclusively for API interactions (also when available). *Agreed*: - Start with watcher integrations. The goal is to move one service client (e.g.: Nova) to the SDK this cycle. - Do not freeze python bindings until SDK support is in place. - A single spec can address the overall plan. Code modernization, dependencies and dead code removal *Discussion*: The discussion covered code modernization using pyupdate and potentially ruff for cleanup. It also addressed the removal of dead code, such as API routes that always raised exceptions and commented-out code in production files. The removal of various client dependencies (Neutron, Glance, Monasca) was also proposed. *Agreed*: - Apply the same pre-commit and ruff linting checks to the tempest plugin and watcher client. - Defer decisions on more typing until necessary, possibly starting with interfaces. - Explore dropping the number of dependencies, such as multiple timezone libraries. Applier's Workflow Execution and Its Interface/Contract *Discussion*: The workflow for the Applier was identified as poorly documented, leading to questions during reviews of new actions. The need for a default Action interface was discussed. *Agreed*: - The assessment of a default Action interface will be based on the chosen path for rollback and aborting topics. - AI(dviroel): Current interface needs to be documented regardless. Applier: Aborting running tasks *Discussion*: The current implementation of the Applier spawning a new green thread for each action, and killing threads for actions that support abort(), was discussed. *Agreed*: - Stop spawning/killing threads on every action. This code can be refactored right after we merge the eventlet changes in the applier (so we don't mix the proposals). - Improve the execute() method in actions to check resource status and abort the process/looping when an action is cancelled/aborted. Applier: Rollback of Action Plans *Discussion*: The current lack of a working rollback mechanism and the fact that the revert() method from Actions is not being tested or called were highlighted. The future of the rollback option was debated. New rollback mechanisms, such as a user-triggered "rollback" action for failed action plans, were also considered. *Agreed*: - Auto-revert does not work and should be removed in the future. - AI(dviroel): Current behavior should be treated as a bug, and the documentation and associated configuration options should be updated accordingly. - A new spec should be proposed for a new action plan to revert workflow. CI Testing and Coverage *Discussion*: The naming and refactoring of watcher CI jobs were discussed. The need for every voting job in check to also run in the gate was emphasized, as well as in stable branches and other watcher projects. Job renames and consolidation were proposed, along with enabling tempest scenario jobs for stable branches and creating a new grenade job for upgrade testing. *Agreed*: - AI(dviroel): Job renames/consolidation: watcher-functional can be merged into other tempest jobs, watcher-tempest-actuator can be merged into strategies job, and ipv6 job should be integrated into other existing tempest jobs. Gate updates will occur after renaming and merging jobs. Backporting changes from master to stable branches is expected. - AI(dviroel): In watcher-tempest-plugin: replace tempest-functional with a tempest job that runs scenario tests for stable branch validation. - AI(chandankumar): A new grenade job to test upgrades between slurp releases and include more testing in existing jobs. - AI(sean-k-mooney): Propose a watcher job to run against OpenStack requirements project. Improving testing coverage for strategies by doing functional testing *Discussion*: A specification proposal and a detailed implementation plan for improving testing coverage for strategies through functional testing were presented. *Agreed*: - A phased approach will be taken: - AI(amoralej): 1st phase: API only GETs/POSTs. - 2nd phase: Adding decision-engine + Nova + Prometheus datastore. - 3rd phase: Adding Applier. Rally Testing in watcher *Discussion*: The current status of Rally testing in Watcher was reviewed, noting that the rally-task-watcher job runs but is not in the Watcher repository. Missing functionalities were identified, such as the inability to pass audit template scope and parameters, lack of auto-triggering, and no support for event, continuous audit, action plans, and actions. *Agreed*: - AI(chandankumar): Move rally-openstack watcher plugin code to watcher repo - AI(chandankumar): Add periodic job to run rally jobs - Revisit this topic in future as we progress on scaling watcher CI Watcher-dashboard improvements *Discussion*: Improvements to the Watcher dashboard were discussed, including adding auto-page refresh for audit/action plan status, a start button for action plan details, and options for bulk archiving. Dashboard testing using the Django test framework and Playwright was also covered. *Agreed*: - Create wishlist bugs or blueprints to track dashboard improvements. - AI(chandankumar): Check with TC regarding the usage of pytest to improve wording and proceed with integration tests implementation - AI(reviewers): Get the spec https://review.opendev.org/c/openstack/watcher-specs/+/963438 merged. - A spec will be required for the audit and action plan bulk archive feature. The future of datamodel list API *Discussion*: The utility of the datamodel list API, beyond tempest tests, was questioned. The possibility of freezing the API with existing content to avoid microversion bumps for new instance/node updates was raised. The addition of new storage models (storage and baremetal) was also discussed. *Agreed*: - Do NOT extend the API to support additional models. - Defer the removal of datamodel list (compute model) to future discussions, but for now, do not extend it further, even if new fields are added to model elements. - AI(dviroel): Add test to avoid new API changes in datamodel list. Eventlet Removal *Discussion*: Changes made in the Flamingo cycle regarding Eventlet removal were reviewed.The need for a collector sync timeout for threading mode and refactoring the action plan cancel workflow were discussed. *Agreed*: - AI(dviroel): Collector timeout topic: REST API calls should continue to have their own timeouts, and an event trigger can assist in stopping the overall sync process. Add a new config option for collector timeout should be considered. - Applier: stop killing threads when an action plan is canceled. We can keep the current behavior for evenlet but it will be a noop for threading mode (minimal impact for eventlet removal changes). - AI(dviroel): Set MAAS to deprecate after PTG. We will send an email to the ML to call for maintainers once again. (same AI in Future of datasources topic) Future of Noisy Neighbort Goal/Strategy *Discussion*: The deprecation of cache monitoring metrics in the kernel and Nova, which formed the basis of the current noisy neighbor strategy, was discussed. The need to replace this strategy and identify new metrics for contention (e.g., CPU steal, CPU pressure, IOWait) and noisy neighbors (e.g., CPU usage from low-priority instances) was explored. Instance priority mechanisms were also considered. *Agreed*: - Remove the current noisy neighbor strategy in 2026.2+ to allow deprecation to ship in a Slurp release. - AI(dviroel): a proof of concept for CPU steal/IOWait/other metrics is a nice to have, to replace current LLC monitoring metrics. - Use instance metadata for PoC, but consider different solutions for identifying/classifying workloads priorities, like tiering based on flavor extra spec information. Scaling Watcher *Discussion*: The limitations of running a single instance of the Decision Engine and Applier were discussed, with ideas for horizontal scalability. Scalability concerns were raised for different watcher resources: audits, actions and action plans. Some of the open issues that were discussed: - Centralized datamodel: it is independently managed by each decision-engine. Move the model from in-memory to memcached or the database was discussed. - ONESHOT audits are not a failover in a multiple-decision engine deployment. - When an applier dies, the action plan remains ONGOING and is only cancelled upon the applier restart. The main issue was identified as continuous audits being associated with a specific decision-engine. The concept of moving to an event-driven model with stateless decision engines and appliers was proposed, where data models would reside in the database or a shared data store. *Agreed*: - Fow now, instrument the decision-engine to measure the size of the model, and time taken to process notifications. - Keep current failover behavior for ONESHOT audits. - We could have a service monitor in the applier to reschedule PENDING action plans and cancel ONGOING ones, providing the proper status message. - There should be a way limit and set concurrency of actions in audits. In this cycle we will propose a mechanism setting them at system or audit level that covers all the strategies. Stacking Strategies *Discussion*: The possibility of having stacking strategies was brainstormed in this topic. This included ideas like sequential execution of multiple strategies, where a mutable cluster data model would be shared among them. This could result in a list of linked action plans, or result in merging all actions into a single action plan to avoid unnecessary steps. This discussion may be revisited soon, with a more detailed use case. *Agreed*: - Get back to this topic when we have more detailed use cases. Propose a spec to highlight the need of this feature in watcher. Please let me know if I missed something. Thanks! Assisted-By: Gemini -- Douglas Viroel - dviroel

1 0

[kolla][OpenStackAnsible] 2025.2 Flamingo release for cycle-trailing projects
by Thierry Carrez 04 Nov '25

04 Nov '25

Hi teams, This is a gentle reminder that the following projects with cycle-trailing release model haven't done their final 2025.2 Flamingo release yet: [kolla] ansible-collection-kolla kayobe kolla-ansible kolla [OpenStackAnsible] openstack-ansible-roles openstack-ansible Note that next week we'll ask for cutting stable/2025.2 branches, as kind of a Feature Freeze, 3 weeks prior the 2025.2 Flamingo Cycle-Trailing Release Deadline on December 4th, 2025 [1] (exactly one month away). [1] https://releases.openstack.org/gazpacho/schedule.html#g-cycle-trail Thanks in advance, -- Thierry Carrez (ttx)

1 0

[nova][ptg] 2026.1 Gazpacho PTG summary
by Rene Ribaud 04 Nov '25

04 Nov '25

Hello everyone, Last week was the PTG, thank you to all who joined and contributed to the discussions! I hope you enjoyed it and found it productive. Again, this cycle we had a packed agenda with active participation throughout the sessions. For the Nova sessions (Tuesday to Friday), we had an average of 15 to 20 participants per session. Attendance was slightly higher during the cross-project discussions, with an average of 25 to 30 people joining. You can revisit all notes and discussions on the Etherpad here: https://etherpad.opendev.org/p/r.9bac75a686ab0941572757b205c9415c Below is a summary of the topics and outcomes from our PTG discussions. I’ve tried to keep it as concise as possible, though we covered quite a lot during the week. **** Tuesday topics **** #### 2025.2 Flamingo Retrospective #### During the Flamingo cycle, 14 blueprints were accepted and 9 were fully implemented, resulting in a 64% completion rate. This is slightly lower than the 71% achieved during the previous Epoxy cycle. In addition, at least 26 bug fixes were merged during the cycle. Review & Process Feedback Eventlet Transition Plan, the migration continues to work as expected and remains well-coordinated. Release process was smooth with no major issues reported. Recognition to Kamil for his work on the conductor, and to Sean, Dan, and others for their consistent review efforts. ✅ Continue to prioritize the Eventlet removal as a key goal for the G cycle. Branch Opening Policy, some uncertainty remains about what can be merged between RC1 and GA, when master is technically open but still sensitive for late merges. Concerns were raised about backports and release stability. ✅ Allow merges on a case-by-case basis once the new master branch opens. ✅ Bauzas to propose a patch updating Nova contributor process <https://docs.openstack.org/nova/latest/contributor/process.html> <https://docs.openstack.org/nova/latest/contributor/process.html> Bug Triage, the number of untriaged bugs remains high (around 180). Efforts were noted, but time allocation and rotation remain challenging. The end-of-meeting triage format was found ineffective for real-time work, though still useful for discussing specific cases. ✅ Consider reviving a rotating triage roster if enough volunteers join. ✅ Keep triage discussions focused during the weekly meeting rather than doing live triage. Review Load, contributors noted increasing difficulty in finding time for reviews. Maintaining review velocity remains a cross-cycle challenge. ✅ Encourage feature-specific syncs or review sprints for high-priority areas. Feature Momentum, weekly syncs and structured updates help large features maintain progress. The VMware CI topic was cited as useful, though too many topics per meeting could cause overload. ✅ Keep dedicated weekly or bi-weekly syncs for major features (e.g. Eventlet removal). ✅ Limit meeting updates to 2-3 features per session. Review Tracking, the previous Etherpad tracker was not effective. However, having a central list remains important to avoid missing patches and to monitor review health. ✅ Discuss an external or simplified review tracker during the next meeting with more core reviewers. ✅ Keep the tracker short and up to date to maintain visibility. #### 2025.2 Flamingo Planning #### Hard spec freeze: December 4th Milestone 2 (M2): January 8th Feature Freeze (FF): February 26th Release: End of March / early April. The team discussed adjusting the timing of the spec freeze. A soft spec freeze was proposed for November 20th, giving three weeks after the PTG for contributors to submit specs. Two spec review days will be scheduled between the soft and hard freezes. Additionally, two implementation review days are planned: one around M2 for early landings, and another roughly two weeks before M3. ✅ Adopt the proposed plan, while staying flexible on a case-by-case basis. ✅ Soft spec freeze on Nov 20th, followed by two spec review days before Dec 4th. ✅ Two implementation review days: one near M2 and one two weeks before M3. ✅ A +2 on a spec implies reviewers commit to reviewing the corresponding implementation. #### Upstream meeting schedule #### ✅ Move the weekly upstream meeting to Monday at 16:00 UTC. ✅ Keep the slot fixed (no alternating or skipping weeks). ✅ Uggla to update the meeting schedule and cycle accordingly. #### Upstream bug triage follow up #### The team discussed how to better organize bug triage follow-up for the Gazpacho cycle. A proposal was made to create a small group of volunteers who would receive a few bugs to triage each week. Each volunteer would handle up to three bugs, distributed by Uggla. During the weekly upstream meeting, the team will check whether a dedicated Q&A session is needed to discuss specific bugs. If required, such a session will be scheduled every two weeks, ideally right after the upstream meeting. Otherwise, discussions will take place within the regular meeting. ✅ The team agreed to give this approach a try in the next cycle. The overall sentiment seemed favorable, though details will likely need adjustment as we go. #### Eventlet removal #### Significant progress was made since Flamingo. In Flamingo, n-api, n-metadata, and n-sch could already be switched to native threading via an environment variable, and the nova-next job runs those services without Eventlet. n-cond was nearly ready but missed feature freeze due to a late futurist bug, which has since been fixed and released. A py312-threading tox target and a Zuul job already validate a large part of the unit tests without Eventlet. In early Gazpacho, native threading for the conductor has successfully landed. Planned tasks for Gazpacho - Add native threading support for n-novncproxy and other proxies. - Add native threading support for n-compute. - Run all unit and functional tests in both threading and Eventlet modes, keeping small exclude lists where needed. - Conduct scale testing using the fake driver, possibly extended to the libvirt driver. - Switch n-api, n-metadata, and n-sch to native threading by default, keeping Eventlet as fallback. - Introduce a nova-eventlet (legacy) CI job to ensure backward compatibility. - Keep nova-next as the only job testing conductor, novncproxy, and compute with threading until the H cycle. - Fix bugs reported after Eventlet removal. - After Eventlet removal, migrate from PyMySQL to libmysqlclient for better DB performance. ✅ Kamil volunteered to handle the novncproxy migration. ✅ Start with nova-compute migration first to build confidence. ✅ Switch the default to threading as early as possible in Gazpacho. ✅ Add a dedicated Eventlet CI job to continue testing legacy mode. ✅ Use the hybrid plug job to test Eventlet mode even for services switched to threading by default. ✅ Keep the work independent, but document the known issue with long-running RPC calls. #### Long-Running Sync REST API Calls #### The team discussed how long-running synchronous REST API actions (like volume or interface attach) can block nova-api threads once Eventlet is removed. With native threading, the limited number of threads makes blocking calls more costly, unlike Eventlet which could spawn many lightweight greenlets. This work is closely related but independent from the Eventlet removal effort. Dan started a spec on the topic, and it may be split into two (volume and interface attach). Detach operations are already async, and Manila shares follow a similar model. ✅ Introduce a new microversion to keep backward compatibility. ✅ Convert volume and interface attach to async operations under that microversion. ✅ Coordinate with Rajesh’s "show finish_time" ( https://review.opendev.org/q/topic:%22bp/show-instance-action-finish-time%22 ) work to avoid race conditions. ✅ Dan and Johannes to update the spec. ✅ Further investigation is needed to determine if WSGI or API tuning could help mitigate blocking. #### vTPM Live Migration #### The team reviewed how to handle TPM secret security policies during instance operations. Changing the assigned policy during resize is not supported, as it adds complexity and can lead to image/flavor conflicts. Rebuilds are already blocked for vTPM instances, so once a policy is set via resize, it remains locked in. Existing instances from previous releases are unaffected. ✅ Do not allow changing the TPM secret security policy after assignment. ✅ Remove the option to select the policy from the image for simplicity. ✅ Default policy is “user”, but compute nodes support all policies by default. ✅ Document in the spec and release notes that deployers must define flavors with hw:tpm_secret_security if they want to enable this. ✅ Mention that [libvirt]supported_tpm_secret_security = ['user', 'host', 'deployment'] can be adjusted by operators. **** Wednesday topics **** #### CPU Comparison During Live Migration #### The team revisited the CPU compatibility check between source and destination hosts during live migration. If skip_cpu_compare_on_dest is enabled, some migrations may fail due to missing pre-checks; if disabled, the check can sometimes be too strict. The original goal is to avoid migrations to nodes running with QEMU emulation, which would cause serious performance issues. ✅ Include getDomainCapabilities in the Nova pre-check to improve accuracy. ✅ Move the configuration flag from workaround to the libvirt section. ✅ Document potential unsafe edge cases so deployers can choose between the safer (Nova) or more permissive (libvirt-only) approach. ### Start cross session with Cinder ### #### NFS / NetApp NFS Online Volume Extend #### The team discussed adding online volume extend support for NFS and NetApp backends. Instance operations (like shutdown or migration) should be blocked during the extend process, and any new task_state must stay compatible with existing microversions. ✅ Avoid using the event API, as it’s meant for fast operations. ✅ Introduce a dedicated API for this feature to manage versioning and states. ✅ Consider making the Cinder call synchronous for better control. ✅ Konrad will update the spec in this way. #### Hard-Coded io=native Setting for Cinder Backends #### The team revisited the hard-coded io=native driver setting used for Cinder volume backends (NFS, FC, and iSCSI). This code dates back nearly ten years and has recently caused performance issues and guest hangs in some environments. The original reason for enforcing io=native is unclear. ✅ No one seems to know the reason for this original choice ✅ Move the configuration to a standard libvirt configuration option. ### End cross session with Cinder ### #### Preserve NVRAM After Reboots and Migrations #### The team discussed the proposal to preserve NVRAM data across instance reboots and migrations. The feature is ready for review, pending available reviewer bandwidth. ✅ Request a definitive answer from libvirt regarding file permission handling through a bug report. ### Start cross session with Ironic/Cinder ### A more complete summary of this joint session will be provided by the Ironic team, as they can better capture the discussions and decisions made. However, the Nova PTG notes still include a few relevant takeaways and references for those interested. ### End cross session with Ironic/Cinder ### ### Start cross session with Manila ### The teams discussed testing and integration plans between Nova and Manila. Testing through Tempest was considered challenging due to configuration complexity and potential circular dependencies. Support for CephFS and NFS, hot attach, and live migration with newer OS versions was identified as a goal. ✅ Manila will draft initial UX and workflow details to share with Nova for review. ✅ Add a backlog spec in the Nova repository to track cross-team work. ✅ Manila contributors may help with open SDK and client patches as needed. ✅ Hot attach and live migration support are valuable goals but will need explicit demand and prioritization to move forward. ### End cross session with Manila ### ### Start cross session with Neutron ### A more complete summary of this joint session will be provided by the Neutron team, as they can better capture the discussions and decisions made. However, the Nova PTG notes still include a few relevant takeaways and references for those interested. ### End cross session with Neutron ### **** Thursday topics **** #### Correct Firmware Detection for Stateless Firmware and AMD SEV/SEV-ES #### The team discussed improving firmware detection for stateless firmware and AMD SEV/SEV-ES. Recent QEMU and libvirt versions now include firmware descriptor files that expose these capabilities, allowing libvirt to automatically select the right firmware without Nova duplicating that logic. The group agreed that leveraging libvirt’s detection is the better long-term approach. To avoid regressions, Nova should continue to preserve existing firmware for running instances during hard reboots, while new instances will rely on libvirt’s detection from the start. ✅ Use libvirt’s firmware detection instead of maintaining custom logic in Nova. ✅ Preserve the existing firmware for current instances on hard reboot. Let libvirt select firmware for new ones. ✅ Takashi will write a spec describing the approach and detailing the upgrade path. ✅ He will also check UEFI/stateless firmware tests in CI and run local SEV/SEV-ES tests. #### ROM-Type Firmware Support #### The team continued the discussion on firmware selection, focusing on ROM-type firmware used by AMD SEV-SNP, Intel TDX, and future Arm CCA. Since recent libvirt versions can automatically select the appropriate firmware and handle SMM when needed, Nova no longer needs to manage this logic directly. ✅ Skip further work in Nova and rely on libvirt’s firmware auto-selection mechanism. #### AMD SEV-SNP Support #### The team discussed initial plans for SEV-SNP support, which depends on firmware and requires recent QEMU (≥9.2) and libvirt versions. Work may be postponed to the 2026.2 cycle due to these dependencies. For SEV-SNP, there is a need to provide hostData, an immutable and attested field (up to 32 bytes) that can be set by Nova or external tools. Using Nova metadata for this purpose was ruled out. ✅ Do not use Nova metadata to provide SEV-SNP host data. ✅ The instance POST API could be extended to allow providing this information. ✅ Takashi will write an initial spec, keeping the design generic for both SEV-SNP and TDX. #### Generalize SEV / SEV-ES Code #### The team reviewed the proposal to generalize the existing SEV/SEV-ES code in preparation for future Confidential Computing architectures such as Intel TDX and Arm CCA. The work focuses on refactoring internal Nova code to make memory encryption handling more generic, without changing any user-visible behavior or API. ✅ Treat this as a code refactoring only (no external impact). ✅ Ensure it does not conflict with Takashi’s ongoing work. ✅ Implementation can proceed as a specless blueprint. ✅ No Tempest job available. Manual validation is acceptable, ideally with screenshots or logs. ✅ Note: It’s possible to run Tempest manually using a flavor requesting SEV/SEV-ES to verify correctness. #### Arm CCA Support #### The team discussed the roadmap for Arm CCA enablement in Nova. Upstream dependencies include Linux Kernel (Dec 2025), QEMU (Feb 2026), and libvirt (Mar 2026), with full availability expected around Ubuntu 26.04 (Apr 2026). As a result, development and testing in OpenStack are expected to start during the 2026.2 cycle. ✅ Features cannot merge until support is available in official distributions (not custom kernels). ✅ CentOS Stream 10 may serve as an early test platform if it gains CCA support before Ubuntu 26.04. ✅ Specs or PoCs can still be prepared in advance to ease future inclusion. ✅ Once libvirt support lands, the team will review the spec, targeting early in the H (2026.2) cycle. #### Show finish_time in Instance Action Details #### The proposal adds a finish_time field to the instance action show API, along with matching updates in the SDK and client. Some review comments still need to be addressed before re-proposing the patch for the G release. ✅ No objection to re-proposing the patch for Gazpacho (G). ✅ The existing spec remains valid. ✅ Gmaan will review previous comments to help move the implementation forward. ### Start cross session with Glance ### A more complete summary of this joint session will be provided by the Glance team, as they can better capture the discussions and decisions made. However, the Nova PTG notes still include a few relevant takeaways and references for those interested. ### End cross session with Glance ### #### Periodic Scheduler Update as Notification #### The proposal suggests exposing Nova’s periodic resource tracker updates as optional versioned notifications. This would help services like Watcher obtain a more complete and up-to-date view of compute resources, including NUMA topology and devices not yet modeled in Placement. While there are concerns about notification stability and data volume, the team agreed it could be useful for cross-service integrations. ✅ Support the idea in principle. ✅ Be mindful of the volume of data included in notifications. ✅ Sean will prepare a spec proposal detailing the approach. #### Resource Provider Weigher and Preferred/Avoided Traits #### The proposal introduces a resource provider weigher to influence scheduling decisions, with a longer-term goal of supporting preferred and avoided traits. This would allow external services, such as Watcher, to tag compute nodes (e.g., with CPU or memory pressure) and guide the scheduler to favor or avoid specific hosts. ✅ There are concerns about validating the weigher’s behavior at scale, ensuring it works correctly with a large number of instances. ✅ Continue developing this approach. ✅ Improve the test framework, despite existing challenges. ✅ Add Monte Carlo–style functional tests to validate behavior in complex, non-symmetric scenarios. #### Proxying OVH’s Monitoring-Aware Scheduling #### The team discussed OVH’s proposal to enhance scheduling decisions by integrating external monitoring data. The idea is to add a filter or weigher that delegates its logic to an external plugin, allowing operators to plug in custom metrics (e.g., Prometheus, InfluxDB) without modifying Nova directly. While the use case is relevant, existing out-of-tree filters and weighers already allow similar integrations. One of the main challenges identified was how to share and maintain such code consistently across the community. ✅ Current Nova interfaces likely already support this use case. ✅ Proposal to create a dedicated repository on OpenDev for sharing and maintaining custom filters and weighers. **** Friday topics **** #### LY Corp – Graceful Shutdown Proposal #### The team discussed LY Corp’s proposal to implement graceful shutdown handling in Nova. The goal is to stop accepting new RPC requests while allowing in-progress operations, such as instance boots or live migrations to complete safely. Different designs were compared, including stopping RPC listeners or using the upcoming oslo.messaging HTTP driver, which can handle clean shutdowns more reliably. The consensus was that the complete solution must involve oslo.messaging, allowing Nova services to dynamically subscribe or unsubscribe from topics. This approach also implies an upgrade impact, requiring versioned changes to compute and scheduler topics. ✅ The current proposal needs further work, as live migration cases were not yet considered. ✅ Gmaan will continue investigation and refine the approach based on oslo.messaging improvements. ✅ A revised spec will be submitted once ready. ✅ Gmaan confirmed commitment to follow up and lead this feature. #### Offline VM Migration Using HTTP-Based File Server #### The team discussed a proposal to enable offline VM migration using an HTTP-based file server. The preferred approach is to use WebDAV, which provides a standard protocol and existing server implementations. Nova itself will not host the web server and will be restricted to paths under /var/lib/nova. ✅ Use WebDAV as the preferred protocol. ✅ Avoid introducing non-standard protocols in Nova. ✅ Nova will not spawn or host the web server directly. ✅ Define a URL template in configuration for the remote filesystem driver (e.g., remotefs_url = https://%(host)s:9999/path/to/dav). ✅ Add DevStack support to test the feature in CI. #### PCI and VGPU Scheduling Performance in Placement The team revisited the performance issues with PCI and VGPU scheduling in Placement. Two workaround sets have already been implemented to reduce the number of valid and invalid candidates, but the core scalability problem of the GET /allocation_candidates query remains, especially in large, symmetric provider trees. A long-term constraint solver approach is under consideration, and the discussion will continue once more data is available from current deployments. ✅ Keep the current workarounds active and gather feedback over the next six months. ✅ Nova-next may run with these settings enabled to collect early results. ✅ Revisit the topic and defaulting decisions at the next PTG. ✅ Keep the issue open for urgent improvements if needed before the next cycle. #### Multithread and Multiqueue Support for Libvirt #### The team discussed adding multithread and multiqueue support in Nova for libvirt-based instances. While QEMU already enables multiqueue for virtio-blk and virtio-scsi by default, the main focus is now on IOThread support and improving parallelism during I/O and live migrations. ✅ Enable IOThreads with one thread per device by default (handled as a specless blueprint). ✅ virtio-blk and virtio-scsi specs are parked. ✅ Cancel the multiqueue work, since QEMU now manages it natively. ✅ The parallel migration blueprint is approved as specless. ✅ The earlier proposed spec will be abandoned in favor of this simplified path forward. #### AZ-Aware Scheduling (Soft Anti-Affinity) #### The team discussed improving availability zone–aware scheduling by introducing a soft anti-affinity policy for server groups. ✅ The overall direction is good, but the spec still needs more work. ✅ The team is broadly supportive of continuing in this direction. ✅ A spec will be proposed by Dmitriy. #### NUMA-Aware Placement (CloudFerro Proposal) #### The team discussed CloudFerro’s proposal to enable NUMA-aware scheduling in Placement, based on the old Ussuri spec. The goal is to represent NUMA nodes as resource providers to allow better locality handling for CPU and memory resources. However, this raises potential performance concerns, especially when combined with PCI in Placement, as both increase candidate permutations. ✅ Ensure NUMA and PCI in Placement work together correctly before expanding scope. ✅ Do not move PCI devices under NUMA subtrees yet. ✅ Add functional tests to validate the combined behavior. ✅ Consider extending CI to run on hosts with real NUMA topology. ✅ Enable NUMA awareness in nova-next (single NUMA node for now). ✅ Investigate with the infra team the possibility of adding dual-NUMA test nodes. ✅ Improvements to Placement’s group_policy remain out of scope for this work. ✅ The team is happy with the current direction and looks forward to future patches landing in master. #### WSGI Script Removal #### The team discussed removing the legacy wsgi_script support from Nova. No objections were raised, and no spec is required. ✅ Proceed with the removal, the team is aligned and ready to move forward. #### Improving Cyborg Support #### The team discussed reviving Cyborg integration efforts in Nova. Several aspects remain unfinished, including proper mdev GPU XML generation, support for additional device buses (block, NVMe, CXL, USB), and handling ownership overlap between Nova and Cyborg in Placement. The plan is to resume this work gradually, starting with the most mature components. ✅ Restart the Cyborg–Nova integration effort. ✅ Split the work into multiple specs, as each part is complex enough to warrant separate discussion. ✅ Focus first on mdev support, then extend to other buses (block, NVMe, etc.). ✅ Investigate Placement ownership to avoid inventory conflicts between Nova and Cyborg. If you've read this far, thank you! 🙏 If you spot any mistakes or missing points, please don't hesitate to let me know. Best regards. René.

2 2

[publiccloud-sig] Public Cloud SIG meeting d.d. 5 November
by Kees Meijs | Nefos 04 Nov '25

04 Nov '25

Hello Stackers, Tomorrow (that's 5 November) the bi-weekly Public Cloud SIG meeting will be held again. The publicly available video meeting will be conducted from 08:00 hrs UTC (09:00 hrs CET) until 08:45 hrs UTC (09:45 hrs CET) here: https://meetpad.opendev.org/publiccloud-sig Additional information to be found on https://wiki.openstack.org/wiki/PublicCloudSIG Cheers, Kees __ Kees Meijs BICT Nefos Cloud & IT <https://nefos.com/contact> Nefos IT bv Burgemeester Mollaan 34a 5582 CK Waalre - NL kvk 66494931 +31 (0)88 2088 188 <tel:+31882088188> nefos.com <https://nefos.com/contact> The information contained in this message is intended for the addressee only and may contain classified information. If you are not the addressee, please delete this message and notify the sender; you should not copy or distribute this message or disclose its contents to anyone. Any views or opinions expressed in this message are those of the individual(s) and not necessarily of the organization. No reliance may be placed on this message without written confirmation from an authorised representative of its contents. No guarantee is implied that this message or any attachment is virus free or has not been intercepted and amended. General terms and conditions ("The NLdigital Terms") apply to all our products and services.

1 0

[ironic][kolla][kolla-ansible] Missing DNS config during introspection
by Thomas Sell 04 Nov '25

04 Nov '25

Hey everyone, I'm facing a weird issue when trying to enroll baremetal compute hosts into Ironic using the self-enrollment/discovery method. Our Openstack is running 2025.1 and is based on Kolla-Ansible/Kayobe. What I'm seeing: - New hosts iPXE boot into IPA ramdisk. - IPA does its thing, gathering information. - IPA tries to report back to http://{{ kolla_internal_fqdn }}:5050 - The API call fails because no nameservers are available. I've tried to figure out where the IPA ramdisk is supposed to get its DNS config from and could not. There are no dhcp-options set in Ironic's dnsmasq.conf and looking at the Ansible template for this file, I believe this is by design. Inside the ramdisk /etc/resolv.conf is empty, yet IPA is configured to call-back to a FQDN, not an IP. I'm a bit puzzled about how this is supposed to work. We're using IPA images downloaded from https://tarballs.openstack.org. Do I need to build my own image? I also cannot find any mention of a separate DNS config for Ironic introspection in the documentation. Any help or hints are appreciated! Best, Thomas

3 3

[neutron] Bug Deputy Report Oct 27 - Nov 02
by Yatin Karel 04 Nov '25

04 Nov '25

Hi Neutrinos, Below is the bug deputy report for last week:- Critical ======== - https://bugs.launchpad.net/neutron/+bug/2130380 - OOM called from keepalived, "neutron-tempest-plugin-openvswitch" job Unassigned - https://bugs.launchpad.net/neutron/+bug/2130416 - [UT] CI job ``openstack-tox-py310`` and ``openstack-tox-py311`` frequently timing out Assigned to Rodolfo Alonso High ==== - https://bugs.launchpad.net/neutron/+bug/2129926 - DNS integration: port deletion fails when reverse DNS zone doesn't exist in Designate Assigned to Helen Chen - https://bugs.launchpad.net/neutron/+bug/2130405 - Neutron and Designate is not properly cleaned when a VM with a FIP is deleted Assigned to Helen Chen - https://bugs.launchpad.net/neutron/+bug/2130541 - [FT] test failure with no child process missing uuid Assigned to Rodolfo Alonso Medium ====== - https://bugs.launchpad.net/neutron/+bug/2129959 - Semantic violated with bulk creation of security groups Unassigned - https://bugs.launchpad.net/neutron/+bug/2130340 - [CI] functional test test__cleanup_previous_tags failure since Oct. 28, 2025 Assigned to Rodolfo Alonso - https://bugs.launchpad.net/neutron/+bug/2130564 - [FT] Error in ``test_initialize_nsrange_support_parallel_exec_diff_init_time`` Assigned to Rodolfo Alonso Wishlist ======== - https://bugs.launchpad.net/neutron/+bug/2130453 - [RFE] Support provider routed networks with multiple segments per host in the ML2/OVN driver Assigned to Miguel Lavalle - https://bugs.launchpad.net/neutron/+bug/2130570 - [RFE] Make "ovn-db-sync" to be pluggable Assigned to Slawek Kaplonski - https://bugs.launchpad.net/neutron/+bug/2130571 - [RFE] Make ovn maintenance tasks to be pluggable Assigned to Slawek Kaplonski New === - https://bugs.launchpad.net/neutron/+bug/2130211 ovn octavia provider related, Checking with froyo on this -- Thanks and Regards Yatin Karel

1 0

[tc][all] OpenStack Technical Committee Weekly Meeting Agenda (2026.1/R-21)
by Goutham Pacha Ravi 03 Nov '25

03 Nov '25

Hello Stackers, After the action-filled OpenInfra PTG week, The OpenStack Technical Committee will resume its regular meeting schedule this week. The next weekly meeting is scheduled to take place today, November 4, 2025, at 1700 UTC. Please find the meeting agenda and details for joining the meeting on the meeting wiki [1]. On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] TC Meeting Agenda, 2025-11-04: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[kolla-ansible][keystone] Could not recognize Fernet token
by Satish Patel 03 Nov '25

03 Nov '25

I am running 3 node controllers in multi-node deployment using the 2023.1 release and life was good until one of my controller nodes died. after losing one of the controller node I have started seeing following error in keystone logs 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application [None req-95a83b6e-dacc-45d4-8d1f-4d5c979f79e6 d58d8e432da74d8ba4416054d1e36a84 d1ef63c18c6648faa56643a942216311 - - default default] Could not recognize Fernet token: keystone.exception.TokenNotFound: Could not recognize Fernet token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application Traceback (most recent call last): 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/token/token_formatters.py", line 89, in unpack 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return self.crypto.decrypt(token.encode('utf-8')) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/cryptography/fernet.py", line 210, in decrypt 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application raise InvalidToken 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application cryptography.fernet.InvalidToken 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application Traceback (most recent call last): 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/token/providers/fernet/core.py", line 99, in validate_token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return self.token_formatter.validate_token(token_id) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/token/token_formatters.py", line 173, in validate_token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application serialized_payload = self.unpack(token) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/token/token_formatters.py", line 91, in unpack 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application raise exception.ValidationError( 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application keystone.exception.ValidationError: Could not recognize Fernet token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application Traceback (most recent call last): 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/flask/app.py", line 1820, in full_dispatch_request 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application rv = self.dispatch_request() 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/flask/app.py", line 1796, in dispatch_request 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/flask_restful/__init__.py", line 467, in wrapper 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application resp = resource(*args, **kwargs) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/flask/views.py", line 107, in view 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/flask_restful/__init__.py", line 582, in dispatch_request 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application resp = meth(*args, **kwargs) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/api/auth.py", line 285, in get 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/common/manager.py", line 115, in wrapped 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/token/provider.py", line 145, in validate_token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application token = self._validate_token(token_id) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/decorator.py", line 232, in fun 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return self.get_or_create( 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/dogpile/cache/region.py", line 1042, in get_or_create 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application with Lock( 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/dogpile/lock.py", line 185, in __enter__ 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return self._enter() 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/dogpile/lock.py", line 94, in _enter 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/dogpile/lock.py", line 178, in _enter_create 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application return self.creator() 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/dogpile/cache/region.py", line 995, in gen_value 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application created_value = creator( 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/token/provider.py", line 158, in _validate_token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application issued_at, expires_at) = self.driver.validate_token(token_id) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application File "/var/lib/kolla/venv/lib/python3.10/site-packages/keystone/token/providers/fernet/core.py", line 101, in validate_token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application raise exception.TokenNotFound(e) 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Could not recognize Fernet token 2025-11-04 01:13:14.514 25 ERROR keystone.server.flask.application I have rotated the fernet key and also verified in docker container and md5sum is match and date/time also very accurate. I am not sure what else I should look for. (keystone)[root@os1-ctrl01 fernet-keys]# md5sum /etc/keystone/fernet-keys/* 4874cf812b73f18cde82f1d00ea681e1 /etc/keystone/fernet-keys/0 e1a40f4cd58cfbc9b3b5af7305a16fc8 /etc/keystone/fernet-keys/214 0f45eeb318e3c4bfb97bb07da790329a /etc/keystone/fernet-keys/215 (keystone)[root@os1-ctrl03 /]# md5sum /etc/keystone/fernet-keys/* 4874cf812b73f18cde82f1d00ea681e1 /etc/keystone/fernet-keys/0 e1a40f4cd58cfbc9b3b5af7305a16fc8 /etc/keystone/fernet-keys/214 0f45eeb318e3c4bfb97bb07da790329a /etc/keystone/fernet-keys/215

1 0

[glance] Gazpacho PTG summary
by Cyril Roelandt 03 Nov '25

03 Nov '25

Hi, We have just had our virtual PTG. The full list of topics, along with notes, can be found on our Etherpad[1]. Here is a summary of the topics that will be targeted during this cycle. If you would like to view one of the recordings, please reach out to us so we can figure out a way to share it. # Configurable pr limits for qemu-img info We have agreed to make the pr limits for qemu-img info configurable, as is the case in Cinder. # Eventlet removal We will keep working towards moving our functional tests away from eventlet. # Download image from specific store We discussed the implementation details of this new feature, that will allow users to specify a specific store from which to download their images. # Eliminate redundant downloads We tried implementing this in the past and realized there were more challenges than we initially realized. We discussed the technical difficulties we faced during last cycle. # Improve image_size_total behavior in glance unified quota There are multiple scenarios that can lead to a user going slightly over quota in terms of the total size of images stored. We discussed ideas on how to mitigate the issue. # Replace glanceclient with OpenStack SDK We intend to have glanceclient go into maintenance mode in the I cycle. Our main concern is that some projects still use glanceclient as a library. We will be making sure they use the OpenStack SDK instead. # Add RBD Erasure-Coded Pools Support Using erasure-coded pools in Ceph used to cause a CPU overhead that was not acceptable for many operators. The situation has improved and the operators may now want to use this feature to save space. # Filesystem store improvement for NFS We will be improving the filesystem store so that operators face fewer issues when an NFS share is disconnected. # S3 Store Boto3 Checksum Behavior Configuration Some users are facing issues with S3-compatible storage if they are running the latest version of boto3. We discussed exactly how to work around the issue. # Image Encryption and disk format enforcement Image encryption has an impact on multiple OpenStack projects: Glance, Cinder and Nova, at least. Happy Hacking! Cyril Roelandt [1] https://etherpad.opendev.org/p/oct2025-ptg-glance

1 0

[neutron] Team and Drivers meeting canceled this week
by Brian Haley 03 Nov '25

03 Nov '25

Hello Neutrinos: As mentioned at the PTG, I am going to cancel the Neutron Team and Drivers meetings this week as I am out of the office. Have a nice week! -Brian

1 0

[tacker] Cancelling weekly IRC meeting
by Yasufumi Ogawa 02 Nov '25

02 Nov '25

Hi team, Due to a holiday in Japan and most of team members aren't join, I'm cancelling today's meeting. Regards, Yasufumi

2 1

[ironic] Stop maintaining sushy as a separate project, let's move it into ironic
by Iury Gregory 01 Nov '25

01 Nov '25

Hello everyone! During the PTG yesterday, we discussed stop maintaining sushy as a separate project. For those who don't know what sushy is: a Python library to communicate with Redfish based systems and is limited to what Ironic supports. The library was never intended for others to consume, yet that has happened, and it being a library has been a noted hurdle to quickly resolving user issues, because it requires releasing a new version. But before we move this forward, we would like to hear from you what are your thoughts. Thanks! -- *Att[]'s* *Iury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic Core - Ironic Event Liaison (DPL)* *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

6 6

[nova][glance] Image corruption during shelving
by Vladimir Prokofev 31 Oct '25

31 Oct '25

Hello everyone. I've discovered a bug in nova/glanceclient interoperation that leads to image corruption during shelving under some circumstances involving connection disruption. I was able to reproduce it in two different production environments running on xena and 2024.2 respectively, I'm also able to reproduce it in devstack though without image corruption - there it just ends up in a shelve failure due to detected corruption. My setup assumes LVM-backed QEMU VMs with backend in CEPH for Glance, but I believe this is applicable to a variety of nova/glance backends. When shelve is triggered, nova-compute creates an image file locally, and then initiates upload of said image file into Glance[0]. If something happens to the connection during the upload("broken pipe", "connection timeout") - nova-compute retries upload operation[1] while image object is removed from CEPH backend by Glance[2] Problem here is that the image_file object that is passed to glanceclient by nova-compute is a byte-stream created with an open()[3] call, and upon retry it resumes the upload from the point where it was interrupted. This is easily confirmed by calling image_data.tell() in glanceclient.v2.images.Controller.upload() function - it will be at zero initially and non-zero on retry. I haven't thoroughly checked protection offered in Glance master, I believe it may have been significantly improved, but in older releases such as 2024.2, this leads to a corrupted shelved image, because upon retry only part of an image object is uploaded, after which original VM is offloaded(removed) and you end up with lost data. Now, my issue here is where to submit this bug: glanceclient or nova? This problem is easily fixable in glanceclient by calling image_data.seek(0) in glanceclient.v2.images.Controller.upload(): it makes sense to always point byte-stream to the beginning before initiating upload, but should it really be responsibility of a client to perform such a sanity check? I'm also not sure if there're cases that this will break, for example if a non-seekable object is passed into glanceclient, but I'm not sure if this is even possible? [0] https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driv… [1] https://opendev.org/openstack/nova/src/branch/master/nova/image/glance.py#L… [2] https://opendev.org/openstack/glance_store/src/branch/master/glance_store/_… [3] https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driv…

2 2

[ptg] Image Encryption Session, Thursday 30th @ 16:00 UTC
by Markus Hentsch 30 Oct '25

30 Oct '25

Dear OpenStack community, in preparation for the vPTG discussion slot for the image encryption (Thursday 30th at 16:00 UTC) I collected a summary explaining the key points and state of the contribution and gathered some discussion topics below. Since this spans multiple projects (mainly Glance, Cinder and Nova) and has been ongoing for quite some years, I tried my best to briefly summarize the most important aspects of the current state so that all those who wish to participate have the latest information. The text below can also be found on the etherpad: https://etherpad.opendev.org/p/2026.1-ptg-glance-planning#L128 Summary: - historically, only Cinder has been able to create and use encrypted images (as a side effect) by dumping LUKS-encrypted volumes 1:1 into images, for exclusive use in Cinder - this contribution aims to introduce encrypted image functionality for the end user: images can be encrypted/decrypted outside of OpenStack and resulting (encrypted) image is stored in Glance and the key in Barbican - goal: standardize encrypted image formats and handling across Glance, Cinder and Nova - it introduces two formats: raw LUKS and qcow2-wrapped LUKS (native qcow2 feature) - it introduces new key handling: if Barbican secret is of type 'passphrase' it is used as the LUKS key as-is; if it is of any other secret type, it is assumed to be binary and converted via 'binascii.hexlify()' (preserves Cinder behavior and compatibility) - it makes use of the new secret consumer API in Barbican to prevent accidental key deletion (since keys may be managed by users, not OpenStack), registering images as consumers to their keys - note: the image encryption feature focuses on Glance and Cinder for now, Nova has been largely excluded so far due to ephemeral storage encryption not being ready yet Implementation State (note: aside from os-brick this is not merged yet!): - os-brick: implemented outsourced utility function to discern encryption between passphrase and binary (hexlify conversion) for shared usage by Nova and Cinder - Glance: implemented new encryption-specific image formats and properties as well as secret consumer handling - Cinder: implemented encrypted image usage for volumes: LUKS is handled as raw (like before) and qcow2+LUKS is converted; image creation from volume stays limited to raw LUKS - Nova: implemented only compatibility with os-brick and Cinder changes so far - barbican-tempest-plugin: full scenario tests (image > volume > server > SSH) for 4 permutations of the image encryption Patchsets: https://review.opendev.org/q/topic:%22LUKS-image-encryption%22 Discussion Topics: 1) Image inspection: related to CVE-2024-32498 concerns were brought up about encrypted image payload containing malicious formats triggering unintended QEMU behavior; how to screen encrypted image payload and reject undesired formats? 1a) limiting to specific formats has consequences for Cinder: attached (secondary) encrypted volumes might contain arbitrary binary data in VMs (user choice), when uploading as image to Glance it might be rejected as prohibited format; additionally, such image vanishes without trace because there is no error state for images (bad Glance <-> Cinder interaction) - how to address this? 1b) how deep do we want to look into encrypted image payload? decrypting a 1TB image might be very costly (CPU load, disk space) - problem: "qemu-img convert" cannot stream to stdout (see https://blogs.igalia.com/berto/2025/07/15/converting-qemu-qcow2-images-dire… ); in the worst case we have to wait for the full 1TB to be decrypted, consuming almost twice the amount of disk space in the process, instead of being able to only read the payload header and abort 2) Nova integration: how to approach future interoperability with ephemeral storage encryption? 2a) can we handle this as separate future expansion to this feature, i.e., merging Glance+Cinder functionality short-term with only compatibility changes in Nova for now and addressing full Nova integration as a next step with dedicated patchsets based on the current work? 3) Better organization: would there be any interest in reviving the image encryption popup-team meeting? https://meetings.opendev.org/#Image_Encryption_Popup-Team_Meeting Best regards, Markus Hentsch -- Markus Hentsch DevOps Engineer Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 00 markus.hentsch(a)cloudandheat.com | www.cloudandheat.com Green, Open, Efficient. Ihr Cloud-Service- und Cloud-Technologie-Provider aus Dresden. https://www.cloudandheat.com/ Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann

2 3

[skyline] possible small bug in skyline
by Taavi Ansper 30 Oct '25

30 Oct '25

Hi I have found a possible small bug in skyline. I didn't know where to post this in bugzilla. https://bugs.launchpad.net/skyline-apiserver/+bug/2129915 The bug comes from the placement API request to: //resource_providers/1ab32e69-ee87-4ccb-bd8a-b39abfa4a980/inventories The same UI view in horizon works nicely and it doesn't append the the double slash in the API request. Where might this come from the code? Something related to this is defined here, but it probably comes from the "Base" stuff: https://github.com/openstack/skyline-console/blob/stable/2025.1/src/client/… https://github.com/openstack/skyline-console/blob/stable/2025.1/src/client/… This issue appeared after upgrading from 2024.2 to 2025.1 Best regards -- ---- Taavi Ansper taavi.ansper(a)cyber.ee

1 0

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 29 Oct '25

29 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7