January 2026 - openstack-discuss - lists.openstack.org

[openstack][nova]qemu system wont release memory
by Nguyễn Hữu Khôi 25 Aug '26

25 Aug '26

Hello. I have an OpenStack instance that was using 14GB out of 16GB of memory. Later, when it was using only 4GB, I checked the QEMU process on the compute node, and it was still consuming 14GB, this means memory isn't released when usage decreases. Do we have solutions for this? I use Ubuntu 24.04 instances. Nguyen Huu Khoi

2 13

kolla-ansible support for Ubuntu 2024.04/ARM64 ?
by dariusz.bursztynowski＠pw.edu.pl 18 Aug '26

18 Aug '26

Hello, Are there plans for kolla-ansible to support Ubuntu 24.04 / ARM64 (e.g., to have LTS support for Raspberry Pi 5)? Best regards, Dariusz Bursztynowski

2 1

[barbican][oslo][nova][glance][cinder] cursive library status
by Brian Rosmaita 26 May '26

26 May '26

Hello Barbican team, Apologies for not including barbican in the previous thread on this topic: http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019430… The situation is that cursive is used by Nova, Glance, and Cinder and we'd like to move it out of the 'x' namespace into openstack governance. The question is then what team would oversee it. It seems like a good fit for Oslo, and the Oslo team seems OK with that, but since barbican-core is currently included in cursive-core, it make sense to give the Barbican team first dibs. From the consuming teams' side, I don't think we have a preference as long as it's clear who we need to bother about approvals if a bugfix is posted for review. Thus my ask is that the Barbican team indicate whether they'd like to move cursive to the 'openstack' namespace under their governance, or whether they'd prefer Oslo to oversee the library. Thank you! brian

6 6

[stackalytics] no "neutron" module for Commits
by Ihar Hrachyshka 07 May '26

07 May '26

Hi, I noticed that "neutron" is not an option in the Module list when Commits are selected, and no neutron commits are registered since ~Oct 2023. On the other hand, other Neutron program repos (e.g. neutron-lib and neutron-tempest-plugin) are available in the list and do register commits. Also, when Reviews are selected, neutron module is back as an option, and review stats are correctly updated. I checked governance yaml and it doesn't strike me as incorrect: https://opendev.org/openstack/governance/src/commit/5f93b99a3c90b452f6c7368… Any ideas what may have happened? Thanks, Ihar

5 10

[nova] Ephemeral storage potential features
by Karol Klimaszewski 24 Apr '26

24 Apr '26

Hello, In my company (CloudFerro) we have developed two features related to nova ephemeral storage (with libvirt virt driver) - "SPDK-based ephemeral storage backend" and "Multiple ephemeral storage backend handling" (both described below). Would these features be appropriate to be added to upstream? If yes, then should I start with creating a blueprint and proposing a spec for each of them so it is more clear what we want to introduce? Of course not both at once, since there is some code that makes one work with the other. For us it would probably be best if we upstreamed "SPDK-based ephemeral storage backend" first and then upstreamed "Multiple ephemeral storage backend handling" Description of SPDK-based ephemeral storage backend: We add a new possible value for [libvirt]/images_type in nova.conf: spdk_lvol. If this value is set then local disks of instances are handled as logical volumes (lvols). of locally run SPDK instance (on the same compute as VM). See https://spdk.io/doc/logical_volumes.html#lvol for docs on this subject. These are essentially a part of local NVMe disk managed by SPDK. We create and manage those lvols by making calls from nova-compute to SPDK instance with RPC (https://spdk.io/doc/jsonrpc.html) using provided python library (https://github.com/spdk/spdk/blob/master/python/README.md) We attach those lvols to instances by exposing them as vhost-blk devices (see https://spdk.io/doc/vhost.html) and specifying them as disks with source_type='vhostuser'. This method of exposing NVMe local storage allows for much better I/O performance than exposing them from local filesystem. We currently have working: creating, deleting, cold-migrating, shelving, snapshots, unshelving VMs with this storage backend. We don't have live migration working yet but have it in plans. This feature includes changes only to nova. Description of Multiple ephemeral storage backend handling: We add a new configuration option [libvirt]/supported_image_types to nova.conf of nova-compute and change the meaning of [libvirt]/images_type to mean default image type. If libvirt:image_type extra_spec is specified in flavor then VM is scheduled on compute with appropriate image type in its [libvirt]/supported_image_types. We use potentially multiple DISK_GB resource providers and construct appropriate request groups to handle this scheduling. This spec is also used to fill driver_info.libvirt.image_type field of BDMs with destination_type=local of this VM. (driver_info is new JSON serialized field of BDM) Then in compute if BDM specifies this field we use its value instead of [libvirt]/images_type to decide on imagebackend to use for it. This method of handling multiple backends only works after being enabled by administrators by setting libvirt:image_type on all their flavors and running nova-manage commands that update existing VMs. Without enabling everything works as it was before. This feature includes changes to nova and some new traits in os-traits (One per possible value of [libvirt]/images_type + one extra) Best regards, Karol Klimaszewski

2 8

Upcoming metrics dashboard changes
by Ildiko Vancsa 21 Apr '26

21 Apr '26

Hi OpenStack community, Since the OpenInfra Foundation joined the Linux Foundation, we have been looking into opportunities to take advantage of the frameworks and tool sets that the Linux Foundation has, and some of you and your organizations are already familiar with. While the dashboards are still being built I would like to let you know that, as part of that effort, we will switch to LFX Insights[1] as the metrics dashboard for OpenInfra projects, including OpenStack, starting on __January 15, 2026.__ A couple things to note: - The new OpenStack metrics dashboard[2] is already available, but it is __still under active construction.__ - The set of metrics that is displayed on the new dashboard will not be identical with the metrics on the dashboard provided by Bitergia. - LFX Insights does not display affiliation information for individuals who are listed on the dashboard as contributors, for privacy reasons. However, if you already have an LF account[3], you will be able to check and update your affiliation information. Or, you can create a new LF account[4]. - You can learn more about the dashboard in the LFX Insights documentation[5]. - If your organization is an OpenInfra Foundation / Linux Foundation member already, the LF Organization Dashboard[6] is also available for a more company-centric view. The OpenStack Bitergia metrics dashboard[7] remains available __until January 14, 2026.__ Please let me know if you have any questions. Best Regards, Ildikö [1] https://insights.linuxfoundation.org/ [2] https://insights.linuxfoundation.org/project/OpenStack/ [3] https://openprofile.dev/ [4] https://docs.linuxfoundation.org/lfx/sso/create-an-account [5] https://insights.linuxfoundation.org/docs/introduction/what-is-insights/ [6] https://lfx.linuxfoundation.org/tools/organization-dashboard/ [7] https://openstack.biterg.io/ ——— Ildikó Váncsa Director of Community OpenInfra Foundation

4 14

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 16 Apr '26

16 Apr '26

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 17

Help with vGPU's using an NV L4 card on U24.04 (6.8)
by Tyler Wilson 15 Apr '26

15 Apr '26

Hello All, I'm trying to get vGPU's to work with openstack and I believe I am almost there, however I can't seem to get them to register to placement. Here is what I have so far OS: Ubuntu 24.04 Kernel: 6.8.0-62-generic Nvidia GRID Version: 570.158.02-570.158.01-573.39 Openstack Deployment: Kolla w/ Docker Openstack Version: 2025.1 I have the SRIOV devices enabled with the systemd service, and can see the vgpu types; # cat /sys/bus/pci/devices/0000:25:00.0/virtfn0/nvidia/creatable_vgpu_types ID : vGPU Name 908 : NVIDIA L4-1B 909 : NVIDIA L4-2B 910 : NVIDIA L4-1Q 911 : NVIDIA L4-2Q 912 : NVIDIA L4-3Q 913 : NVIDIA L4-4Q 914 : NVIDIA L4-6Q 915 : NVIDIA L4-8Q 916 : NVIDIA L4-12Q 917 : NVIDIA L4-24Q 918 : NVIDIA L4-1A 919 : NVIDIA L4-2A 920 : NVIDIA L4-3A 921 : NVIDIA L4-4A 922 : NVIDIA L4-6A 923 : NVIDIA L4-8A 924 : NVIDIA L4-12A 925 : NVIDIA L4-24A My kolla node nova custom config (for nova-compute) is [DEFAULT] debug = true verbose = true [devices] enabled_mdev_types = nvidia-918 [mdev_nvidia-918] max_instances = 8 [libvirt] live_migration_downtime = 500000 live_migration_downtime_steps = 3 live_migration_downtime_delay = 3 I've also created flavors, classes, and traits with; openstack flavor create l4-1a --ram 8192 --disk 40 --vcpus 4 openstack flavor set l4-1a --property "resources:VGPU=1" --property "trait:CUSTOM_NVIDIA_918=required" openstack resource class create CUSTOM_NVIDIA_918 openstack trait create CUSTOM_NVIDIA_918 However I can't seem to get placement to show any vGPU's or any of the traits I registered: This just has an empty line # openstack allocation candidate list --resource VGPU=1 and this will just show the standard vCPU/Memory/Disk # openstack resource provider inventory list <Host UUID> Have I missed a step somewhere, do I need to prepare the devices further before nova can pick them up? Thanks for any and all help!

3 3

[openstack][share]Openstack Workload Balancer
by Nguyễn Hữu Khôi 15 Apr '26

15 Apr '26

Hello, I have a script to make Openstack workload balance(CPU and RAM). I would like to share it. This script is not perfect but I hope it will be useful for you. https://github.com/nguyenhuukhoi/OpenstackWBalancer Nguyen Huu Khoi

1 1

[all][keystone] Keystone PTG summary
by Artem Goncharov 08 Apr '26

08 Apr '26

Hey guys Let us give you an update of the OpenStack Keystone future following the great PTG week we just had. Some news has already spread across the community, causing excitement and anxiety. It is not uncommon that changes cause fear. Some time ago, in the age of the Inquisition, people were even burned because of fear, misunderstanding, or questioning the rules. Let us clarify things. Nearly 13 years ago, the Identity v3 API was introduced. Since then, the world has changed dramatically. Libraries we use got abandoned, standards went legacy and replaced with new ones, ecosystem components, certain functionality is based upon are being sunsetted by OS vendors (mod_auth_mellon is archived upstream and gone in CentOS and RHEL [6]), new security vulnerabilities (classes of vulnerabilities) appear like mushrooms without us being able to address them. Cloud providers today face new challenges that were not present 10-15 years ago. Nowadays, a public cloud requires proper federation support with complete self-service. Users request passkey support, and security departments require additional hardening. These things cannot be covered in today's Keystone architecture and API version. As a result, work on an optional extension to Keystone [1] began some time ago, allowing CSPs to deploy it in parallel with the Python Keystone (of any v3 version) and access the new features. To simplify parallel deployment, it has been decided to start introducing v4 with the new features (since those ARE for cloud end-users). The work was done in Rust due to multiple factors: lack of necessary python libraries with trustable maintenance and/or licensing, necessity for enforcing security standards and practices using memory safe programming language, demand to address python ecosystem madness of past years (I guess majority would agree, that in the past few years we were mostly only chasing the python ecosystem breaking us on setuptools, eventlet, passlib, …). During the work, it became clear that implementing the mentioned features requires support of a given percentage of the existing functionality: users, groups, projects, roles, and tokens. And so it was implemented. A side effect of it became immediately visible: performance. Token check is THE most frequently called API in the OpenStack cloud. Right now, caching is used to reduce system stress. But in Rust, token decryption, without caching, is much faster than in python with caching (we expect a 2-3x faster order of magnitude). As mentioned above, new vulnerability classes and attack vectors require us to disable caching, making such an improvement highly valuable. The new stack is fully async, uses parallel DB queries, and runs background tasks for periodic cleanups, synchronization, and similar. This allows us to achieve significant performance and maintainability improvements, eliminate the old ballast, and significantly reduce transitive dependency version conflicts. Current measurements also indicate that we would likely be able to eliminate the need for caching in Keystone. For python, on the other hand, we do not have much to start with and would need to find and migrate to the new web framework, find maintained libraries with suitable licenses, and implement new features. We also need to rework the DB access patterns due to the number of bugs and the performance penalty caused by suboptimal DB access (many rows are selected multiple times during processing of a single request). With new authentication flows, we would need to do a wide security assessment of Keystone from scratch anyway. With that work in mind and facing the prospect of having a major rework in python Keystone anyway (due to the abandoned complex dependencies), we, as the Keystone team, decided to try to focus the work on the Rust implementation, making it a proper evolution of the python Keystone and building a necessary community around it. This adoption is conditional on adding new contributors to support the project. We believe this approach allows the best of both worlds: the trusted maturity of Keystone’s Python code-base, combined with the modern, high-safety, high-performance capabilities of Rust where they matter most. Logically, this raises questions, so let's try to answer some of them: 1. Rust is not officially allowed in OpenStack. How are you dealing with that - The development is now going on on GitHub outside of OpenStack governance. - We would apply for OpenStack universe inclusion. - Once the TC policy allows Rust or the overall project inclusion requirement is reworked, the project will work toward becoming the official OpenStack project. - No one in the team is willing to separate from OpenStack, but with the current policies, we have no other way. Everybody and everything in the OpenStack ecosystem would benefit from that project being official to have first-class native support of the new functionality. - Multiple CSPs expressed support for such a move, desiring to see Rust in the ecosystem for better performance, maintainability, and resource utilization - A nice coincidence is a freshly sent announcement from Debian that introduces a strong rust dependency in the packaging tooling, stating something similar to: either you work to comply with this requirement in the next 6 months or sunset the port [5]. - Rust allows cross-integration with python with a high number of cryptography and typing libraries for python written directly in Rust. We are going to be able to reuse certain pieces of python in Rust during porting and Rust in python later in the future. - Keystone is a very sensitive component and in our eyes that fully justifies use of rust to provide a much stronger security on the language level. 2. Do you deprecate python Keystone. - No - v3 is there, and we continue maintaining it - We would unlikely accept major changes - Keystonemiddleware, keystoneauth, etc, are still there. They would likely be adopted for v4 support at some point in the future (in OpenStack services, we already communicate with “external” software, so there should be no problem). Those libraries are crucial for other services and thus are there to stay while necessary. - At some point, we will no longer be able to keep it alive when abandoned dependencies are physically gone. We are not going to invest our time in maintaining the abandoned software; we do not have enough time to do so. 3. What is the community around it? - As of now, there are provisional commitments from a few vendors and CSPs to collaborate on it. - Existing software is being transferred under the openstack-experimental organization to help build the community [3]. -The initial author (gtema) personally commits to continue working on that independently of others. 4. What are the /v3 features that would go away? - OIDC federation in its current form. v4 reimplements it from scratch natively without depending on external software (mod_auth_oidc, etc) since those make self-service impossible. - Saml2, this is a legacy protocol that is very unlikely to even get fixes for security vulnerabilities, and looks to be PQC unsafe. - Oauth1. 5. What new features are there in v4? - OIDC with self-service and CLI integration - Workload federation (Zuul, GitHub, Gitlab do not need to have secrets to access OpenStack anymore) - Service accounts (many use cases, one of the prominent ones: OpenStack on Kubernetes, GitHub workflow) - Passkeys (webauthn) - SCIM - Tight Open Policy Agent integration. We plan to consider the reintroduction of central policy storage in Keystone (not enforcing, but for central storage)[4] We will use the next 2-3 months to gather interest in contributing to this approach and the new Rust-based code. There was a lot of interest in the new features listed right above during gtema’s sessions in OpenInfra Summit in October 2025. Still, it has to translate into active contributors for this effort to take off. If you have an interest in those new features, join us! We hope this clarifies the situation. Feel free to reach out with other questions. Thanks to everybody who participated in the discussions. Regards, Keystone team [1] https://github.com/gtema/keystone [2] https://www.youtube.com/watch?v=XOHYqE2HRw4&list=PLKqaoAnDyfgr91wN_12nwY321… [3] https://github.com/openstack-experimental/keystone [4] https://www.youtube.com/watch?v=_B4Zsd8RG88&list=PLKqaoAnDyfgr91wN_12nwY321… [5] https://lists.debian.org/debian-devel/2025/10/msg00285.html [6] https://access.redhat.com/solutions/2743391

11 21