- openstack-discuss - lists.openstack.org

[nova] NVLink passthrough
by Pavlo Shchelokovskyy 05 Nov '25

05 Nov '25

Hi all, This is a continuing thread from 3 years ago here: https://lists.openstack.org/pipermail/openstack-discuss/2022-June/029273.ht… We've recently faced the same issue re Nvidia NVLink / NVSwitch, and I wanted to share how we are solving it. Given: - server with 8x Nvidia H200 pGPUs aimed to be used as PCI passthrough - separate pGPUs can be connected with NVLink but not any with any - there are 4 specific combinations to use when pairing 2 gpus - gpu1+gpu3, gpu2+gpu4, gpu5+gpu7, gpu6+gpu8 - and there are only 2 possible combinations of pairing 4 gpus - gpu{1..4} and gpu{5..8} The requirement is to have a single flavor that would ask for 2 GPUs, and the scheduling would pick only appropriate pair where NVLink between GPUs is possible, and compute will enable/disable NVLink for those GPUs as appropriate (same for 4 GPUs flavor). The compute part is solved by a libvirt hook provided by Nvidia here https://github.com/NVIDIA/libvirt-hooks/. Given the correct devices are chosen by Nova to be passed into the VM, the hook will enable NVLink between them. However, that scheduling part is currently rather impossible with Nova as-is. The best one can do is assign separate trait to appropriate pairs and quartets of GPUs, and then have 4 flavors for 2-GPU configuration and 2 flavors for the 4-GPU one. This if course is very inefficient and brittle, and can not work in a self-service cloud. What we are trying to do is the following: - add a new flavor extra spec `placement:same_subtree`. The value will be passed directly to placement's allocation candidates API. - in order for this to be usable in a flavor, we need predictable resource group names generated for PCI devices. Right now they are auto-generated from request-id, which won't work. - hence we extended `pci_passthrough:alias` spec to have an optional `:group_prefix` part. Then, what we do is reshape the PCI resource provider tree a bit. We introduce two extra layers of providers, let's call them `group4` and `group2`, each group has a specific trait, for example `CUSTOM_PCI_GROUP_OF_2` and `CUSTOM_PCI_GROUP_OF_4`. Two `group4` providers have compute nodes as a parent, and are parent to two `group2` providers each. Then we re-parent PCI resource providers to `group2` providers, so that each `group2` and `group4` tree corresponds to a set of GPUs that can be connected via NVLink. Luckily enough, resource tracker of nova-compute does not care about the parent provider, as long as the name and ID of PCI resource providers are not changed, so the reshape has no effect on the resource tracker. We also need to use all this with Aggregate Extra Spec filter active, so we've duplicated the `group_policy` flavor extra spec to `placement:group_policy` one. And of course, for all of this to work, one has to enable "PCI in placement" feature too https://docs.openstack.org/nova/2025.2/admin/pci-passthrough.html#pci-track… As a result, assuming the PCI alias for the GPUs is `h200` and its resource class is `PGPU`, we can create the following flavor extra spec: pci_passthrough:alias=h200:2:_nvlink2 trait_groupof2:CUSTOM_PCI_GROUP_OF_2=required placement:group_policy=none placement:same_subtree=_nvlink2-0,_nvlink2-1,_groupof2 that will result in the following placement query: GET /allocation_candidates ?resources_nvlink2-0=PGPU:1 &resources_nvlink2-1=PGPU:1 &required_groupof2=CUSTOM_PCI_GROUP_OF_2 &group_policy=none &same_subtree=_nvlink2-0,_nvlink2-1,_groupof2 That will select only a pair of GPUs that are children of a single provider with `CUSTOM_PCI_GROUP_OF_2` trait - which is exactly what we need! You can find this work proposed as a patch at https://review.opendev.org/966205. This is definitely just a beginning of discussion, and most certainly would require a spec, so I am eager to hear your thoughts and comments :-) Cheers, pas-ha -- Dr. Pavlo Shchelokovskyy Principal Software Engineer Mirantis Inc www.mirantis.com

2 1

[ironic] Proposed: Reviewer & Approver team updates
by Jay Faulkner 05 Nov '25

05 Nov '25

Ironic approvers had a discussion about the current state of the core team, and we propose the following changes. As some context for those not familiar, Ironic has two review groups: - Ironic-Approvers -- this group has the ability to +2 changes and Workflow +1 changes - Ironic-Reviewers -- this group has the ability to +2 changes, but cannot Workflow +1 changes I am proposing the promotion of our two inaugural members of ironic-reviewers into approvers. Both of these contributors have used their +2 judiciously and have earned the trust of the community to use that same judgement about which changes to approve: * Afonne-CID (cid) from GR-OSS (move from ironic-reviewer to ironic-approver) * Doug Goldstein (cardoe) from Rackspace (move from ironic-reviewer to ironic-approver ) Additionally, we'd like to add an existing community member to ironic-reviewers. Jakub Jelinek has become the community expert on RAID support in IPA and has deepened his knowledge across the codebase. We'd like to recognize that by asking him to join the ironic-reviewers team. * Jakub Jelinek (kubajj) from CERN (add to ironic-reviewer) Finally, we would like to move an existing ironic-approver and longtime contributor to emeritus status. Mark Goddard is no longer active in the OpenStack community, and I propose he be moved to emeritus status. This just means that while he may be removed from the team, if at some future point he returns to the Ironic community we may expedite his return to the approvers group. * Mark Goddard (remove from ironic-approvers) Thanks everyone for your contributions to Ironic, current and future. It's nice to see the growth of the team, and hope this opens the door for CID, cardoe, and kubajj to grow themselves and the team further. Thanks, Jay Faulkner G-Research OSS

1 0

[horizon][PTG] 2026.1 Gazpacho PTG summary
by Tatiana Ovchinnikova 05 Nov '25

05 Nov '25

Hello everyone, A huge thank you to everyone who joined the OpenStack Horizon team for the Gazpacho PTG, who organized sessions, presented topics, and actively contributed to the discussions! Thank you for making the PTG a success! The full list of sessions and topics can be found on our Etherpad [1] Here is the summary of the topics: # Flamingo Retrospective A review of the previous Flamingo cycle, highlighting positive developments like switching to SDK for certain services, improving TOTP setup, and improving region/domain selection. # Gazpacho Cycle Planning Planning for the current Gazpacho cycle, establishing AngularJS removal as the top priority. Other key goals include investigating the removal of old integration tests and planning for Django 5.2 support. # Open Discussion A session for discussing various in-progress code reviews. Topics included a discussion on a feature to add changing instance password support. # Django 5.2 Support Discussion on the plan to adopt Django 5.2, the latest LTS release. # Tests Review of current test stability and plans for test code cleanup. The team noted that both horizon-integration-pytest and horizon-ui-pytest are currently stable and plans to remove old integration tests from the repository are moving forward. We also discussed progress on implementing Manila UI integration tests as a template for other plugin teams. # Towards New OpenStack contributors: An OpenInfra User Group Kenya Story A presentation by Agnes Nduta to share the experiences and strategies of the OpenInfra User Group Kenya in successfully onboarding new contributors into the OpenStack ecosystem. # Contributor/Maintainer Surveys and Metrics Analysis A review of current project health using metrics and analysis from contributor and maintainer surveys. # How to create a Cursor MCP for analyzing Horizon code A walk-through demonstrating the steps required to set up a Cursor Maintainer Candidate Program (MCP) for analyzing Horizon code. # Multi-Realm Support and Multi-Region Support discussion A dedicated cross project session to discuss the implementation and implications of providing Multi-Realm and Multi-Region support within the Horizon dashboard. More details can be found on the separate Etherpad [2]. [1] https://etherpad.opendev.org/p/horizon-gazpacho-ptg [2] https://etherpad.opendev.org/p/multi-realm With kind regards, Tatiana

1 0

Nova error for fetch or server create after upgrade
by Ahmad Milad Pour 05 Nov '25

05 Nov '25

Hello there, I did upgrade the packages and repo, and something were upgraded: BEFORE: root@gregion83:~# apt-cache policy nova-api | head -n 10 nova-api: Installed: 3:29.2.0-0ubuntu1.1 AFTER: root@gregion83:~# apt-cache policy nova-api | head nova-api: Installed: 3:31.0.0-0ubuntu1.2~cloud0 I also did add "DEBUG=True" under "default" section of /etc/nova/nova.conf in my region and tried to create server, but I cannot find any important logs or details to see what's wrong: ==> /var/log/nova/nova-api.log <== 2025-11-05 14:42:44.524 2960 INFO nova.osapi_compute.wsgi.server [-] 185.137.25.83 "GET /v2.1 HTTP/1.1" status: 200 len: 790 time: 0.0164449 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi [None req-892920aa-c4be-4aea-abab-aa8e9fe8c7e0 64c74c80ec9a4a31ae27b82af846b36a 906db1c682bc4e3db 822c2e6dbc9cb6a - - default default] Unexpected exception in API method: neutronclient.common.exceptions.InternalServerError: Request Failed: internal serve r error while processing your request. Neutron server returns request_ids: ['req-80f3ccda-11ee-4417-a27c-2437540e6ee0'] 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.InternalServerError: Request Failed: internal server error while processing your request. 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-80f3ccda-11ee-4417-a27c-2437540e6ee0'] 2025-11-05 14:42:45.615 2960 ERROR nova.api.openstack.wsgi 2025-11-05 14:42:45.628 2960 INFO nova.api.openstack.wsgi [None req-892920aa-c4be-4aea-abab-aa8e9fe8c7e0 64c74c80ec9a4a31ae27b82af846b36a 906db1c682bc4e3db822c2e6dbc9cb6a - - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'neutronclient.common.exceptions.InternalServerError'> 2025-11-05 14:42:45.632 2960 INFO nova.osapi_compute.wsgi.server [None req-892920aa-c4be-4aea-abab-aa8e9fe8c7e0 64c74c80ec9a4a31ae27b82af846b36a 906db1c682bc4e3db822c2e6dbc9cb6a - - default default] 185.137.25.83 "GET /v2.1/servers/c18f669d-1338-42dd-947f-c40217238450 HTTP/1.1" status: 500 len: 660 time: 1.1018338 My last openstack version was 6.6.0, and I did upgrade after another email I sent "How to install Flamingo openstack in Ubuntu 24". Before upgrade, everything was working fine, but now I have issue. Would you please help me for this? Regards, Ahmad

2 3

[oslo][all] Proposing oslo.wsgi
by Stephen Finucane 05 Nov '25

05 Nov '25

o/ I'd like to propose the creation of a new "oslo.wsgi" project that service projects should adopt, where it makes sense to. My reasons for this proposal are three-fold: * As we've been working on the OpenAPI work for various services, we've found ourselves copy-pasting a lot of the code between these services. This has occasionally led to issues when those copies are incomplete [1][2]. Historically speaking, copy-pasting of code between services is a sure sign that this code should be generalised and placed in an oslo library * I have previously noted [3] that oslo.service contains some non- eventlet-related WSGI routing code, and there are questions around whether (a) that code should have been placed there in the first place and (b) how we can maintain that long-term once the eventlet code is removed (the 'oslo_service.wsgi' module has side effects since it imports the backend configuration stuff). * I have also raised some concerns around our use of paste.deploy and routes [1], both of which are minimally maintained nowadays. Many oslo libraries are designed to abstract their underlying libraries, allowing users to switch between e.g. different messaging backends (oslo.messaging) or caching services (oslo.cache). Taken together, I think now would be a good time to provide a new library. I have an initial prototype available [4] which is effectively just a refactoring of the code from oslo.service, but I expect to add both more functionality to this existing code from existing consumers (Nova, Manila, Cinder etc.) and to add the openapi validation machinery. I imagine longer-term, this should evolve to allow us to replace the underlying libraries. However, we should obviously not get into the business of writing our own web framework. I also don't think it would make sense for all services to adopt all aspects of this project, since not all projects have any OpenAPI efforts ongoing at the moment and not all of them use the pastedeploy-routes-webob combo. Given the above rationale and caveats, what do people think? Are there any objections to this idea? Please let me know. If there is not, I will go figure out how to get the ball rolling on this. Cheers, Stephen PS: Why oslo.wsgi instead of a more generic name like debtcollector or futurist? I don't think this library will be useful outside of OpenStack, so the oslo prefix makes sense. [1] https://review.opendev.org/c/openstack/ironic/+/963938 [2] https://review.opendev.org/c/openstack/ironic/+/960291 [3] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [4] https://github.com/stephenfin/oslo.wsgi

6 10

How to install Flamingo openstack in Ubuntu 24
by Ahmad Milad Pour 05 Nov '25

05 Nov '25

Hello there, I use Ubuntu 24 and I did try to install openstack, but it seems it's not the latest one. root@gregion83:~# cat /etc/os-release PRETTY_NAME="Ubuntu 24.04.3 LTS" NAME="Ubuntu" VERSION_ID="24.04" VERSION="24.04.3 LTS (Noble Numbat)" VERSION_CODENAME=noble ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL=" https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=noble LOGO=ubuntu-logo root@gregion83:~# openstack --version openstack 6.6.0 root@gregion83:~# apt-cache policy nova-api | head -n 10 nova-api: Installed: 3:29.2.0-0ubuntu1.1 Candidate: 3:29.2.0-0ubuntu1.2 Version table: 3:29.2.0-0ubuntu1.2 500 500 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages *** 3:29.2.0-0ubuntu1.1 100 100 /var/lib/dpkg/status 3:29.0.1-0ubuntu1.4 500 500 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages root@gregion83:~# apt-cache policy openstack-dashboard | head -n 10 openstack-dashboard: Installed: 4:24.0.1-0ubuntu0.24.04.1 Candidate: 4:24.0.1-0ubuntu0.24.04.1 Version table: *** 4:24.0.1-0ubuntu0.24.04.1 500 500 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages 100 /var/lib/dpkg/status 4:24.0.0-0ubuntu1 500 500 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages root@gregion83:~# curl -s -H "X-Auth-Token: $(openstack token issue -c id -f value)" http://$HOSTNAME:8774 | jq { "versions": [ { "id": "v2.0", "status": "SUPPORTED", "version": "", "min_version": "", "updated": "2011-01-21T11:33:21Z", "links": [ { "rel": "self", "href": "http://gregion83.domain.com:8774/v2/" } ] }, { "id": "v2.1", "status": "CURRENT", "version": "2.96", "min_version": "2.1", "updated": "2013-07-23T11:33:21Z", "links": [ { "rel": "self", "href": "http://gregion83.com:8774/v2.1/" } ] } ] } That's the information I thought is good for the help. And that's the error I have: root@gregion83:~# openstack server rebuild --image 608349cf-8743-4339-8e2c-e707f89be64f --no-user-data c18f669d-1338-42dd-947f-c40217238450 --os-compute-api-version 2.54 or greater is required to support the --no-user-data option root@gregion83:~# openstack server rebuild --image 608349cf-8743-4339-8e2c-e707f89be64f --user-data /dev/null c18f669d-1338-42dd-947f-c40217238450 --os-compute-api-version 2.54 or greater is required to support the --user-data option I see this link from which I can install: https://docs.openstack.org/horizon/2025.2/install/install-ubuntu.html How to resolve this issue? Regards, Ahmad

3 4

[PTG] Call for Summaries, Attendance Numbers, and Team Photos
by Kendall Nelson 04 Nov '25

04 Nov '25

Hello Folks! I am working on a summary post for our OpenInfra Blog as usual and I am missing A LOT of attendance numbers from OpenStack teams. Specifically, I am missing attendance numbers from: - Barbican - Blazar - Cinder - Cloudkitty - Cross Deploy - Designate - Eventlet Removal - Glance - Heat - Horizon - i18n - Ironic - Keystone - Kolla - Magnum - Neutron - OpenStack Ansible - OpenStack Helm - Sunbeam - Swift - Tacker - Telemetry - Watcher If you were a moderator for this session please send me your approximate attendance to ptg(a)openinfra.org ASAP. If you simply attended one of these sessions, and you would be so kind as to remind the moderator to send me the info I would be very grateful! Additionally, I will happily accept any team photos that were taken and links to blog posts or other summaries that I can include in OpenInfra Blog post as well. There are a lot of open source communities outside of the OpenInfra Foundation that are curious about how we conduct our PTGs and these posts are incredibly helpful at educating others on how to follow the four opens and build a stronger community. Thank you for all of your support! Looking forward to the next PTG! -Kendall

1 0

[kolla-ansible] Upgrade from 2024.1 to 2024.1 neutron-l3-agent unhealthy
by hamid.lotfi＠gmail.com 04 Nov '25

04 Nov '25

Hi everyone, I’m hitting a weird issue after a kolla-ansible upgrade and would appreciate any pointers. Environment kolla-ansible 18.8.1 rabbitmq 4.1.3 (deployed by Kolla) upgrade path: 2024.1 → 2025.1 What happened Ran kolla-ansible -i … upgrade from 2024.1 to 2025.1. The upgrade finished without obvious errors in kolla logs. Right after the upgrade every neutron-l3-agent container on all control/compute nodes is marked unhealthy (no ERROR/FATAL lines in the container logs). All tenant instances lose external connectivity. Simply docker restart neutron_l3_agent on each node is enough: containers become healthy and external access is restored instantly. No other OpenStack service shows the same behaviour. Any idea why the agents start in “unhealthy” state until manually restarted? Happy to collect more data if you tell me what to look for.

1 0

[tc][horizon]Request for guidance on improving Python PTI doc to include pytest for Horizon plugins testing
by Chandan Kumar 04 Nov '25

04 Nov '25

Hello TC, We are writing to seek your guidance regarding the Python testing standards (PTI) as they relate to Horizon plugins. The current integration tests of the watcher-dashboard project (the Horizon plugin for OpenStack Watcher) are broken due to integration code changes in the Horizon integration suite. We are currently working on rewriting the watcher-dashboard integration tests. We noted that the Horizon project itself has developed a robust set of reusable, pytest-based integration tests, fixtures, tox targets and zuul job[1]. We also see that other plugins, like manila-ui, are already reusing these pytest fixtures.[2]. Several other projects within the OpenStack ecosystem (such as skyline-apiserver, rally, and projects under the Airship and StarlingX namespaces) are already using pytest. This presents a conflict for the Watcher team. The official Python PTI states that tests should be written using the Python stdlib unittest module[3]. The Watcher team advocates for adhering to the Python PTI and using unittest. Our main watcher project uses unittest, and we prefer to maintain this standard for consistency and PTI compliance in watcher-dashboard. This topic came up during watcher PTG discussion[4]. This leaves us with a dilemma: - Follow the Python PTI: This aligns with the PTI and our team's standards but requires us to ignore Horizon's reusable pytest tests and build our own testing framework from scratch, duplicating effort. - Follow the parent project (Horizon) to use pytest to reuse their fixtures. This would be more efficient but appears to violate the Python PTI and creates inconsistency with our main project. - Do we want to improve Python PTI documentation to include pytest usage? We just need guidance on this topic. Links: [1]. https://github.com/openstack/horizon/tree/master/openstack_dashboard/test/s… [2]. https://github.com/openstack/manila-ui/tree/master/manila_ui/tests/selenium [3]. https://github.com/openstack/governance/blob/master/reference/pti/python.rs… [4]. https://etherpad.opendev.org/p/watcher-2026.1-ptg#L404 With Regards, Chandan Kumar

1 0

[qa] Tempest gate blocked and dropping python 3.9
by Ghanshyam Maan 04 Nov '25

04 Nov '25

Hi All, Requirement repo has dropped the Python 3.9 constraints[1], and due to that, the Tempest gate is blocked because the Python 3.9 unit test job is failing. We discussed this type of issue in the past[2], that if the requirement repo drops older Python version constraints, then Tempest can drop the support of that Pythion version. Tempest master can still be used to test the Python 3.9 env; for that, you need to install tempest in a Python 3.10 (or higher) env. I have proposed the change to drop py3.9[3], and Yatin's change will unblock the stable branch testing[4]. Hold your recheck until those fixes are merged. [1] https://review.opendev.org/c/openstack/requirements/+/957587 [2] https://lists.openstack.org/pipermail/openstack-discuss/2022-May/028711.html [3] https://review.opendev.org/c/openstack/tempest/+/966101 [4] https://review.opendev.org/c/openstack/tempest/+/965903 -gmaan

1 1