- openstack-discuss - lists.openstack.org

[kolla-ansible] Upgrade 2024.1 to 2025.1 inconsistent behavior
by Eugen Block 14 Aug '26

14 Aug '26

Hi, I have a (virtual) multinode lab environment deployed with kolla-ansible: - 1 management node - 3 control nodes - 2 compute nodes (9 running instances) - 3 gateway nodes (loadbalancer) Before actually upgrading OpenStack, I also perform a release-upgrade from Ubuntu 22.04 to 24.04. To make my life easier, I created snapshots from all the involved VMs to be able to start over with the actual OpenStack upgrade. I've tried that a couple of times and the outcome is not always the same. My current starting point is when all nodes are on Ubuntu 24.04 and all the new images have been pulled successfully (2025.1-ubuntu-noble). RabbitMQ has been reconfigured as well (quorum queues), I merged the passwords.yml, updated globals.yml and my multinode file. The prechecks run successfully, the cloud is fully operational. I've been facing different errors and I can't seem to find an explanation for all of them. The first issue was that for some reason the module "List all hypervisors" failed with a 503. Everything I tried from within the kolla toolbox worked, so I'm clueless. But it hasn't failed in every upgrade attempt, that's the weird part. I've been digging a lot and couldn't find a reasonable explanation. At one point after the upgrade stopped I did a kolla-ansible reconfigure and that fixed it and new nova containers were deployed. Then there's this task that also seems to fail only sometimes: "Check if running on all OVN NB DB hosts" For some reason control-3 was not in the target list. But this error also doesn't show up every time. So again, I'm clueless why that could be. Today a new error popped up, rabbitmq wasn't able to upgrade due to: Ranch acceptor reducing accept rate: out of file descriptors The current workaround is to give rabbit more file descriptors (systemctl edit kolla-rabbitmq-container.service) before the upgrade procedure. I found the launchpad for the rabbit issue, but nothing for the rest. Is this some sort of limitation with my tiny management node (2 VCPUs, 2 GB RAM)? Or does anyone have any idea what I'm missing here? Thanks in advance! Eugen

1 4

[neutron] Bug deputy report for week of August 10th
by Miro Tomaska 14 Aug '26

14 Aug '26

Here is my bug deputy report for the week of August 10. It is a little early as I will be on vacation next week. I also not be attending the Neutron meeting where this will be discussed. I tried to comment of each LP *Medium*: Bug 2163189 - neutron-vpnaas rally job is failing. Link https://bugs.launchpad.net/neutron/+bug/2163189 Assignee Lajos Katona Bug 2163236 - Out of sync ipallocations and IPAM tables lead to failing new FIP requests. Link https://bugs.launchpad.net/neutron/+bug/2163236 Assign Andrei Dragan Bug 2163291 - Return explicit error when adding multiple address families into a "evpn" router. Link https://bugs.launchpad.net/neutron/+bug/2163291 Assignee: Miro Tomaska *Low*: Bug 2163184 - When disabling Port Forwarding service plugin OVN DB objects remain. Link https://bugs.launchpad.net/neutron/+bug/2163184 Assignee Unassigned I think this one is pretty straightforward; we can probably fix it in maintenance task Bug 2163310 - add password support to evpn frr templates. Link https://bugs.launchpad.net/neutron/+bug/2163310. Assignee: Unassigned. Someone working on "evpn" can pick it up, but I dont think it is urgent fix. Bug 2163394 - fwaas tests which are in neutron-tempest-plugin should be based on the base neutron-tempest-plugin classes. Link https://bugs.launchpad.net/neutron/+bug/2163394 Assignee: Slawek Kaplonski Bug 2163415 - ProcessManager deletes pid file on reload when get_stop_command is set Link https://bugs.launchpad.net/neutron/+bug/2163415 Assignee: Miro Tomaska -- Miro Tomaska

1 0

[release] Release countdown for week R-6
by Thierry Carrez 14 Aug '26

14 Aug '26

Development Focus ----------------- Work on libraries should be wrapping up, in preparation for the various library-related deadlines coming up. Now is a good time to make decisions on deferring feature work to the next development cycle in order to be able to focus on finishing already-started feature work. General Information ------------------- We are now getting close to the end of the cycle, and will be gradually freezing feature work on the various deliverables that make up the OpenStack release. This coming week is the deadline for general libraries (except client libraries): their last feature release needs to happen before "Final release for non-client libraries" on 20 August, 2026. Only bugfixes releases will be allowed beyond this point. When requesting those library releases, you can also include the stable/2026.2 branching request with the review (as an example, see the "branches" section here: https://opendev.org/openstack/releases/src/branch/master/deliverables/pike/… In the next weeks we will have deadlines for: * Client libraries (think python-*client libraries), which need to have their last feature release before "Final release for client libraries" (27 August, 2026) * Deliverables following a cycle-with-rc model (that would be most services), which observe a Feature freeze on that same date, 27 August, 2026. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. As we are getting to the point of creating stable/2026.2 branches, this would be a good point for teams to review membership in their $project-stable-maint groups. Once the stable/2026.2 branches are cut for a repo, the ability to approve any necessary backports into those branches for 2026.2 "Hibiscus" will be limited to the members of that stable team. If there are any questions about stable policy or stable team membership, please reach out in the #openstack-stable channel. Upcoming Deadlines & Dates -------------------------- Non-client libraries final feature release: 20 August, 2026 (R-6 week) Client libraries final feature release: 27 August, 2026 (R-5 week) Hibiscus-3 milestone: 27 August, 2026 (R-5 week) Cycle Highlights Due: 3 September, 2026 (R-4 week) 2026.2 "Hibiscus" final release: 30 September, 2026 OpenInfra Summit Asia: 8-9 September, 2026 in Shanghai -- Thierry Carrez (ttx)

1 0

[kolla-ansible] Rocky 10 and Rocky 9 mixed environment: Ansible delegation issues
by Gian-Andrea Conzett 13 Aug '26

13 Aug '26

Hi all, I'm trying to add a Rocky 10 Compute Host to an existing Cluster with Rocky 9 nodes. Bootstrapping worked fine, but it seems that now, when trying to deploy the services onto the new host, all the tasks that are being delegated, fail as it uses the python interpreter of the new host (python3.12). In the inventory I have defined ansible_python_interpreter pointing to python3.9 on the Rocky 9 hosts and python3.12 for the new rocky node. However, if delegate tasks are run, like fact gathering, it fails as it tries to run it with python3.12 on a Rocky 9 host. I could fix the fact gathering by setting the ansible_python_interpreter variable inside the task and it worked, so it seems that the global ansible_python_interpreter, set in the inventory is not being used in those delegation tasks. Example: /opt/kolla-venv-2025.1/share/kolla-ansible/ansible/gather-facts.yml ansible_python_interpreter: "{{ hostvars[item]['ansible_python_interpreter'] }}" However, there are many more tasks that do delegate, so I do not wish to set this manually in every task. Most of the delegation happens when you run --limit, but I also noticed some other tasks failing that have "unavoidable" delegating, like fx. "- name: Waiting for nova-compute services to register themselves" in the nova-cell playbook. Do I miss to set some specific host options somewhere or do you have any tips or ideas on how to fix this? In the end our goal is to migrate an existing infrastructure to Rocky 10 and OpenStack 2026.1 (from 2025.1). As Rocky 9 is not supported on 2026.1, we first have to go to Rocky 10 before we can to the OpenStack upgrade. If you have any other device for OpenStack Upgrades including OS Upgrades, they would also be very welcome. Thank you, Giani

1 0

Everything Open Brisbane 2027 – Call for Sessions (Open Infrastructure, Cloud, Ops & Security)
by open infra 13 Aug '26

13 Aug '26

Hi, The Call for Sessions for Everything Open 2027 is now open. If you are working on something interesting around OpenStack, open infrastructure, cloud platforms, Linux, automation, operations, cybersecurity, open source, open AI, or open data, this could be a great opportunity to share your knowledge and experience with the wider community. Session formats include: - 45-minute talks - 25-minute short talks - 90-minute hands-on tutorials There are also specialist tracks covering Kernel Development, Operations & Security, and the Public Sector. 📅 Submission deadline: 6 September 2026 (AoE) Accepted speakers receive a complimentary Professional conference pass. More information and proposal submission: https://2027.everythingopen.au/news/call-for-sessions/ It would be great to see more OpenInfra contributors and OpenStack-related projects represented at Everything Open 2027. Please consider submitting a proposal or sharing this opportunity with others in the community. Best regards, -- Danishka Navin

1 0

[nova] GPU PCI passthru of NVIDIA H200 vs. PCI MMIO Hole handling by QEMU
by Christian Stelter 13 Aug '26

13 Aug '26

Hi! As someone put the first machine with 2x NVIDIA H200 in one of our racks it appears we are running into a known issue regarding how QEMU handles the PCI MMIO hole. Up to the NVIDIA H100 NVL we hadn't had any issues to pass those beast thru to an instance via pci passthru. But when trying to launch a H200 the same way we get the following error: [Fri Aug 7 18:04:42 2026] NVRM: This PCI I/O region assigned to your NVIDIA device is invalid: NVRM: BAR0 is 0M @ 0x0 (PCI:0000:05:00.0) [Fri Aug 7 18:04:42 2026] nvidia: probe of 0000:05:00.0 failed with error -1 [Fri Aug 7 18:04:42 2026] NVRM: The NVIDIA probe routine failed for 1 device(s). [Fri Aug 7 18:04:42 2026] NVRM: loading NVIDIA UNIX Open Kernel Module for x86_64 595.84 Release Build (dvs-builder@U22-I3-AM25-26-2) Wed Jun 10 21:06:37 UTC 2026 Our environment: OpenStack 2024.1 deployed by kolla-ansible on the hypervisor side and Ubuntu 24.04 on the guest side. As for Nova we set the cpu mode to host-passthrough on that host. We set the following properties for the images in addition to our regular images: hw_firmware_type='uefi' hw_machine_type='q35' We also tried to set the property hw:maxphysaddr_mode='passthrough' for the flavor, but that does not work for the included libvirt version (8.0.0). According to docs on openstack.org[1] libvirt v8.7.0+ and QEMU v2.7.0+ are required to support this. We also found an open issue/feature request[2] to allow the configuration of additional arguments for the firmware configuration (opt/ovmf/X-PciMmio64Mb). We tried to use a wrapper for calling qemu as an interim solution, but at least under 2024.1 the scheduling fails as soon as we add the maxphysaddr_mode. We have also a 2025.1 environment (with smaller GPUs) where the wrapper does work for starting the qemu with this extra argument and the maxphysaddr_mode set. BTW. We also tried to use libvirt webhooks to change the qemu commandline, but we failed. Would be nice to know whether this would also a valid approach to inject commandline args. So as I've seen (very few) posts in here of other users working with the H200. How do you deal with these beasts? :-D What do we miss? Is maxphysaddr_mode even needed or only if the total VRAM exceeds the 1TB which need to be addressed by the CPU? Any hints welcome. The all-knowning trash heap (Google) could not help yet. Also the usual friendly AI helpers (or trash pre-processors) failed. Best regards, cos [1] https://specs.openstack.org/openstack/nova-specs/specs/2024.1/implemented/l… [2] https://bugs.launchpad.net/nova/+bug/2160972

3 2

[neutron] Drivers meeting canceled Friday, August 14th
by Brian Haley 13 Aug '26

13 Aug '26

Hello Neutrinos: I am going to cancel tomorrow's Drivers meeting as there are no topics on the agenda. Just a reminder that our open spec list is at [0]. Have a nice weekend! -Brian [0] https://review.opendev.org/q/status:open+project:openstack/neutron-specs

1 0

[OSSN-0107] Ironic-Python-Agent: Container HardwareManager Security Model Misimplemented
by Jay Faulkner 13 Aug '26

13 Aug '26

Ironic Python Agent Container HardwareManager Security Misimplemented --- ### Summary ### Ironic Python Agent's ContainerHardwareManager plugin, shipped in 2025.2, was merged with a misimplemented security model. Ironic developers have pushed an updated version of this feature, including patches for Ironic and Ironic Python Agent, with properly implemented security controls. These patches will not be universally backported as they are not backwards-compatible. ### Affected Services / Software ### - ironic-python-agent: >=11.0.0, <12.0.1 ### Discussion ### The Ironic Python Agent uses plugins called HardwareManagers (HWMs) to expose new in-band steps for Ironic cleaning, servicing, or deployment. In the 2025.2 release, a Container HWM was added, giving operators who added a container runner (such as ``podman`` or ``docker``) to their ramdisk the option to download and execute containers as Ironic steps. This initial implementation had several security flaws and was implemented in such a way that we could not backport fixes without breaking existing deployments. These issues included ignoring the value of the ``[container]/allow_arbitrary_containers`` safety mechanism. ### Recommended Actions ### * Operators who are using OpenStack-supplied ramdisks are not vulnerable. These images, for the impacted releases, do not come with ``podman`` or ``docker`` installed, disabling the feature. You are secure without taking any action and will upgrade into the fixed version in OpenStack 2026.2 or later. * Operators who are using ramdisks with ``docker`` or ``podman`` installed, for example, via the ``ironic-python-agent-podman`` element in ``ironic-python-agent-builder`` but are not interested in the Container HWM should add ``deploy.container_clean_step`` and ``deploy.generic_container_step`` to ``[api]/disallow_service_steps``, ``[api]/disallow_clean_steps``, and ``[api]/disallow_deploy_steps``. This will disable the insecure code. Patches to add the ``disallow_*_steps`` options are available in OSSA-2026-025. * Operators currently using the Container HWM or who wish to use it should backport the patches from https://review.opendev.org/q/hashtag:%22container-hwm-patches%22 to the branch they are currently using. Then, evaluate your use case against the updated documentation https://docs.openstack.org/ironic/latest/admin/container-based-steps.html to ensure the changes were not breaking for any existing deployment. ### Credits ### - Tuomo Tanskanen, Ericsson Software Technology (Metal3.io Security Team) - Riccardo Pittau, Red Hat (Metal3.io Security Team) ### Contacts / References ### Authors: - Jay Faulkner, G-Research OSS This OSSN: https://wiki.openstack.org/wiki/OSSN/OSSN-0107 Original Launchpad bug: https://bugs.launchpad.net/ironic/+bug/2160143 Mailing List : [security-sig] tag on openstack-discuss(a)lists.openstack.org OpenStack Security : https://security.openstack.org/ CVE: none

1 0

[OSSA-2026-035] OpenStack Octavia: Unauthorized QoS policy deletion lock (CVE pending)
by Jeremy Stanley 13 Aug '26

13 Aug '26

==================================================== OSSA-2026-035: Unauthorized QoS policy deletion lock ==================================================== :Date: August 13, 2026 :CVE: PENDING Affects ~~~~~~~ - Octavia: <16.0.2, ==17.0.0, ==18.0.0 Description ~~~~~~~~~~~ Chen YuXiang with the Institute of Computing Technology, Chinese Academy of Sciences, reported a vulnerability in Octavia quality of service (QoS) policy authorization. By associating another project's QoS policy with an amphora, an authenticated user may prevent deletion of that policy. All Octavia deployments are affected. Patches ~~~~~~~ - https://review.opendev.org/1000296 (2025.1/epoxy) - https://review.opendev.org/1000295 (2025.2/flamingo) - https://review.opendev.org/1000094 (2026.1/gazpacho) - https://review.opendev.org/998935 (2026.2/hibiscus (development)) Credits ~~~~~~~ - Chen YuXiang from Institute of Computing Technology, Chinese Academy of Sciences References ~~~~~~~~~~ - https://launchpad.net/bugs/2161500 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=PENDING Notes ~~~~~ - A CVE assignment was requested from MITRE on 2026-08-05 (request 2079027); this advisory will be updated with the ID in an errata publication once it becomes available. -- Jeremy Stanley OpenStack Vulnerability Management Team https://security.openstack.org/vmt.html

1 0

[neutron-vpnaas] Setup with caracal/epoxy
by Niklas Schwarz 12 Aug '26

12 Aug '26

Hey there, I am currently testing/trying to setup openstack neutron with vpnaas using ovn. In our current setup we are using vpnass but with linux-bridge on openstack caracal on ubuntu 22.04 and we are required to migrate to ovn as soon as possible. During the setup of neutron with ovn in our dev-environment I am able to setup vpnaas but in our test scenarios the site connections stay in status down. Because I do not know if the error is in our dev-environment or something else I have run the tests against a dev-stack (release caracal - 2024.1) with the same results. While caracal is unmaintained I have also setup the dev-stack with epoxy, 2025.1 on ubuntu 22.04 as well as 24.04 with similar results (site connections stay in status PENDING_CREATE). The configuration used for the dev-stack is added down below [2]. But am I missing something obvious or is vpnaas with ovn not used/recommended any more with the current openstack releases. What might be the problem why the vpn is not working. If required I can provide the setup for the vpn but as already mentioned the setup works with neutron and vpnaas running on linuxbridge. What I can tell is that the additional transit networks as well as the routers are created as documented in the spec [1]. I can also see the new namespace created and the charon/strongswan process running in the namespace. Any idea what might be wrong? Best regards and thanks in advance Niklas [1] https://opendev.org/openstack/neutron-specs/src/branch/master/specs/xena/vp… [2] Dev-stack configuration single node ``` [[local|localrc]] ADMIN_PASSWORD="YOUR_PASSWORD_HERE" DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_HOST="192.168.0.100" HOST_IP="192.168.0.100" FIXED_RANGE="10.4.128.0/20" FLOATING_RANGE="192.168.0.128/25" DEVSTACK_RELEASE="stable/2025.1" // or unstable/2024.1 LOGFILE=/opt/stack/logs/stack.sh.log LOGDAYS=7 LOG_COLOR=False Q_AGENT=ovn Q_ML2_PLUGIN_MECHANISM_DRIVERS=ovn,logger Q_ML2_PLUGIN_TYPE_DRIVERS=local,flat,vlan,geneve Q_ML2_TENANT_NETWORK_TYPE="geneve" enable_service ovn-northd enable_service ovn-controller enable_service q-ovn-metadata-agent enable_service q-ovn-agent enable_service q-ovn-vpn-agent enable_service q-svc disable_service q-agt disable_service q-l3 disable_service q-dhcp disable_service q-meta enable_plugin neutron https://opendev.org/openstack/neutron $DEVSTACK_RELEASE disable_service q-trunk enable_service q-dns enable_service q-port-forwarding disable_service q-qos enable_service neutron-segments enable_service q-log disable_service n-net enable_plugin neutron-vpnaas https://opendev.org/openstack/neutron-vpnaas $DEVSTACK_RELEASE enable_plugin neutron-vpnaas-dashboard https://opendev.org/openstack/neutron-vpnaas-dashboard $DEVSTACK_RELEASE IPSEC_PACKAGE="strongswan" enable_service horizon OVN_BUILD_MODULES=True ENABLE_CHASSIS_AS_GW=True Q_USE_SECGROUP=True PUBLIC_NETWORK_NAME=public PUBLIC_NETWORK_GATEWAY="192.168.0.254" PUBLIC_PHYSICAL_NETWORK=public PUBLIC_SUBNET_NAME=public-subnet Q_FLOATING_ALLOCATION_POOL=start=192.168.0.240,end=192.168.0.250 FLOATING_RANGE="192.168.0.0/24" IPV4_ADDRS_SAFE_TO_USE="10.0.0.0/24" PUBLIC_INTERFACE=enp4s0 Q_USE_PROVIDERNET_FOR_PUBLIC=True OVS_PHYSICAL_BRIDGE=br-ex PUBLIC_BRIDGE=br-ex OVS_BRIDGE_MAPPINGS=public:br-ex [[post-config|$NOVA_CONF]] [scheduler] discover_hosts_in_cells_interval = 2 ```

1 0