October 2024 - openstack-discuss - lists.openstack.org

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 19 May '25

19 May '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 9

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

[oslo] python-consul2 is unmaintained
by Takashi Kajinami 21 Apr '25

21 Apr '25

Hello, I recently noticed that python-consul2 library[1], which is used by the consul coordination driver in tooz, hasn't been updated for 3+ years. Looking at issues in the project, there is an indication that the library is not compatible with ACL API in consul >= 1.11[2]. There is another fork created[3] and this includes fixes to support new API endpoints. Although the driver in tooz does not directly use the feature, I think it's better to discuss our plan to mitigate risks caused by the unmaintained library. As far as I can think of, there are three options now 1. Deprecate consul driver and remove it 2. Keep using python-consul2 now. At least basic functionality is proven to work by our functional tests (Note that the tests do not cover all configuration patterns) 3. Replace python-consul2 by py-consul Before we start the overall discussions I'd like to ask a few questions to the wider audience, as our inputs. - Is anyone using the consul coordination driver now ? - Does anyone have experience with python-consul2 and/or py-consul ? Note that this discussion affects the http messaging driver[4] currently proposed, because it also requires client library to communicate with Consul. Actually I noticed the situation when I noticed that the requirement check job is failing because the library is not part of global requirements list... Thank you, Takashi Kajinami [1] https://github.com/poppyred/python-consul2 [2] https://github.com/poppyred/python-consul2/issues/42 [3] https://github.com/criteo-forks/py-consul [4] https://review.opendev.org/c/openstack/oslo.messaging/+/912499 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 3

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

GPU PCI passthrough woes.
by Andy Speagle 09 Apr '25

09 Apr '25

Hey Folks, I could use a little assistance getting GPU passthrough working. I had this working already for one flavor of nvidia gpu... and I've added some hosts with a much newer gpu... I've updated the pci_alias and pci_passthrough variables and those seem to be getting set properly in nova.conf passthrough_whitelist = [{"vendor_id":"10de", "product_id":"1b06"},{"vendor_id":"10de", "product_id":"26b9"}] alias = {"name": "gpu", "product_id": "1b06", "vendor_id": "10de"} alias = {"name": "gpu-l40s", "product_id": "26b9", "vendor_id": "10de"} I believe I have all of the iommu stuff configured and have the pci- stub module entries... dmesg output shows that the GPUs are being claimed by the stub module. $ openstack flavor show t1.small_gpu_l40s +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 0 | | id | af70c94e-0026-4a39-bc1e-dfb93b286a54 | | name | t1.small_gpu_l40s | | os-flavor-access:is_public | True | | properties | pci_passthrough:alias='gpu-l40s:1' | | ram | 2048 | | rxtx_factor | 1.0 | | swap | 0 | | vcpus | 1 | +----------------------------+--------------------------------------+ Yet... I can't seem to get an instance to run using that new flavor... keeps complaining about there not being enough hosts available. Fault: code: 500 created: 2024-10-30T02:04:33Z message: "No valid host was found. There are not enough hosts available." details: | Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 1580, in schedule_and_build_instances host_lists = self._schedule_instances(context, request_specs[0], File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 940, in _schedule_instances host_lists = self.query_client.select_destinations( File "/usr/lib/python3/dist- packages/nova/scheduler/client/query.py", line 41, in select_destinations return self.scheduler_rpcapi.select_destinations(context, spec_obj, File "/usr/lib/python3/dist-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/usr/lib/python3/dist- packages/oslo_messaging/rpc/client.py", line 189, in call result = self.transport._send( File "/usr/lib/python3/dist- packages/oslo_messaging/transport.py", line 123, in _send return self._driver.send(target, ctxt, message, File "/usr/lib/python3/dist- packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send return self._send(target, ctxt, message, wait_for_reply, timeout, File "/usr/lib/python3/dist- packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send raise result nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. Traceback (most recent call last): File "/usr/lib/python3/dist- packages/oslo_messaging/rpc/server.py", line 241, in inner return func(*args, **kwargs) File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 223, in select_destinations selections = self._select_destinations( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 250, in _select_destinations selections = self._schedule( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 416, in _schedule self._ensure_sufficient_hosts( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 455, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason) nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. Any clues on how/where to dig into this more to see what might be missing? Thanks. -- Andy Speagle

3 2

[all][tc][ptl][qa] Migrating upstream CI/CD jobs to Ubuntu Noble (24.04)
by Ghanshyam Mann 14 Mar '25

14 Mar '25

Hello Everyone, As per the 2025.2 cycle testing runtime, we need to run upstream CI/CD jobs on Ubuntu Noble (24.04) [1]. For better tracking and speeding up this work, it is the community-wide goal for this cycle[2]. This is the etherpad which has the migration plan and will be used for the tracking: - https://etherpad.opendev.org/p/migrate-to-noble I have prepared the change(WIP) for base jobs migration on Noble (openstack tox jobs, devstack, and tempest) which will migrate most of the projects jobs. Tempest change[3] include dependency on openstack tox, devstack base jobs changes so testing project gate using the the tempest change will test the complete set. Refer to the etherpad for the complete migration plan/steps. * Deadline: Nov 29th 2024 (R18) is the deadline to test/fix the projects jobs and to merge the base jobs running on Noble (openstck tox, devstack, and tempest). [1] https://governance.openstack.org/tc/reference/runtimes/2025.1.html [2] https://governance.openstack.org/tc/goals/selected/migrate-ci-jobs-to-ubunt… [3] https://review.opendev.org/c/openstack/tempest/+/932156 -gmann

2 9

[kolla] [nova] API flags ignored in Wallaby
by Albert Braden 12 Mar '25

12 Mar '25

After rebuilding our Train clusters on Wallaby we are seeing some API issues. It appears that flags that work in Train are still documented in Wallaby, but are ignored. Do we have something setup wrong, or are we encountering bugs? The first thing is the “locked” list: Train: openstack server list --all-projects --locked --os-compute-api-version 2.73 <empty list because nothing is locked> openstack server list --all-projects --unlocked --os-compute-api-version 2.73 <list of every VM in the cluster> Wallaby (locked flag is ignored): openstack server list --all-projects --locked --os-compute-api-version 2.73 <list of every VM in the cluster> openstack server list --all-projects --unlocked --os-compute-api-version 2.73 <list of every VM in the cluster> The second problem is the hypervisor list. We can’t get “long” output in Wallaby: Train: openstack hypervisor list --long | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | vCPUs Used | vCPUs | Memory MB Used | Memory MB | Wallaby (long flag is ignored): openstack hypervisor list --long | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |

6 14

[tc][all][monasca ] Retiring the Monasca project
by Ghanshyam Mann 16 Feb '25

16 Feb '25

Hello everyone, As you might know, the Monasca project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Monasca retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

6 17

[ironic]About the OS supported by Diskimage-builder
by Junya Noguchi (Fujitsu) 03 Feb '25

03 Feb '25

To whom it may concern. My name is Junya Noguchi. I have some questions about the supported OS for Diskimage-builder operation. Please respond. References https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html Does it mean that the OS that says "work but receive only limited testing and support" works but is not supported? References https://docs.openstack.org/diskimage-builder/latest/ This document describes RHEL and other operating systems that are not supported in the previous question. Are these operating systems supported? Also, are there any plans to support more operating systems in the future? Best regards, Junya Noguchi

5 7

[glance][ops] proposal to remove the glance-scrubber
by Brian Rosmaita 19 Jan '25

19 Jan '25

Hello Glance users, The Glance project team wants to determine whether anyone is using the glance-scrubber utility [0]. If not, we are thinking of deprecating it this cycle to be removed in the appropriate future cycle. If you have not previously heard of the glance-scrubber, that's most likely because it's only useful for particular deployment configurations of which yours is not one. See the doc [0] for details if you are curious. Our suspicion is that the glance-scrubber may have outlived its usefulness for modern OpenStack configurations. (The last time I remember it being discussed at a PTG was during the snowstorm in Dublin, which I believe was the Rocky development cycle). On the other hand, the team also has some ideas for improving the scrubber, but there's no point doing that if it's not actually being used in the first place. So if removal of the glance-scrubber would negatively affect you, please respond to this email so we can get some data points. Thanks! [0] https://docs.openstack.org/glance/latest/cli/glancescrubber.html

2 2

[i18n][infra][horizon] Zanata translation job failures after Python 3.8 deprecation
by Ian Y. Choi 19 Jan '25

19 Jan '25

Hi all, During last week's PTG, the I18n SIG identified Zanata translation job failures for Horizon (and likely related plugins). The root cause appears to be Python 3.8 deprecation on Horizon [1]. Failed jobs have been observed on Zuul [2], and translation proposals were made on Gerrit [3]. Since translation sync on Horizon stable branches is functioning correctly, the error message seems to be related to Python 3.8 deprecation: ERROR: Cannot install -r doc/requirements.txt (line 3) due to conflicting package version dependencies. To address this, I'd like to raise two points: 1/ To resolve the issue, would you support updating translation sync jobs with Bionic-based Zuul jobs by either upgrading to Jammy or modifying the Bionic image to support Python >3.8? 2/ At least me from I18n SIG acknowledges that recognizing these translation job errors was delayed. Moving forward, would it be beneficial to implement periodic (e.g., monthly) reporting on translation job successes and failures? Thank you, /Ian [1] https://docs.openstack.org/releasenotes/horizon/2024.2.html#upgrade-notes [2] https://zuul.opendev.org/t/openstack/builds?job_name=propose-translation-up… [3] https://review.opendev.org/q/owner:proposal-bot+project:openstack/horizon+b…

2 2

[cinder][kubernetes] pvc stuck in reserved state
by Eugen Block 10 Jan '25

10 Jan '25

Hi, we have a couple of k8s clusters in our openstack environment (Victoria) and are facing some problems when pods get moved to different worker nodes. The PVCs get stuck in "reserved" state, and they only get attached after resetting the volumes to "available" state. First I thought it was due to the missing multiattach volume type which seems to work fine in our customer's cluster. I also found the "duplicate connector" error messages in the nova-compute.log: > Unable to update attachment.(Invalid volume: duplicate connectors > detected on volume But after adding the multiattach volume type and retyping the affected volumes, the PVCs still get stuck in "reserved" state while I don't see any error messages on the hypervisors anymore. So it seems that something has changed/improved, but now I don't have a trace where to look next. Could this be a cinder-csi driver issue or something? Our cinder backend is ceph (Pacific). Not sure which information could help so please ask for anything I can provide. Any pointers are highly appreciated! Thanks! Eugen

1 2

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 23 Dec '24

23 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla-ansible][nova] Refresh cell cache in nova scheduler
by Satish Patel 04 Dec '24

04 Dec '24

Folks, What is wrong here, I do have the following hostname resolvable from all the nodes even if they are in the /etc/hosts file, Why is the following task saying UNREACHABLE and from where its trying to reach ? PLAY [Refresh nova scheduler cell cache] ***************************************************************************************************************************************************** TASK [nova : Refresh cell cache in nova scheduler] ******************************************************************************************************************************************* fatal: [os2-bos-ctrl01]: UNREACHABLE! => {"changed": false, "msg": "Data could not be sent to remote host \"os2-bos-ctrl01\". Make sure this host can be reached over ssh: ", "unreachable": true} fatal: [os2-bos-ctrl02]: UNREACHABLE! => {"changed": false, "msg": "Data could not be sent to remote host \"os2-bos-ctrl02\". Make sure this host can be reached over ssh: ", "unreachable": true} fatal: [os2-bos-ctrl03]: UNREACHABLE! => {"changed": false, "msg": "Data could not be sent to remote host \"os2-bos-ctrl03\". Make sure this host can be reached over ssh: ", "unreachable": true}

4 5

[cinder] What is the "official" name for Cinder in the service catalog?
by Stephen Finucane 29 Nov '24

29 Nov '24

o/ I've proposed a series of patches to "clean up" the service catalog that DevStack creates in a deployment. Currently, DevStack creates two endpoints for Cinder: one called 'cinderv3' of type 'volumev3' and another called 'cinder' of type 'block-storage'. For example: { "services": [ { "name": "cinderv3", "description": "Cinder Volume Service V3", "id": "65460706a9864e71926f13042f526aeb", "type": "volumev3", "enabled": true, "links": { "self": "http://10.0.108.50/identity/v3/services/65460706a9864e71926f13042f526aeb" } }, { "name": "cinder", "description": "Cinder Volume Service", "id": "cd406a0e296d47dca1e481e992851bb2", "type": "block-storage", "enabled": true, "links": { "self": "http://10.0.108.50/identity/v3/services/cd406a0e296d47dca1e481e992851bb2" } }, // ... ], "links": { "next": null, "self": "http://10.0.108.50/identity/v3/services", "previous": null } } They both ultimately point to the same API, the Cinder v3 API, meaning one of these endpoints is redundant. My assumption was 'block-storage' was the correct one, since that's the official service type [1], and that 'cinder' was the appropriate name since it mirrors what we use for other services (nova, neutron, keystone, glance, ...). However, tkajinam raised the point [2] that the Install Guide currently recommends use of the 'volumev3' type and that projects like Nova and Glance have an '[cinder] catalog_info' that defaults to 'volumev3'. If so, is 'block-storage' really the official service type, or is it 'volumev3'. If it is, should we be changing the Install Guide and the various config option defaults? Cheers, Stephen [1] https://specs.openstack.org/openstack/service-types-authority/

6 18

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

[horizon][ceph] object storage container storage policy missing
by Satish Patel 24 Nov '24

24 Nov '24

Folks, Openstack Release: Zed Recently I have integrated ceph RGW service with openstack/keystone and testing with command line tool "openstack container create foo1" and it works but when I go to Horizon UI I am seeing it doesn't have storage policy dropdown list and because of that I am not able to create container or upload file from Horizon GUI https://ibb.co/KhYW6Rd It works with commands like but not horizon. $ openstack container create foo1 +---------------------------------------+-----------+------------------------------------------------+ | account | container | x-trans-id | +---------------------------------------+-----------+------------------------------------------------+ | AUTH_d4d82849885e4126aef402c408ad2c5c | foo1 | tx00000c057454841b13497-0065b558b4-42ff2eb-ahm | +---------------------------------------+-----------+------------------------------------------------+ $ openstack container list +------+ | Name | +------+ | foo1 | +------+

2 2

[ironic][networking] Interest in starting a baremetal networking working group?
by Julia Kreger 20 Nov '24

20 Nov '24

Greetings folks, During the PTG this past week, the Ironic community including developers and operators reached a conclusion that we need to start back up a networking focused working group to try and move the current state of capabilities forward when bare metal machines are involved. Specifically, management of physical switch connections to bare metal *and* programming DPU/IPU devices for networking attachments. As we're all quite aware, it is a complex topic with many different threads. Given resurgent interest and a mutual desire to find some better solutions, it just makes sense to have regular discussions. That being said we need to identify an ideal day of week, frequency, and preferred time. I'd recommend sometime between 1500 UTC and 1800 UTC. Preferably a Tuesday or Wednesday. Would every two weeks work? Or would folks prefer to meet weekly to try and build the most possible velocity through regular interaction? Thoughts? -Julia

10 16

[ironic] Having issue with deploying ISO image to node
by James Leong 17 Nov '24

17 Nov '24

Hi all, I am trying to play around with ironic, and I am having an issue with deploying an Ubuntu 20.04 ISO image onto my bare metal node. Since I am quite new to bare metal, I am not entirely sure what went wrong as I couldn't find any error messages shown on my log file (ironic-conductor, ironic-api, ironic_neutron_agent, and nova-compute-ironic). This might be my guess; the ironic inspection is hitting timeout, which is causing the deployment to timeout. However, I have no idea why the inspection is failing at timeout. Any help would be appreciated. Best, James

6 7

[horizon] SCSS compilers
by Radomir Dopieralski 15 Nov '24

15 Nov '24

The openstack dashboard uses scss for styling, and so far we have used a python scss library for compiling those files to css for use in the browser. However, that library (pyscss) had in recent years some problems, and is not developed or supported by the authors anymore. So it's time to switch to something with a better support. The django-compressor plugin that we use for handling the compilation of files allows us to use any command-line tool to do the compilation, so we can switch to an scss compiler that is supported and used by the wider community. However, there are several options we can choose from: sassc python-libsass pysassc ruby-sass ruby-sassc node-node-sass Each of those would introduce a non-python dependency on the dashboard. The question for the packagers and distro maintainer is which of those tools would be preferable in terms of support, packaging, and maintenance for the long term? Do you have opinions? Maybe there is another tool that would be even better that I didn't find?

2 3

[OpenStack-Ansible] Major Upgrade from Yoga to Zed/Antelope
by Roby G 12 Nov '24

12 Nov '24

Hello everyone, Current environment: - Ubuntu 20.04 - OpenStack-Ansible Yoga - no LXC, services are installed bare metal - `install_method` is `distro` My goal is to upgrade this environment to Zed, or skip Zed and upgrade directly to Antelope, since Zed is also in Unmaintained state. What would you recommend? Which order should I follow - upgrade operating system or OpenStack first? The decision is a bit difficult as all 3 releases (Yoga, Zed and Antelope) support different Ubuntu versions in install_method distro and not the same one.

2 1

[kolla-ansible]: guidance needed for neutron setup
by turkey automation 12 Nov '24

12 Nov '24

Hey folks, I am new to Openstack and want to try it on KVM setup with 1 controller and 2 compute VMs. In my first attempt, I was able to install Openstack using kolla-ansible(2024.1 version) and also run-initonce completed without any issues. Now, I want to create instances that can access my back-bone i.e. KVM virtual network(192.168.121.x/24) but either floating IP concept or extending my KVM network to openstack(I don't know if it's feasible!) Some info about my infra: a. Controller 1: Interface eth0 has 192.168.121.5 IP address. eth1 has no IP. b. compute 1: Interface eth0: 192.168.121.10, eth1 has no IP. c. compute 2: interface eth0: 192.168.121.15, eth1 has no IP. What configuration do I need to enable in globals.yml and what all I need to do post deployment of openstack to ensure I have virtual instances on openstack that have IPs from 192.168.121.x network(dhcp can be controlled by KVM network?) and are accessible from other KVM based VMs that I created outside of openstack. I am a learner to please be elaborate if possible. Thanks in advance to the community. - Best regards!

3 6

Upstream Oslo IRC Meetings Are Back.
by Daniel Bengtsson 12 Nov '24

12 Nov '24

Hello everyone, I hope you are all doing well! I sent an email a while ago to reestablish the Upstream Oslo IRC meetings. With the removal of Eventlet, this makes even more sense. I have received two positive responses, and we also mentioned it during the PTG last week. I propose to hold these meetings every other Monday, starting today, at 4:00 PM UTC. Does this work for everyone? Thank you for your feedback! -- Daniel Bengtsson Software engineer

6 18

#eventlet-removal - OFTC IRC meeting time slot poll - Express your choice!
by Herve Beraud 12 Nov '24

12 Nov '24

Hey Stackers, We propose to schedule a one hour OFTC IRC meeting (#opentack-eventlet-removal) each first and third week of each month. Let's find together the right UTC time slot for our one hour OFTC IRC meeting. Once the vote is done, meetings will be scheduled each first and third <selected time slot in the poll> of each month. Please, vote here: https://framadate.org/rUghO3j6ZkRhKD7m Votes will be considered as closed by Nov. 8th First meeting will be forced to 4pm UTC on Tuesday Nov 5th and then moved to the selected time slot for all the following meetings. See the proposed time slots in the poll link above. Join us on the #opentack-eventlet-removal OFTC IRC channel. Thanks in advance for your votes -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 2

OpenStack Cinder - Volume-ID in cinder DB
by Harsh Ailani 12 Nov '24

12 Nov '24

Hello contributors, I am working on clone operation for a volumegroup and in that the volume ID auto-generated by OpenStack is not the one with which the clone volumes are being created on my IBM storage backend. This is as per design that IBM Storage backend will take the source-volume ID in the source-volumegroup and add a suffix to it while cloning. Now since there is a mismatch in the IDs between cinder and storage backend, because of which I am unable to perform any volume operations on this cloned volume. So, I was wondering if I can change the volume ID in cinder DB? I know that it’s a unique key and there is a limit of 36 characters for this field in cinder DB, but since I cannot change the volume in storage backend, I wanted to check if there is a way I can alter the volume ID in cinder DB? Kindly share your thoughts/suggestions on this matter. Thanks and Regards, Harsh Ailani

2 4

[Cinder][Ceph] Volumetria errada no cluster ceph
by lfsilva＠binario.cloud 07 Nov '24

07 Nov '24

Hello, I'm experiencing an issue with the volume usage of the Ceph cluster, which is currently used in OpenStack for volumes. I am working with Ceph version Octopus (15.2.17). When I run the ceph df command, I get this output: --- RAW STORAGE --- CLASS SIZE AVAIL USED RAW USED %RAW USED hdd 120 TiB 33 TiB 87 TiB 87 TiB 72.72 TOTAL 120 TiB 33 TiB 87 TiB 87 TiB 72.72 --- POOLS --- POOL ID PGS STORED OBJECTS USED %USED MAX AVAIL device_health_metrics 1 32 69 MiB 101 137 MiB 0 11 TiB images 2 256 1.3 TiB 166.90k 2.5 TiB 10.72 11 TiB vms 3 64 574 KiB 21 2.5 MiB 0 11 TiB volumes 4 2048 41 TiB 3.94M 82 TiB 79.42 11 TiB backups 5 1024 407 GiB 111.85k 818 GiB 3.63 11 TiB And when I run the rbd du -p volumes command, I get this total: NAME PROVISIONED USED <TOTAL> 14 TiB 8.3 TiB This "volumes" pool is currently set to replica 2, and mirroring is not enabled. I have checked for any locked or deleted snapshots but found none. I also ran a ceph osd pool deep-scrub volumes, but it didn't resolve the issue. Has anyone encountered this problem before? Could someone provide assistance?

3 21

Re: GPU PCI passthrough woes.
by Andy Speagle 04 Nov '24

04 Nov '24

Aye, that seems to have done it. Strange that this wasn't necessary for my old 1080 cards. Thanks! On Wed, 2024-10-30 at 08:12 +0000, Alexander Dibbo - STFC UKRI wrote: > ⚠️CAUTION⚠️ > This email originated outside of Toyon's network. Do not download > attachments or click links unless you know the content is safe. > > Hi Andy, > > We've seen many issues with pci passthrough and I suspect that the > l40s will have a seen similar issues with the a100s > > Here is an alias string that works for our a100s: > `alias = {"vendor_id": "10de", "product_id": "20f1", "device_type": > "type-PF", "name": "nvidia-tesla-a100-pcie-vga"}` > > I believe it is the `device_type` that will be your issue > > Regards > > Alexander Dibbo – Cloud Architect / Cloud Operations Group Leader > For STFC Cloud Documentation visit > https://urldefense.us/v3/__https://stfc.atlassian.net/wiki/spaces/CLOUDKB/o… > To raise a support ticket with the cloud team please email > cloud-support(a)stfc.ac.uk > To receive notifications about the service please subscribe to our > mailing list at: > https://urldefense.us/v3/__https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=S… > To receive fast notifications or to discuss usage of the cloud please > join our Slack: > https://urldefense.us/v3/__https://stfc-cloud.slack.com/__;!!A4KLn-w15EM4BR… > > -----Original Message----- > From: Andy Speagle <aspeagle(a)toyon.com> > Sent: 30 October 2024 02:15 > To: openstack-discuss(a)lists.openstack.org > Subject: GPU PCI passthrough woes. > > Hey Folks, > > I could use a little assistance getting GPU passthrough working. I > had > this working already for one flavor of nvidia gpu... and I've added > some hosts with a much newer gpu... > > I've updated the pci_alias and pci_passthrough variables and those > seem > to be getting set properly in nova.conf > > passthrough_whitelist = [{"vendor_id":"10de", > "product_id":"1b06"},{"vendor_id":"10de", "product_id":"26b9"}] > > alias = {"name": "gpu", "product_id": "1b06", "vendor_id": "10de"} > alias = {"name": "gpu-l40s", "product_id": "26b9", "vendor_id": > "10de"} > > I believe I have all of the iommu stuff configured and have the pci- > stub module entries... dmesg output shows that the GPUs are being > claimed by the stub module. > > $ openstack flavor show t1.small_gpu_l40s > +----------------------------+--------------------------------------+ > > Field | Value | > +----------------------------+--------------------------------------+ > > OS-FLV-DISABLED:disabled | False | > > OS-FLV-EXT-DATA:ephemeral | 0 | > > access_project_ids | None | > > description | None | > > disk | 0 | > > id | af70c94e-0026-4a39-bc1e-dfb93b286a54 | > > name | t1.small_gpu_l40s | > > os-flavor-access:is_public | True | > > properties | pci_passthrough:alias='gpu-l40s:1' | > > ram | 2048 | > > rxtx_factor | 1.0 | > > swap | 0 | > > vcpus | 1 | > +----------------------------+--------------------------------------+ > > Yet... I can't seem to get an instance to run using that new > flavor... > keeps complaining about there not being enough hosts available. > > Fault: > code: 500 > created: 2024-10-30T02:04:33Z > message: "No valid host was found. There are not enough hosts > available." > details: | > Traceback (most recent call last): > File "/usr/lib/python3/dist- > packages/nova/conductor/manager.py", > line 1580, in schedule_and_build_instances > host_lists = self._schedule_instances(context, > request_specs[0], > File "/usr/lib/python3/dist- > packages/nova/conductor/manager.py", > line 940, in _schedule_instances > host_lists = self.query_client.select_destinations( > File "/usr/lib/python3/dist- > packages/nova/scheduler/client/query.py", line 41, in > select_destinations > return self.scheduler_rpcapi.select_destinations(context, > spec_obj, > File "/usr/lib/python3/dist-packages/nova/scheduler/rpcapi.py", > line 160, in select_destinations > return cctxt.call(ctxt, 'select_destinations', **msg_args) > File "/usr/lib/python3/dist- > packages/oslo_messaging/rpc/client.py", line 189, in call > result = self.transport._send( > File "/usr/lib/python3/dist- > packages/oslo_messaging/transport.py", line 123, in _send > return self._driver.send(target, ctxt, message, > File "/usr/lib/python3/dist- > packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send > return self._send(target, ctxt, message, wait_for_reply, > timeout, > File "/usr/lib/python3/dist- > packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send > raise result > nova.exception_Remote.NoValidHost_Remote: No valid host was > found. > There are not enough hosts available. > > Traceback (most recent call last): > File "/usr/lib/python3/dist- > packages/oslo_messaging/rpc/server.py", line 241, in inner > return func(*args, **kwargs) > > File "/usr/lib/python3/dist- > packages/nova/scheduler/manager.py", > line 223, in select_destinations > selections = self._select_destinations( > > File "/usr/lib/python3/dist- > packages/nova/scheduler/manager.py", > line 250, in _select_destinations > selections = self._schedule( > > File "/usr/lib/python3/dist- > packages/nova/scheduler/manager.py", > line 416, in _schedule > self._ensure_sufficient_hosts( > > File "/usr/lib/python3/dist- > packages/nova/scheduler/manager.py", > line 455, in _ensure_sufficient_hosts > raise exception.NoValidHost(reason=reason) > > nova.exception.NoValidHost: No valid host was found. There are > not > enough hosts available. > > Any clues on how/where to dig into this more to see what might be > missing? Thanks. > > -- > Andy Speagle > -- Andy Speagle Sr. Site Reliability Engineer Toyon Research Corporation 316.617.2431

3 2

Is HPET recommended for x86 instance?
by Jiatong Shen 04 Nov '24

04 Nov '24

Hello Community Experts, I found that to enable the HPET timer for x86 guests, the operator should set hw_time_hpet on the given image. By default, HPET is disabled. My question is what are the pros and cons for using HPET? Thanks. -- Best Regards, Jiatong Shen

2 3

[neutron][ovn-octavia-provider][octavia][ovn] Octavia OVN DBs sync cmd
by Rico Lin 04 Nov '24

04 Nov '24

Hi all I would like to raise attention to the patch set that introduces Octavia OVN DBs sync cmd. patches: https://review.opendev.org/q/topic:%22add-ovn-db-sync-utils%22 With command `octavia-ovn-db-sync-util {--debug}` It should allows us to do the OVN DB sync similar to what `neutron-ovn-db-sync-util` does. And thanks to Fernando Royo, the patches are small and earlier to review in focused functions. Would be great if we could have some more reviews for it. *Rico Lin*

2 1

[devstack][nova][cinder][ironic][glance][swift][neutron][all] Deprecating/removing non-uWSGI deployment mechanisms?
by Stephen Finucane 01 Nov '24

01 Nov '24

o/ tl;dr: I'd like to remove the ability to deploy services under mod_wsgi from DevStack. I'd also like to remove the ability to deploy services under eventlet, where this feature is not currently being tested, and deprecate it where it is. I took a foray through DevStack last week and realised that, depending on the service in question, there are currently 3 possible ways an API can be deployed by DevStack: * Eventlet server * Apache + uWSGI * Apache + mod_wsgi In addition, there are a number of overlapping config options that could be set to modify which of these was in use: * ENABLE_HTTPD_MOD_WSGI_SERVICES * WSGI_MODE * Service-specific options like NOVA_USE_MOD_WSGI, CINDER_USE_MOD_WSGI etc. Potential confusion aside, there are two issues I see with this. Firstly, and most obviously, we are working on eventlet removal, which will necessarily involve eventlet server removal [1]. In addition, there is a goal [2][3] to remove use of '.wsgi' scripts in favour of Python module paths. Quite a few services have migrated to this already, and those that haven't are going to need to do so by time pip 25.0 comes out (since that removes the deprecated legacy editable install flow [4] meaning these files will not be generated by pbr and DevStack deployments will start failing...you have been warned). However, mod_wsgi need .wsgi scripts. I didn't realise we cared about mod_wsgi in DevStack still before last week, so to complete the wsgi script -> module migration for services that can be deployed with mod_wsgi (e.g. Keystone), we're going to have to either (a) package a .wsgi script in DevStack or (b) remove the ability to deploy with mod_wsgi. My immediate thought on this was to simply remove all non-uWSGI deployment paths immediately and deprecate the config options for removal to force DevStack plugins to adapt. However, some services still want to test eventlet server for their API while they stabilise support for standard WSGI-based deployments (neutron), while others explicitly rely on evenlet to generate a single standalone binary (ironic, glance, ...). That means we can't just go remove everything now and must instead take a phased approach. I would therefore like to propose we do the following: * Remove the ability to deploy services under mod_wsgi from DevStack and deprecate/remove the WSGI_MODE flag. To be clear, other deployment tools can continue to use mod_wsgi but DevStack won't. * Entirely remove the ability to deploy under eventlet from services that are not currently testing this. This includes Nova and Cinder. * Invert the default for services that are testing eventlet, so that they test with uWSGI by default now and jobs will need to opt-in to eventlet rather than opt-in to uWSGI. I think this just affects Neutron. * Optionally, and likely only one Neutron is no longer testing eventlet (next cycle?), deprecate ENABLE_HTTPD_MOD_WSGI_SERVICES and {SERVICE}_USE_MOD_WSGI config options entirely and replace it with {SERVICE}_STANDALONE options like Glance has. This will affect Ironic and maybe more. I have proposed a series that will do much of this. The WIP patches indicate the things that I am less clear about, such as Swift's readiness for deploying under uWSGI. I would be grateful if folks from the individual service teams could take a look and let me know what they think: https://review.opendev.org/c/openstack/devstack/+/932193/1 Cheers, Stephen [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [2] https://review.opendev.org/c/openstack/governance/+/902807 [3] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [4] https://ichard26.github.io/blog/2024/08/whats-new-in-pip-24.2/

6 10

[openstack helm] [Cinder] [Ceph] keyring not getting decode when doing cinder-storage-init
by daniel890723＠gmail.com 31 Oct '24

31 Oct '24

Hello today i was testing to deploy openstack with helm in my kubernetes cluster and i encounter an error when trying to deploy cinder Error: UPGRADE FAILED: post-upgrade hooks failed: 1 error occurred: * job cinder-storage-init failed: BackoffLimitExceeded when i print the log of the cinder-storage-init pod it shows 2024-10-30T03:22:14.303+0000 7efc1353d640 -1 auth: failed to load /etc/ceph/ceph.client.admin.keyring: (5) Input/output error 2024-10-30T03:22:14.303+0000 7efc1353d640 -1 auth: error parsing file /etc/ceph/ceph.client.admin.keyring: error setting modifier for [client.admin] type=key val=: Malformed input [buffer:3] 2024-10-30T03:22:14.303+0000 7efc1353d640 -1 auth: failed to load /etc/ceph/ceph.client.admin.keyring: (5) Input/output error 2024-10-30T03:22:14.303+0000 7efc1353d640 -1 monclient: keyring not found [errno 5] RADOS I/O error (error connecting to the cluster) and the keyring secret is use looks like this: apiVersion: v1 data: key: AQDtmCFnh3STCBAAGKQIyIGuZjnmX3L4mPCQXw== kind: Secret metadata: name: pvc-ceph-client-key namespace: openstack after checking secret and the rook-ceph cluster works fine i change the template and makes the pod not executing the command so i can look inside the pod and it's keyrings the keyring looks like this: [client.admin] key = �!g�tȁ�f9�_r��_ so apparently the keyring secert or when it mount into the pod got some problem and change the key into the normal one fix the problem is there a reason why it cause this ? because the secret and ceph-cluster are all works fine

2 4

[manila] 2025.1 Epoxy PTG summary
by Carlos Silva 31 Oct '24

31 Oct '24

Hello everyone! Thank you for the great participation at the PTG last week. We've had great discussions and a good turnout. The recordings for the sessions are available on YouTube [0]. If you would like to check on the notes, please take a look at the PTG etherpad [1]. *2024.2 Dalmatian Retrospective* ========================== - New core reviewers in the manila group were impactful in reviews, we should continue actively working on maintaining/growing the core reviewer team. - We had the mid-cycle and managed to combine it with our well-known collaborative review sessions, around feature proposal freeze. This had a good impact on raising awareness on the changes being proposed, as well as prioritizing the reviews. - Great contributions ranging from new third party drivers to successful internships on manila-ui, bandit and the ongoing OpenAPI internships. *Action items:* - Carlos (carloss) will work with the manila team to help people gain context on the bug czar role and work with the team to rotate it. - Vida Haririan (vhari) will jot down the details of the Bug Czar role - Follow the discussions on teams joining the VMT and get Manila included too. - Spread the word on the removal of the manila client and switch to OpenStackClient *Share backup enhancements* ========================= - Out of place restore isn't supported currently. We have agreed that this is a good use case and that a design specification should be proposed to document this. - DataManager / BackupDriver - forcing the backup process to go through the DataManager service is supported through a config option, but Manila is currently not honoring it. We agreed that this is an issue in the code, and we will review the proposed change [2] to make the data manager honor this config. - DataManager to allow for a backup driver to provide reports on API call progress: Currently, the data manager fetches the progress of a backup using a generic get progress call, but it is failing with the generic backup driver. We suggested that this should be fixed in the base driver. - Context for Backup API calls: currently, only objects representing a Share and Backup are passed to the backup driver. The request context should also be forwarded in these calls. The backup driver interface can be changed for this, but we should be mindful of out of tree drivers that could break. *Action items:* - Zach Goggins (zachgoggins) will look into: - Proposing a spec for the share backup out of place restore. - Updating the backup driver interface and adding context to the methods that need it. - Updating the backup driver interface and adding the abstract methods/capabilities that will help with the `get_restore_progress` and `get_backup_progress` methods. - The manila team will provide feedback on [2] *All things CephFS* =============== *Updates from previous cycles* ------------------------------------------ *State of the Standalone NFS Ganesha protocol helper:* - We added a deprecation warning at the end of the previous SLURP release, and we are planning to complete the removal during the 2025.1/Epoxy release. There were no objections to this so far at the PTG. When this is removed, CephFS-via-NFS will only work with cephadm deployed ceph-nfs clusters. *Testing and stabilization:* - devstack-plugin-ceph has been refactored to deploy a standalone NFS-Ganesha service with a ceph orch deployed cluster. We also dropped support for package-based and container-based installation of ceph. cephadm is used to deploy/orchestrate ceph. - Bumped Ceph version to Reef in Antelope, Bobcat, Caracal, Dalmatian, as well as started testing with Squid. - There are some failures on stable branches jobs which are being triaged and fixed. *Manage/unmanage:* - Implementation completed in Dalmatian and the documentation has been updated. We are currently working to enable the tests on CI permanently, as well as doing some small refactors to the CI jobs. *Ensure shares:* - Merged in Dalmatian but testing is still challenging, as running the tests mean that the service would temporarily have a different status and shares within the backend would have their status changed, which is harmful for test concurrency. *Preferred export locations and export location metadata:* - The core feature merged, but we are still working to get the newly implemented tests passing and merged. *Plans for 2025.1/Epoxy* -------------------------------- - NFSv3 + testing: we are looking into enabling NFSv3 support as soon as the patch is merged in Ceph. We agreed that we should enable the tests within manila-tempest-plugin and make any necessary changes to the tests structure, so we can ensure that we are testing some scenarios with both NFSv3 and NFSv4. - We will start to investigate support for SMB/CIFS shares and look at the necessary changes for setting up devstack and testing. *Action items:* - Carlos (carloss) will write an email to the openstack-discuss mailing list announcing the removal of the deprecated ganesha helper - Carlos pursue the manage/unmanage testing patches to have tests enabled in the CephFS jobs during Epoxy. - Carlos will look into approaches to test ensure shares APIs. - Ashley (ashrod98) will continue working on the export location metadata tempest changes and drive them to completion. - The manila team will look into updating manila-tempest-plugin tests and enabling NFSv3 tests in the Ceph NFS jobs - Goutham (gouthamr) will be submitting a prototype of the SMB/CIFS integration *Tech Debt* ======== *Eventlet removal* ---------------------- Our main concerns: - Performance should not be degraded with the default configuration when we switch. - Synchronous calls do not take a big hit and become asynchronous. - Impact to the SSH Pool (used by many drivers) should be minimal. *Action items for 2025.1 Epoxy:* - Tackle the low-hanging-fruit changes. - Participating in the pop-up team discussions. - Removing the affected console scripts in Manila. - Working on performance tests to understand what will be the impact on the SSH pool that is used by some drivers. - Look into enhancing our rally/browbeat test coverage. *CI and testing images* ------------------------------- We started working on the migration of the CI to Ubuntu 24.04 in all of the manila repositories (manila-image-elements, python-manilaclient, manila-ui, manila, manila-specs). Currently, the Ceph job is broken [3]. *Action items:* - We should clean up our CI job variants, as they have a lot of workarounds and we can start moving away from them. *Stable branches* ---------------------- We currently have 5 "unmaintained" branches, so we should be looking at sunsetting them. *Action items:* - Carlos (carloss) will start the conversation for the transition of some of these branches in the openstack-discuss mailing list. *Allowing force delete of a share network subnet* ======================================= We currently can add subnets (which translates to adding new network interfaces) to a share server but we can't remove them. This is a proposal to add this removal feature and being able to detach a network interface of a share server. We agreed that: - This is a good use case and something that can be enhanced. - The enhancement should add a force-delete API. - We should not allow the last subnet to be deleted, otherwise the shares won't have an export path. - A bug should be filed for a tangential issue that the NetApp driver is using "neutron_net_id" (and possibly "neutron_subnet_id" to name resources on the backend: ipspaces, broadcast domains, and possibly concurrency control / locks. *Action Items:* - sylvanld will look into proposing a spec to document this behavior *NetApp Driver: Enforcing lifs limit per HA pair* ===================================== - The NetApp ONTAP storage has a limit of network interfaces per node in a HA pair. In case the sum of allocated network interfaces in the two nodes of the HA pair is bigger than the limit of the single node, then the failover operation is compromised and will fail. - NetApp maintainers would like to fix this issue, and we agreed that: - The fix should be as dynamic as possible, not relying on users/admin input or configuration. - The ONTAP driver must look up all of the interfaces already created and allow/deny the request in case it would compromise the failover. - The NetApp ONTAP driver should keep an updated capability with the max network interfaces support number, and possible the number of allocated network interfaces at the moment. *NetApp Driver: Implement Certificate based authentication* ================================================ - The NetApp ONTAP driver currently handles only user/password authentication, but in an environment that password should change quarterly, this means updating the local.conf at least every three months. This enhancement proposes also adding the possibility of adding certificate based authentication. - We agreed that this is something that is going to be important for operators and will allow them to add their certificates with a longer expiration date, avoiding the disruptions caused by needing to update the user/password. *Manage Share affinity relationships by annotation/label* ============================================= Currently the manila scheduler uses affinity/anti-affinity hints and we base ourselves on share IDs. The idea now would be to have the affinite hints to be based in an affinity policy, as possible with Nova. We considered the proposed approaches, and agreed that: - If we are adding new policies, they should end up becoming a new resource/entity within the manila database - If there is a way to reuse the share groups mechanism, we should prioritize it *Action items:* - Chuan (chuanm) will propose a design spec to document this new behavior. *Share encryption* ============== This feature is currently waiting for more reviews and testing on gerrit. In the Dalmatian release mid-cycle we talked about the importance of testing this feature against a first party driver, to ensure that the APIs and integration with Barbican and Castellan work. We agreed that: - We should do some research on how to do this testing with the generic driver (which uses Cinder and Nova) - The testing will focus on the APIs and behavior of this feature, not the encryption of the shares. *Action items:* - gouthamr will help with some research on how to test this with the generic driver - The manila team will discuss this again in the upcoming manila weekly meetings. [0] https://www.youtube.com/watch?v=8UxrjEr6yik&list=PLnpzT0InFrqDHGfSDPhiGtSeX… [1] https://etherpad.opendev.org/p/epoxy-ptg-manila [2] https://review.opendev.org/c/openstack/manila/+/907983 [3] https://www.spinics.net/lists/ceph-users/msg83201.html

1 0

[qa][all] stable/2023.1 gate is blocked (hold the recheck)
by Ghanshyam Mann 31 Oct '24

31 Oct '24

Hello Everyone, Yatin pointed out that the python 3.8 drops impacted the stable/2023.1 focal jobs also. As per the stable/2023.1 testing runtime[1], we run at least one job on focal and the rest all on jammy. Python 3.8 drops broke the focal job[2]. To solve this, we need to pin the Tempest on stable/2023.1 testing (Yatin has proposed it[3]) We are testing it on neutron gate - https://review.opendev.org/c/openstack/neutron/+/933705 Until the fixes[3] are merged, please hold off on rechecking stable/2023.1 focal job failure. [1] https://governance.openstack.org/tc/reference/runtimes/2023.1.html#addition… [2] https://bugs.launchpad.net/devstack/+bug/2085886 [3] https://review.opendev.org/q/topic:%22bug/2085886%22 -gmann

1 1

[python-openstackclient] List security group IDs used by a VM with "openstack server show"
by Sebastian Luna Valero 31 Oct '24

31 Oct '24

Hello, Currently "openstack server show" provides the list of security groups used by the VM. However, this is a list of names only [1]. Therefore a VM may contain multiple security groups with the same name but different IDs. However, getting the IDs of the security groups used by a VM seems possible according to the API [2]. Would it be possible to make this available via the python-openstackclient CLI? Many thanks, Sebastian [1] https://docs.openstack.org/api-ref/compute/#show-server-details [2] https://docs.openstack.org/api-ref/compute/#list-security-groups-by-server

2 1

[barbican] KMIP integration
by przemyslaw.kuczynski＠ochk.pl 31 Oct '24

31 Oct '24

Why KMIP integration always create OPAQUE secret with name Opaque Object . There can be created only 1 entry Is there bug ? https://bugs.launchpad.net/barbican/+bug/2033620

2 1

How best to find a compatible cpu model for disparate CPUs?
by Andy Speagle 31 Oct '24

31 Oct '24

Hey Folks, I have some wildly disparate CPUs throughout my nova-compute environment and I'm trying to figure out the best way to find the correct cpu_model and any "extra_flags" that might need to be passed. According to "virsh capabilities" I have Broadwell-IBRS, Skylake- Server-IBRS and SapphireRapids-noTSX ... And... my attempts at finding a compatible model/flags has failed miserably. I tried getting all of the "capabilities" in one file and passing that through "virsh cpu-baseline" but that returned something completely incompatible with the Broadwell ... I looked at "virsh domcapabilities" and found that they all support "Broadwell-noTSX-IBRS" ... and nova- compute will start up using that... but, live migrations between the Broadwell and Skylake CPUs to the SapphireRapids doesn't work properly. tl;dr ... any tips on making this work with wildly disparate CPUs? -- Andy Speagle

3 2

[skyline]Using multi-domain management with the skyline UI
by 啧啧啧 31 Oct '24

31 Oct '24

Hi , Skyline Team I'm trying the skyline UI using multi-domain management, find some problems。 I created a new domain for domain-test1 using the openstack backend, create a test1 project under domain-test1. I used the skyline UI to add users to the test1 project and found that the user under the default domain could be added and given the admin role, but it was not available. I understand that this assignment is wrong and that the roles of project management should be different for projects that are not in the same domain. At the same time, I used skyline to try to integrate keystone LDAP. Reference https://docs.openstack.org/keystone/latest/admin/configurati…. I used the openstack backend to create a new domain for domain-test2 and create a test2 project under domain-test2 to configure ldap. I used the skyline ui to view the users in the domain-test2 domain, and all the users configured in ldap could be queried. The project in the domain-test2 domain could not manage the view users and was authorized.

1 0

[skyline]Using multi-domain management with the skyline UI
by 任亮 30 Oct '24

30 Oct '24

Hi , Skyline Team I'm trying the skyline UI using multi-domain management, find some problems。 I created a new domain for domain-test1 using the openstack backend, create a test1 project under domain-test1.  I used the skyline UI to add users to the test1 project and found that the user under the default domain could be added and given the admin role, but it was not available. I understand that this assignment is wrong and that the roles of project management should be different for projects that are not in the same domain. At the same time, I used skyline to try to integrate keystone LDAP. Reference https://docs.openstack.org/keystone/latest/admin/…. I used the openstack backend to create a new domain for domain-test2 and create a test2 project under domain-test2 to configure ldap. I used the skyline ui to view the users in the domain-test2 domain, and all the users configured in ldap could be queried.  The project in the domain-test2 domain could not manage the view users and was authorized.

1 0

[PTL][release][stable] 2023.1 Antelope transition to Unmaintained
by Elõd Illés 30 Oct '24

30 Oct '24

Hi teams, As 2024.2 Dalmatian was released last week and we are in a less busy period, now is the good time to call your attention to the following: At the end of October 2023.1 Antelope is planned to transition to Unmaintained phase [1] (planned date: 2024-10-30). I have generated the list of the current *open* and *unreleased* changes in stable/2023.1 for every repositories [2] (where there are such patches). These lists could help the teams who are planning to do a *final* release on Antelope before moving stable/2023.1 branches to unmaintained/2023.1. Feel free to edit and extend these lists to track your team's progress! Note that the last hash on stable/2023.1 branch will be tagged ('<series>-eom' tag) and transition patches will be generated soon. After the tagging and cutting unmaintained/2023.1 branch, the new unmaintained branch will be open to for bug fixes, but there won't be official releases for 2023.1 Antelope anymore. (You can get some information about the Unmaintained state on stable policy page [3]). *NOTE*: teams, please focus on wrapping up your libraries first if there is any concern about the changes, in order to avoid broken (final!!) releases! [1] https://releases.openstack.org/ [2] https://etherpad.opendev.org/p/2023.1-antelope-final-release-before-eom [3] https://docs.openstack.org/project-team-guide/stable-branches.html#unmainta… Thanks, Előd Illés irc: elodilles @ #openstack-release

2 1

[manila] Cancelling Oct 31 weekly meeting
by Carlos Silva 30 Oct '24

30 Oct '24

Hello everyone! Traditionally in the past cycles we cancelled the upstream weekly meeting that followed the PTG to get closure to some PTG AIs. I believe we can continue that tradition this time too. We'll get back to the weekly meetings next week, Nov 7th. If you have something urgent, please ping me on IRC. Regards, carloss

1 0

[tooz] Recommended driver
by William Szumski 30 Oct '24

30 Oct '24

Hi all, Just wondering what the recommend driver is these days. The docs[1] seem to suggest zookeeper is the most stable and feature rich. Is that still the case? Is it advisable to use zookeeper over redis/valkey? What gets the most testing in upstream CI? Thanks, Will [1] https://docs.openstack.org/tooz/latest/user/drivers.html

3 2

[Openstack-Helm] 2025.1 Epoxy PTG session summary
by Vladimir Kozhukalov 30 Oct '24

30 Oct '24

Hi Openstack-Helmers, Thank you for attending our 3 hours 2025.1 (Epoxy) PTG session. The session etherpad is here [0] and the following is the short summary. Items listed below are more or less sorted by their priority. ### Charts Agreed to fix the issue regarding overriding charts when publishing them with the same version. There must be a spec that clearly describes the workflow of how we are going to bump versions, build and publish charts. Items: - Migrate from Helm API v1 to Helm API v2 for Chart.yaml (including tracking dependencies in the Chart.yaml file instead of requirements.yaml) - Move overrides to a separate directory - Add CHANGELOG.md to every chart - Since we are using kubernetes-entrypoint [1] for orchestration, let's consider removing post-* hooks. This is going to make it easier to integrate with third party CI/CDs. - Let's consider switching to the same major version for all charts. If the subchart is changed the minor chart version must be bumped. ### Secrets Some upstream charts provide the possibility to refer to existing secrets. See for example [2]. This allows easy integration with various CI/CDs. We agreed to create a design spec and implement a similar pattern to make it possible for users to get sensitive info from external predeployed secrets. Also in some cases users would like to stick to third party projects when talking about Openstack backend like MariaDB and RabbitMQ. The spec must also describe the way how we are going to make OSH charts compatible with this. In case of Rook Ceph we use `ceph-adapter-rook` chart which takes care of creating secrets that are then consumed by the rest of charts. Probably the same approach can work for other cases. ### Images - We agreed to de-duplicate the image build code for Openstack images and switch to using Loci images. - Loci tries to go its own way using one Dockerfile for building all Openstack projects. This leads to having project/version specific parts in the build workflow that makes it harder for users to customise the build workflow. Let's be more straightforward here. There are examples like this [3] where per project Dockerfiles are used that utilize two base images for `python-venv-builder` and `python-runtime`. Let's consider adopting this approach. Agreed to prepare a spec for this. ### Gateway API, Ingress, NetworkPolicy, PodDisruptionBudget ... Gateway API is coming and is supposed to substitute the Ingress API. We agreed to prepare the design to support Gateway API. Also let's try to re-factor the way how the Openstack-Helm charts deploy Ingress objects and try to follow more common practices regarding this. Instead of having these two types of Ingresses cluster and namespace it is better to have a way for a user to define explicitly the list of Ingresses with their routes. Let's also use similar pattern for managing other optional K8s API obejects like NetworkPolicies and PodDisruptionBudget. It is better for users when they are able to define explicitly what they want to deploy instead of using hidden logic like what we have for managing Ingresses. Design spec is needed. ### Noble It is not clear at the moment whether Ceph will officially support Noble by 2025.1. So perhaps we'll be forced to build 2025.1 Jammy images. Anyway we are going to have two sets of test jobs with Jammy and Noble as a host OS. This is to make sure users can upgrade the host OS smothly. At the moment there are no volunteers to add support any other OS. ### Testing There are lots of test cases and having all these test jobs in the check pipeline not necessary. Instead we agreed to have a meta layer for tracking all these test cases and generate zuul jobs files programmatically. ### OceanBase OceanBase is a distributed database compatible with MariaDB. This is the blog post [4] about how it can be used with Openstack as the DB backend. The OceanBase team showed a demo during OpenInfra days in Asia in Sep/2024. We agreed that this use case can be interesting at least for some users so let's have a test job for this. [0] https://etherpad.opendev.org/p/oct2024-ptg-os-helm [1] https://github.com/airshipit/kubernetes-entrypoint.git [2] https://github.com/bitnami/charts/blob/f14d986d3713716a9b263075ac784ef74943… [3] https://github.com/vexxhost/atmosphere/tree/main/images [4] https://en.oceanbase.com/blog/13879352832 -- Best regards, Kozhukalov Vladimir

1 0

[cinder] No meeting today (10/30)
by Jon Bernard 30 Oct '24

30 Oct '24

Hello all, I'm not feeling well today and need to cancel today's meeting. See you all next wednesday. -- Jon

1 0

[horizon][ptg] 2025.1 Epoxy PTG summary
by Radomir Dopieralski 30 Oct '24

30 Oct '24

We had two days of Horizon discussion on the last PTG, first the internal one: https://etherpad.opendev.org/p/horizon-epoxy-ptg and then the cross-project sessions with other teams: https://etherpad.opendev.org/p/horizon-feature-gap On the first day we looked back at the previous cycle and summarized what was achieved, and what needs further attention. We mentioned the work done for Neutron API migrating it to OpenStackSDK, the improvements of integration and UI tests, and the work done by our Outreachy student. We discussed ways to make Horizon easier to approach for new developers. We also discussed the problems with AngularJS and a plan to remove it from Horizon. On the second day, we met with other project teams and discussed what needs to be done in Horizon to bring it up to speed with the progress made in other projects. We also discussed further work on migrating to OpenStackSDK. After further discussion, we decided to focus in this cycle on the following issues: • New SASS compiler library • Remove AngularJS • Research Unified Limits • Placement API Discussion — follow-up meeting with the Nova team • Use "direct" upload mode by default in the operator configuration • Start migration to OpenStackSDK for Nova • GUI for managing Keystone 2FA • Configuring integration tests for different themes • Better grooming of upstream bugs and easy-to-start issues In addition, if there are new contributors or we have spare time, we consider working on: • Keystone Re-enable user • Refresh Cinder API microversions • Hidden image in Glance • Filtering of images dropdown in rebuild dialog • Show backends in Glance information

1 0

[Horizon] Cancelling weekly IRC meeting tomorrow
by Tatiana Ovchinnikova 29 Oct '24

29 Oct '24

Hello team, Thank you for joining the vPTG last week! Since there's a week of chaos because of the time change, and part of the team is on PTO, I'm canceling our weekly meeting tomorrow. See you all next Wednesday! With kind regards, Tatiana

1 0

[i18n][ptg] I18n SIG vPTG summary
by Ian Y. Choi 29 Oct '24

29 Oct '24

Hi all, I would like to share with you a summary of the recent discussions held during the 2025.1 Epoxy PTG regarding the Internationalization (I18n) Special Interest Group (SIG). During the period, the I18n SIG explored approaches to improve multilingual support, discussing Weblate migration, AI-based translation implementation, sustainable documentation translation strategies, and addressing current translation tool issues. Special thank you to TC and Nova team for elaborate discussions. Please refer to the below brief summary from Etherpad: https://etherpad.opendev.org/p/oct2024-ptg-i18n . 1. Overall Translation Status discussion - Clarifying that StringFreezes applies to Horizon and dashboard plugin projects [1] - Confusion exists about the current state of translations. Horizon translations were added months ago but not proposed by the tooling. : No translation proposals on the master branch after April 19, 2024. [2] : Horizon removed Python 3.8 support [3], while translation jobs depend on Ubuntu Bionic. 2. Towards sustainable Documentation Translation - Discussion on utilizing AI machine translation: : Proposal to use Weblate's machine translation feature. : Consideration of open-source tools like LibreTranslate [4]. : Concerns raised about legal issues. - Emphasis on the necessity of human reviewers: : Suggestion to create a 'language-moderator' group in Gerrit? : Need to add voting labels for translation patches. 3. Weblate Migration - Ongoing migration of translated project files from Zanata to Weblate. - Integration with Zuul is the second-phase goal. - Difficulties encountered due to API usage issues. 4. Weblate Issues - There was an authentication issue with Weblate and openinfraid (note: but it appears to be resolved now. Thank you @rosmaita!) 5. AI Translation Approach - Step-by-step ways a. Model selection b. Community review of the model c. Model utilization d. Model + RAG (Reviewed/Translated OpenStack documents, glossary) utilization - Plan to conduct PoC with Nova RST documents. 6. Action items - To start a 3-month PoC to test AI translation with Nova RST files (ianychoi) - Zuul investigation for Weblate migration (seongsoocho) - Horizon Zuul job issue check (discussion with other teams) Thank you all, /Ian [1] https://review.opendev.org/c/openstack/releases/+/930929 [2] https://review.opendev.org/q/owner:proposal-bot+project:openstack/horizon+b… [3] https://docs.openstack.org/releasenotes/horizon/2024.2.html#upgrade-notes [4] https://github.com/LibreTranslate/LibreTranslate

1 0

[all[qa] Tempest dropping python 3.8 support
by Ghanshyam Mann 29 Oct '24

29 Oct '24

Hello Everyone, As you know, services and Oslo libs are dropping Python 3.8 in this cycle. Tempest is branchless, and we use Tempest master to test all supported stable branches (currently 2023.1 to 2024.2). Because Python 3.8 is still supported in stable/2023.1 to stable/2024.1 Initially, we thought of continuing the py 3.8 support in Tempest, but without Oslo libs, Tempest cannot support it. One way is to keep the python 3.8 constraints separately (either a separate file in Tempest or separate constraints for python3.8 in requirements), which stops Tempest from adding any new features from oslo/deps. Another way (this is what we did in the previous Python version bump) is to drop the Python 3.8 support, and if anyone wants to test the cloud with Python 3.8, they can install and run Tempest on Python >=3.9 virtual env. In upstream testing, we also install and run Tempest from the virtual environment. or use the Tempest 41.0.0 (the last release which supports py3.8) to test python 3.8 cloud. This is what we did during py3.6/3.7 support removal. - https://review.opendev.org/c/openstack/tempest/+/933398 -gmann

1 1

Re: [Cinder][Ceph] Volumetria errada no cluster ceph
by Luiz Fernando Silva 29 Oct '24

29 Oct '24

I am sending the file with the ceph osd df tree and ceph pg ls-by-pool volumes output as an attachment. ceph -s: cluster: id: ccdcc86e-c0e7-49ba-b2cc-f8c162a42f91 health: HEALTH_OK services: mon: 3 daemons, quorum hc-node02,hc-node04,hc-node03 (age 5w) mgr: hc-node02(active, since 4M) osd: 54 osds: 54 up (since 14h), 54 in (since 3M) task status: data: pools: 5 pools, 3424 pgs objects: 4.25M objects, 43 TiB usage: 88 TiB used, 32 TiB / 120 TiB avail pgs: 3424 active+clean io: client: 16 MiB/s rd, 5.7 MiB/s wr, 856 op/s rd, 483 op/s wr Thank you very much, we will adjust to replica 3.

1 0

Re: [barbican][vault]
by Andy Speagle 29 Oct '24

29 Oct '24

On Thu, 2024-10-24 at 18:06 -0700, melanie witt wrote: > On 10/24/24 16:43, Andy Speagle wrote: > > Hey Folks, > > > > I'm having some trouble getting barbican to generate and/or store > > keys > > in our hashicorp vault in our charmed openstack environment. I have > > issues with both TPM and SSL related keys. > > > > I get extremely generic errors back that I can't seem to figure out > > how > > to troubleshoot: > > > > 'Traceback (most recent call last):\n File "/usr/lib/python3/dist- > > packages/nova/conductor/manager.py", line 688, in build_instances\n > > scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist- > > packages/nova/scheduler/utils.py", line 998, in populate_retry\n > > raise > > exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetries > > Exce > > eded: Exceeded maximum number of retries. Exceeded max scheduling > > attempts 3 for instance 0c50b2d7-8ac5-41a2-90ab-1e33f7aa1c22. Last > > exception: Key manager error: Internal Server Error: Secret > > creation > > failure seen - please contact site administrator.\n' > > Just FYI this is the generic MaxRetriesExceeded error which means > that > the server action failed on nova-compute(s). So you'll want to look > in > your nova-compute.log for the error that triggered the retries and > eventual MaxRetriesExceeded. > > > I'm using barbican 14.0.2 in yoga/stable ... anyone have an ideas > > how > > to troubleshoot this? > > I'm guessing in nova-compute.log you will see an error 403 forbidden > for > creating a barbican secret. If so, the most common reason I have seen > for this is if you have in your barbican.conf > [oslo.policy]enforce_new_defaults = False, then secret creates > require > the requester to have the 'creator' role assigned to them (you can do > this by 'openstack role add <options> <role>' [1]). > > When [oslo.policy]enforce_new_defaults = True, the 'creator' role is > not > required. Yup... nailed it. I really appreciate the insight. :) > The tl;dr is 'enforce_new_defaults' means to use the newer secure > RBAC > policies and the secure RBAC changes are shown in the Wallaby release > notes:

1 0

[neutron] Moving CI Meeting to Monday 1500 UTC
by Yatin Karel 29 Oct '24

29 Oct '24

Hello Neutron Team !! As discussed/agreed over the last Neutron CI meeting, pushed the changes[1][2] for the new timing. [1] https://review.opendev.org/c/opendev/irc-meetings/+/933637 [2] https://etherpad.opendev.org/p/neutron-ci-meetings -- Thanks and Regards Yatin Karel

1 0

[qa][ptg] QA 2025.1 vPTG Summary
by Martin Kopec 29 Oct '24

29 Oct '24

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Dalmatian (2024.2) Retrospective == At the beginning of the cycle a devstack optimization change was merged [2] that uses the OSC service. If you haven't already, give it a try and enable it in your jobs. If it doesn't work for your project, let us know. == Migrating upstream CI/CD jobs to Ubuntu Noble (24.04) == Details about the migration can be found in [3] and [4] together with the testing status of all projects. We have time until Nov 29th to test the newer Ubuntu version and deal with any issues, after that, QA would like to proceed with the migration. == Triage previous and write new priority items == The list of the priority items planned for this cycle as well as the ones which were carried over from the previous cycles, can be found here [5]. == Open Discussion == There have been issues with a few jobs and tests that made an assumption of having the amount of certain resources, such as CPUs, guaranteed. However, that's not the case and the amount of the resources is not guaranteed in the upstream CI. No tests or jobs should make this assumption. Apart from CPUs, the same applies on floating ips, public addresses etc. [1] https://etherpad.opendev.org/p/oct2024-ptg-os-qa [2] https://review.opendev.org/c/openstack/devstack/+/676016 [3] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [4] https://etherpad.opendev.org/p/migrate-to-noble [5] https://etherpad.opendev.org/p/qa-epoxy-priority Thank you all for attending the PTG and contributing to the discussion, -- Martin Kopec Red Hat EMEA IM: kopecmartin

1 0

[nova][ptg] 2025.1 Epoxy PTG summary
by Sylvain Bauza 29 Oct '24

29 Oct '24

(resending the email as the previous one was blocked to an attached etherpad backup txtfile larger than the max size) Hey all, First, thanks for having joined us if you were in the vPTG. We had 15-20 people every day for our nova sessions, I was definitely happy to see new folks :-) If you want to see our PTG etherpad, please rather look at https://etherpad.opendev.org/p/r.4f297ee4698e02c16c4007f7ee76b7c1 instead of the main nova etherpad as I don't want that the etherpad would have a wrong traduction or having some paragraphs to be removed. As I say every cycle, just take a coffee (or a tea) now as the summary will be large. ### Dalmatian retrospective and Epoxy planning ### 6 of 15 approved blueprints were eventually implemented. We also merged more than 31 bugfixes during Dalmatian. We agreed to be explaining on the IRC channel when we have meetings for discussing some feature series (like the one we did every week for the manila/virtiofs series) and providing some public invitations. We could do this again this cycle for other features, we'll see. We will also try to have a periodic integration-compute job that pulls OSC and SDK from master. Our Epoxy deadlines will be : two spec review days (R-16, R-2), a soft spec approval freeze by R-16 and then hard spec approval freeze by R-12. That means that contributors really need to provide their specs before mid-December. Bauzas (me) will add these deadlines into the Epoxy schedule : https://releases.openstack.org/epoxy/schedule.html ### vTPM live migration ### We agreed on the fact that a vTPM live-migration feature is a priority for Epoxy given Windows 11. artom will create a spec proposing an image metadata property saying 'do I want to share my secret with nova service user ?' and also providing a new `nova-manage image_property set migratable_something` command so operators could migrate the existing instances for getting the Barbican secrets, if really the operators wants. ### Unified limits wrap-up ### We already have two changes needing to be merged before we can modify the default quota driver (in order to default to use unified limits). We agreed on reviewing both patches (one for treating unset limits as unlimited, the other about adding a nova-manage command for automatically creating nova limits) but we also discussed about a latter patch that would eventually say which nova resources need to be eventually set (so we *have to* enforce them anyway). melwitt agreed on working on that latter patch. ### per-process health checks ### We already had one series and we discussed it again. Gibi agreed on taking over it and he will re-propose the existing spec as it is. We also discussed the first checks we would have, like RPC failures and DB connection issues, we'll review those when they are in Gerrit. ### sustainable computing (a.k.a. power mgmt) ### When someone (I won't say who [1]) implemented power management in Antelope, this was nice but we eventually found a long list of bugs that we fixed. Since we don't really want to reproduce that experience, we had this kind of post-mortem where we eventually agreed on two things that could avoid reproducing that problem : a weekly periodic job will run whitebox tempest plugins [2] with nova-compute restarts also covered by a whitebox tempest plugin. Nobody is committed against those two actions but we hope to identify someone soon. As a side note, gibi mentioned RAPL MSR support [3], notifying us that we would have to support that in a later release (as the libvirt implementation is not merged yet) ### nvidia's vGPU vfio-pci variant driver support ### Long story short, since the linux kernel removed some feature in release 5.18 (IOMMU backend support for vfio-mdev) this impacted the nvidia driver which now detects that and then creates vfio-pci devices instead of vfio-mdev devices (as vGPUs). This has a dramatic impact on Nova as we relied on the vfio-mdev framework for abstracting virtual GPUs. By the next release, Nova will need to inventorize the GPUs by rather looking at SRIOV virtual functions which are specific to the nvidia driver (we call them vfio-pci variant driver resources). The nova PTG session focused on the required efforts to do so. We agreed on the fact it will require operators to propose different flavors for vGPU where they would require distinct resource classes (all but VGPU). Fortunately, we'll reuse existing device_spec PCI config options [4] where the operator would define custom resource classes which would match the PCI addresses of the nvidia-generated virtual functions (don't freak out, we'll also write documentation). We'll create another device type (something like type-VF-migratable) for describing such specific nvidia VFs. Accordingly the generated domain XML will correctly write the device description (amending the "managed=no" flag for that device). There will be an upgrade impact: existing instances will need to be resized to that new flavor (or instances will need to be shelved, updated for changing the embedded flavor and unshelved). In order to be on par with existing vGPU features, we'll also need to implement vfio-pci live-migration by detecting the VF type on the existing SRIOV live-migration. Since that effort is quite large, bauzas will incept a subteam of interested parties that would help him implement all of those bits in the short timeframe that is one upstream cycle. ### Graceful shutdowns ### A common pitfall that was told by tobian-urdin is when you want to stop nova-compute services. In general, before stopping the service, we should be sure that all RPC calls are done, which means we would no longer accept RPC calls after asking to stop the nova-compute and just awaiting the current calls to be done before stopping the service. For that, we need to create a backlog spec for discussing that design and we would also need to modify oslo.service for unsubscribing the RPC topics. Unfortunately, this cycle we won't have any contributor for working on it, but gibi could try to at least document this. ### horizon-nova x-p session ### We mostly discussed the Horizon feature gaps [5]. The first priority would be Horizon to use OpenStackSDK instead of novaclient, but then supporting all of the new Nova API microversions. Unfortunately, we are no sure that we could have Horizon contributors that could fix those, but if you're a contributor and you want to help Horizon to be better, maybe you could do this ? If so, please ping me. ### Ironic-nova x-p session ### We didn't really have topics for this x-p session. We just quickly discussed some points, like Graphical Console support. Nothing really worth noting, maybe just that it would be nice that we could have readonly graphical console. We were just happy to say that the ironic driver now works better thanks to some features that were merged last cycles. Kudos to those who did them. ### HPC/AI optimized hypervisor "slices" ### A large topic to explain, I'll try to keep it short. Basically, how Nova slices the NUMA affinity between guests is nice but hard for HPC usecases where sometimes you need to better explain how to slice the NUMA dependent devices depending on the various PCI topologies. Eventually, we agreed on some POC that johnthetubaguy could work on by trying to implement a specific virt driver that would do something different from the existing NUMA affinities. ### Cinder-nova x-p session ### Multiple topics were discussed there. First, abishop wanted to enhance cinder's retyping of in-use boot volumes which means that the Nova os-attachments API to get a new parameter. We said that he needs to create a new spec and we agreed on the fact that the cinder contributors need to discuss with QEMU folks to know about the qemu writes. We also discussed about a new nova spec which is about adding burst length support to Cinder QoS [6]. We said that we need to both (nova and cinder) review this spec. About left residues when detaching a volume, we also agreed on the fact this is not a security flaw and the fact that os-brick should delete them, not nova (even if nova need to ask os-brick to look at that, either by a periodic run or when attaching/detaching). whoami-rajat will provide a spec for it. ### Python 3.13 support ### We discussed a specific issue for py3.13, the fact that the crypt module is no longer in stlib for py3.13, which impacts nova due to some usage in nova.virt.disk.api module for passing an admin password for file injection. Given file injection is deprecated, we have three possibilities: either removing admin password file injection (or even file injection as a whole), adding the new separate crypt package in upper-constraints or using oslo_uitls.secretutils module. bauzas (me) will provide an email to openstack-discuss for asking operators whether they are OK with deprecating file injection or just admin password injection and then we'll see the direction. bauzas or sean-k-mooney will also try to have py3.13 non-voting jobs for unittests/functtests. ### Eventlet removal steps in Nova ### I won't explain why we need to remove eventlet, you already know, right ? We rather discussed about the details in our nova components, including nova-api, nova-compute and other nova services. We agreed on removing direct eventlet imports where possible, move nova entrypoints that don't use eventlet to separate modules that don't monkeypatch the stdlib, look at what we can do with all our scatter_gather methods which asynchronously calling cells DB for using threads instead and check whether those calls are blocking on DB (and not on the MQ side). Gibi will shepherd that effort and provide some audit on the eventlet usage in order to avoid any unexpected but unfortunate late discoveries. ### Libvirt image backend refactor ### If you like spaghettis, you should pay attention to the libvirt image backend code. Lots of assumptions and conditionals make any change to that module hard to be written and hard to be reviewed, leading to errorprone situations like the ones we had when fixing some recent CVEs. We all agreed on the quite urgent necessity to refactor that code and melwitt proposed a multi-stage effort about that. We agreed on the proposal for the first two steps with some comments, leading to future revisions of the proposal's patches. The crucial bits with the refactor are about test coverage. ### IOThreads tuning for libvirt instances ### An old spec was already proposed for defining iothreads to guests. We agreed on reviving that spec, where a config option would define either no iothread or one iothread per instance (with a potential for a latter option value to be "one iothread per disk"). Depending on whether emulator_thread_policy is provided in the flavor/image, we would set the iothread on that policy or we would put the iothread floating over the shared CPU set. If no shared CPUs are configured but the operator wants iothreads, nova-compute would refuse to start. lajoskatona will work on such an implementation that will be designed in a blueprint that doesn't require a spec. ### OpenAPI schemas progress ### Nothing specific to say here, bauzas and gmann will review the series this cycle. That's it. I'm gone, I'm dead [7] (a cyclist metaphor) but I eventually skimmed the very large nova etherpad. Of course, 99% of chances that I'll write some notes incorrectly, so please correct if I'm wrong, I won't feel offended, just tired. Thanks all (and I hope your coffee or tea was good) -Sylvain [1] https://geek-and-poke.com/geekandpoke/2013/11/24/simply-explained [2] https://opendev.org/openstack/whitebox-tempest-plugin [3] https://www.qemu.org/docs/master/specs/rapl-msr.html [4] https://docs.openstack.org/nova/latest/configuration/config.html#pci.device… [5] https://etherpad.opendev.org/p/horizon-feature-gap#L69 [6] https://review.opendev.org/c/openstack/nova-specs/+/932653 [7] https://www.youtube.com/watch?v=HILcYXf8yqc

1 0

[neutron] How to disable Floating IP?
by yardalgedal＠gmail.com 28 Oct '24

28 Oct '24

Hi, Openstack Team, Is there a way how to temporarily disable floating ip? Like we have admin_state_up for many resources, but we don't have it for floating ip. admin_state_up of floating_port_id not working in Wallaby and I'm not sure it's working in later releases.

2 1

[tc][all] OpenStack Technical Committee Weekly Meeting Agenda (2025.1/R-22)
by Goutham Pacha Ravi 28 Oct '24

28 Oct '24

Hello Stackers, As you're aware, we concluded the virtual Project Teams Gathering event last week. This week, we’ll start addressing the to-do lists we created. The Technical Committee met for six hours, in addition to participating in several discussions with various project teams. I’m working on sharing a summary of the week’s proceedings with you via this mailing list. In the meantime, we’ll be catching up on 2024-10-29 at the TC’s weekly IRC meeting in OFTC’s #openstack-tc channel. The agenda for this meeting is available on the Meeting Wiki [1]. I hope you’ll be able to join us. === Governance Proposals === ==== Merged ==== - Retire kuryr-kubernetes and kuryr-tempest-plugin | https://review.opendev.org/c/openstack/governance/+/922507 - Propose a pop-up team for eventlet-removal | https://review.opendev.org/c/openstack/governance/+/931978 ==== Open for Review ==== - Add Cinder Huawei charm | https://review.opendev.org/c/openstack/governance/+/867588 - Add watcher DPL for Epoxy | https://review.opendev.org/c/openstack/governance/+/933018 - Add Axel Vanzaghi as PTL for Mistral | https://review.opendev.org/c/openstack/governance/+/927962 - Propose to select the eventlet-removal community goal | https://review.opendev.org/c/openstack/governance/+/931254 Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair P.S: OpenStack TC PTG YouTube Playlist is shaping up here [2]. Please feel free to browse through [1] TC Meeting Agenda, 2024-10-29: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://youtube.com/playlist?list=PLhwOhbQKWT7XGjIwT0mtPpixpuY-tKYoh&featur…

1 0

[glance] Hold your rechecks
by Abhishek Kekane 28 Oct '24

28 Oct '24

Hi Team, Currently glance gate is failing for glance-cinder-multistore-import job with quota exceeding issue. The quota issue seems to be occurring from a recent change in devstack (removing volumev3 endpoint) causing issues with openstack client commands (set quotas). The details of the issue and fix are in this release patch https://review.opendev.org/c/openstack/releases/+/933578 Until this release patch is merged, kindly avoid posting recheck on your patch. Thanks & Best Regards, Abhishek Kekane

1 1

[Neutron] Bug Deputy Oct 21 - 27
by Miro Tomaska 28 Oct '24

28 Oct '24

Hello All, Here is the report for the week from Oct 21 - 27. We had a busy week. *Critical:* https://bugs.launchpad.net/neutron/+bug/2085462 - [OVN] "test_trunk_subport_lifecycle" unstable Assigned: Rodolfo Alonso https://bugs.launchpad.net/neutron/+bug/2085543 - [OVN] Port device_owner is not set in the Trunk subport Assigned: Yatin Karel *High*: https://bugs.launchpad.net/neutron/+bug/2084977 - Deleting a security group fails "cannot find port_group" Unassigned https://bugs.launchpad.net/neutron/+bug/2085422 - [OVN] "test_overlapping_sec_grp_rules" is unstable Unassigned https://bugs.launchpad.net/neutron/+bug/2084977 - Deleting a security group fails with "Cannot find Port_Group with name=pg_<id>" when neutron-api is run by httpd + mod_wsgi Assigned: Terry Willson https://bugs.launchpad.net/neutron/+bug/2085422 - test_overlapping_sec_grp_rules" is unstable Unassigned *Medium*: https://bugs.launchpad.net/neutron/+bug/2085491 - How to run neutron fwaasV2 ovn driver Unassigned *Low*: https://bugs.launchpad.net/neutron/+bug/2085099 -[OVN] E/W traffic fragmentation for routed traffic Assigned: Rodolfo Alonso https://bugs.launchpad.net/neutron/+bug/2085447 - Fast exit ``_ensure_external_network_default_value_callback`` if network not external Assigned to Rodolfo Alonso *Undecided:* https://bugs.launchpad.net/neutron/+bug/2085421 - Tempest failing at teardown -- Miro Tomaska Senior Software Engineer

1 0

#eventlet-removal - PTG October 2024 Summary
by Herve Beraud 28 Oct '24

28 Oct '24

Last week was the OpenStack PTG. The Eventlet removal initiative ran two PTG sessions. The first session [1] was a presentation of the initiative, where the second session [2] was a technical open discussion. Please find out below the main key points and the proposed answers. ## Key Points The first key point made during these PTG sessions is that without consistency in our choices and decisions we will create more complexity than anything else. It is crucial to avoid having multiple ways to start services across different projects. A unified approach will simplify maintenance and enhance user experience. Leading us to the following questions: - how to remove the WSGI features provided by Eventlet and how to move to another HTTP server (WSGI/ASGI); - how to manage tasks. The WSGI/HTTP server discussion just come in time with the proposal made by Stephen Finucane to Deprecate/remove non-uWSGI deployment mechanisms from Devstack: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… We think that this topic overlaps with the WSGI aspects of Eventlet removal, for this reason, with the aim to answer these questions we propose to create specialized working groups (see the section below). We also made comparisons between available alternatives, here is an example with the SGI question https://wiki.openstack.org/wiki/SGI-servers-comparison. Find out all our comparisons in the official wiki page: https://wiki.openstack.org/wiki/Eventlet-removal#The_HTTP_SGI_Working_Group ## Specialized working groups The main role of these working groups is to avoid having multiple ways to manage things. Groups would be responsible for producing a unified approach that will simplify maintenance and enhance user experience. Our discussions have brought out 3 specialized working groups: - The HTTP SGI Working Group - The Tasks/Process Working Group - The Architecture, Interoperability and Technical Standardization Working Group If you are interested in one or many of these specialized working groups, then, please raise your hand and join the discussions. Please also take a look to the official page: https://wiki.openstack.org/wiki/Eventlet-removal#Specialized_Working_Groups… ## Latest Updates - Glance took full advantage of the PTG discussion. They got an overview of their WSGI migration options and they gain a better understanding of their current situation (https://etherpad.opendev.org/p/oct2024-ptg-glance#L277) and the Glance team were able to set up a migration plan: https://etherpad.opendev.org/p/oct2024-ptg-glance#L340 - Ironic Python Agent (IPA), is also experimenting about their Eventlet removal, their works is strongly related to the WSGI feature replacement: https://review.opendev.org/c/openstack/ironic-python-agent/+/924634 - The threading executor of oslo.messaging could shorten our migration path concerning the RPC aspects. For this reason we proposed to start the deprecation of the Eventlet executor within oslo.messaging. https://review.opendev.org/c/openstack/oslo.messaging/+/933399 - Another topic that arose during the PTG is how to manage timeouts. Some proposal have been made, they propose to rely on SQLAlchemy or Ceph to handle timeout management: https://etherpad.opendev.org/p/oct2024-ptg-eventlet-removal#L103 https://etherpad.opendev.org/p/oct2024-ptg-os-tc#L298 ## Important Dates Nov. 5th is the first IRC meeting on #openstack-eventlet-removal, do not hesitate to join us. We will cover all the topics addressed in this mailing thread. ## SLURP vs non-SLURP The PTG sessions came with timeline and cadence discussions. For this reason we made some proposal about workload regulation: https://wiki.openstack.org/wiki/Eventlet-removal#Workload_regulation_2 In substance we propose to base our cadence on SLURP and non-SLURP releases. Epoxy is a SLURP release, for this reason we should not expect breaking changes happening during this series, however, SLURP series are good opportunities to deprecate things, remove useless workaround, and start small clean up in preparation of wider changes. ## Final Thoughts We strongly encourage a collaborative selection of our candidate(s) for HTTP server replacement .This selection will dictate how we deal with server gateway interfaces for the coming decade, so we should focus our coming discussion on this specification, for this reason we invite you to join the specialized working groups. Thanks to all the people who joined the various discussions. Thank you for your dedication and hard work! [1] https://etherpad.opendev.org/p/oct2024-ptg-os-tc#L195 [2] https://etherpad.opendev.org/p/oct2024-ptg-eventlet-removal -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[FOSDEM] Call for participation: Virtualization and Cloud infrastructure Room at FOSDEM 2025
by Piotr Kliczewski 28 Oct '24

28 Oct '24

We are excited to announce that the call for proposals is now open for the Virtualization and Cloud infrastructure devroom at the upcoming FOSDEM 2025, to be hosted on Sunday (Feb 2) 2025. This devroom is a collaborative effort, and is organized by dedicated folks from projects such as OpenStack, Xen Project, KubeVirt, QEMU, KVM, and Foreman. We invite everyone involved in these fields to submit your proposals by December 8th, 2024. About the Devroom The Virtualization & IaaS devroom will feature session topics such as open source hypervisors or virtual machine managers such as Xen Project, KVM, bhyve and VirtualBox as well as Infrastructure-as-a-Service projects such as KubeVirt, Apache CloudStack, OpenStack, QEMU and OpenNebula. This devroom will host presentations that focus on topics of shared interest, such as KVM; libvirt; shared storage; virtualized networking; cloud security; clustering and high availability; interfacing with multiple hypervisors; hyperconverged deployments; and scaling across hundreds or thousands of servers. Presentations in this devroom will be aimed at developers working on these platforms who are looking to collaborate and improve shared infrastructure or solve common problems. We seek topics that encourage dialog between projects and continued work post-FOSDEM. Important Dates Submission deadline: 8th December 2024 Acceptance notifications: 10th December 2024 Final schedule announcement: 15th December 2024 Devroom: 2nd February 2025 Submit Your Proposal All submissions must be made via the Pretalx event planning site[1]. It is a new submission system so you will need to create an account. If you submitted proposals for FOSDEM in previous years, you won’t be able to use your existing account. During submission please make sure to select Virtualization and Cloud infrastructure from the Track list. Please provide a meaningful abstract and description of your proposed session. Submission Guidelines We expect more proposals than we can possibly accept, so it is vitally important that you submit your proposal on or before the deadline. Late submissions are unlikely to be considered. All presentation slots are 30 minutes, with 20 minutes planned for presentations, and 10 minutes for Q&A. All presentations will be recorded and made available under Creative Commons licenses. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example: "If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>." In the Submission notes field, please also confirm that if your talk is accepted, you will be able to attend FOSDEM and deliver your presentation. We will not consider proposals from prospective speakers who are unsure whether they will be able to secure funds for travel and lodging to attend FOSDEM. (Sadly, we are not able to offer travel funding for prospective speakers.) Code of Conduct Following the release of the updated code of conduct for FOSDEM[3], we'd like to remind all speakers and attendees that all of the presentations and discussions in our devroom are held under the guidelines set in the CoC and we expect attendees, speakers, and volunteers to follow the CoC at all times. If you submit a proposal and it is accepted, you will be required to confirm that you accept the FOSDEM CoC. If you have any questions about the CoC or wish to have one of the devroom organizers review your presentation slides or any other content for CoC compliance, please email us and we will do our best to assist you. Questions? If you have any questions about this devroom, please send your questions to our devroom mailing list. You can also subscribe to the list to receive updates about important dates, session announcements, and to connect with other attendees. See you all at FOSDEM! [1] https://pretalx.fosdem.org/fosdem-2025/cfp [2] virtualization-devroom-manager at fosdem.org [3] https://fosdem.org/2025/practical/conduct/

1 0

[neutron][ironic] Network VLAN ranges discussion during the PTG
by Rodolfo Alonso Hernandez 28 Oct '24

28 Oct '24

Hello all: During the PTG it was discussed how to handle the static configuration of the allowed VNI ranges for a network type. In particular it was discussed the VLAN ID ranges, that has the particularity of being defined by a physical network (unlike the tunnelled protocols that have a set of ranges but without any network association). During the PTG it was mentioned the possibility of creating an API to handle the VLAN network ranges. After a quick talk with Slawek, he pointed me to the network segment ranges API [1]. This API allows the creation of new segment ranges not configured in the static "ml2_type_vlan.network_vlan_ranges" configuration variable (again, I'm focusing only on the VLAN type network). It is possible, for example, not to define any range statically and then create, per physical network, a set of allowed ranges. Some considerations: * The configured value in "ml2_type_vlan.network_vlan_ranges" (static configuration) will be read during the Neutron API start process and will create a "default" network segment range register. This register is immutable and cannot be modified nor deleted. * Each network segment range will have a "used" reference, that is the number of VLAN IDs assigned in this range. If there is one single network with a VLAN ID on this range, it won't be possible to update or delete this network segment range register. * These ranges apply to the auto allocation functionality. If an admin (that is by default an admin privilege configured in the policies) creates a network with a defined VLAN outside any set of ranges, that will be allowed. I hope that will help you. And if you have a proposal to upgrade the current API, that will be welcome. Regards. [1] https://docs.openstack.org/neutron/latest/admin/config-network-segment-rang…

1 0

[glance] Epoxy PTG summary
by Abhishek Kekane 28 Oct '24

28 Oct '24

Hi Team, We had our Epoxy virtual PTG between 21st to 25th October 2024. Thanks to everyone who joined the virtual PTG sessions. Using gmeet we had lots of discussion around different topics for glance, glance + cinder, eventlet removal etc. You can find the topics discussed in etherpad [1] with Notes from the session and which also includes the recordings of each discussion. Here I am going to highlight a few important topics which we are going to target this cycle. # Property protection Access to extra properties through Glance’s public API calls may be restricted to certain sets of users, using a property protections configuration file. It can be either role based protection or policy based protection. During this session we have decided to conduct a survey to verify who is using this feature, whether they are using role based or policy based protection. Can we remove either of the protection and do the refactoring? If not then glance will update the documentation for property protection to include reserved properties. Recording: https://drive.google.com/file/d/1WrhkM2dVk6E8KQVDJCt1Bc2aaHi-txGJ/view?usp=… # Periodic job for cache-cleaner and cache-pruner At the moment we have command line utilities to perform these jobs which need to be configured as cron jobs. If we move this under service as a periodic job we can eliminate maintenance of these command line tools and avoid external configuration processes. In this session we decided to introduce two new admin only API calls which will help to clean the cached images. This will help to have an easier deployment process. Recording: https://drive.google.com/file/d/1pmYmVD72zl97gdXRWegYZskDvP50wi6E/view?usp=… # New API to list cached images on all glance nodes As we now have a centralized database as one of the cache drivers, we can now have an api to list images cached on different nodes to give end user overall view of the cache. The actual plan was to list all images cached across the nodes but during the session we found that it is better to get a picture of a single image cached on the nodes. Here we will add a new API call (admin only?) which will accept image id as an input and return the node details where the said image is cached. Recording: https://drive.google.com/file/d/1fv0tBEWbtHcKKfMVLVwc8lPHwN5OM3uh/view?usp=… # Distributed image download if filesystem is enabled We already use distributed imports to avoid configuring shared staging areas across glance nodes, similarly we can use this mechanism to download the image if the filesystem driver is used and the operator wants to avoid configuring shared filesystem in case we have more than one glance node. We have decided to introduce a new configuration option `distributed_download_strategy` with options `None,proxy,redirect`. If it is None then we will return a 404 File Not Found error to the user. If it is a proxy then we will proxy the request to the node where image data is actually present. If it is redirect then we will return a fully accessible image url to the user which he can use to issue a new download request. Recording: https://drive.google.com/file/d/1W5mwifcEfne5HJfNRJ84EyuYGc7NCDJK/view?usp=… # Remove deprecated features/options * sqlite cache driver - Remove in F cycle * Glance scrubber - Remove in Epoxy * Windows support - Sync with gmann to check whether we should remove it now or follow deprecation process of 2 cycles * glance-cache-prefetcher, glance-cache-cleaner, glance-cache-pruner - Deprecate these in Epoxy and remove those in G cycle Recording: https://drive.google.com/file/d/1DFmpy6JRIZw9OTPaw19FD4bW7SR36iNF/view?usp=… # Deprecating python-glanceclient We need to merge pending patches in OSC on priority basis. We will be announcing the deprecation of glance client on openstack mailing list which will help stakeholders to plan OSC migration in next two cycles. Plan is to deprecate the shell in Epoxy with a deprecation warning to each command and then remove the glance client in G cycle. Recording: https://drive.google.com/file/d/1gL6oEpoRL1igjervLFLkNE6eHVWJp3A2/view?usp=… # Horizon Feature Gaps - Cross project session with Horizon team Horizon team is stepping up to provide user interface for new features introduced in glance. You can find the details about it in etherpad [2]. Glance team will help them to migrate from glance-client to OSC and verify and review new user interface. Recording: https://drive.google.com/file/d/12AeTnaQytUhG_tHWN06G78kFYg2YuSGl/view?usp=… # S3 support In this session we have discussed the feature gaps in S3 and other backends of glance. Plan is to find out how we can configure S3 store using swift or ceph and then add a CI job upstream to find out if there are any failures in current supported features. Recording: https://drive.google.com/file/d/1MKEGh7qR9HgCnlhsa4MDjH6Rpl0OaUXe/view?usp=… # Add Glance as first-line defense for image format attacks Here we want glance to act as primary defender for any kind of security vulnerability. You can find the initial idea about it in glance spec [3]. In this session we discussed; * adding new disk-format `gpt` * leaves raw as unstructured data which will be rejected by nova to boot from it. * Migration of existing images from raw to gpt * LUKS inspector (see image encryption session for more details) * Impact on image conversion plugin if gpt disk-format is added Recording: https://drive.google.com/file/d/1xWM7IMCuZ_UenlMUd_rh1KmeuLY6bzSS/view?usp=… # Image Encryption(cross project session with cinder and nova) Here we have discussed the changes in current proposed design with CVE-2024-32498 and how to inspect an image if it is encrypted. Plan is to re-propose the spec with current changes like; * Keep the qcow format as it is and discover the encrypted qcow through metadata (encryption_key_id) * Introduce LUKS as new disk-format Recording: To be updated* # New Location APIs Adoption in Nova and Cinder As Image encryption cross project session took much of the time, we have decided to discuss this topic in the following weekly meeting. Plan is to make nova and cinder use this new location API call so that we can get rid of split deployment (glance internal vs glance external) in this cycle. # Eventlet removal It is a community goal to remove eventlet from OpenStack by 2027.2 (see https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html) Eventlet is causing issues with every new CPython release (at the time of writing, tests are hanging on CPython 3.13). This is a SLURP release so we cannot have "big" changes, but: * we can get "easy" fixes out of the way * we should start working on this so we can merge patches right at the start of the F cycle Glance planning for eventlet removal: * WSGI server Make sure all jobs are running on uwsgi Deprecate/disable the wsgi eventlet server Remove it in 3/4 cycles * Plan for epoxy cycle Deprecate eventlet/wsgi related config option Devstack to by default configure uwsgi Migrate jobs to use uwsgi Migrate from eventlet threadpool to native or io thread pool * Plan for F cycle Fix any issues if occurs for uwsgi * Plan for G cycle Remove wsgi functionality Recording: https://drive.google.com/file/d/1J5nYOCwp2MrrFZAIIvg1YGH4UnaVpk6s/view?usp=… Apart from above topics you can find other miscellaneous topics discussed in PTG etherpad [1]. If you have any questions/suggestions then please join us in our weekly meeting (each Thursday #openstack-meeting irc channel at 1400 UTC). *PS: Since Image encryption was a cross-project session with cinder and nova, once cinder publishes recordings I will update it here on thread. [1] https://etherpad.opendev.org/p/oct2024-ptg-glance [2] https://etherpad.opendev.org/p/horizon-feature-gap [3] https://review.opendev.org/c/openstack/glance-specs/+/925111 Thanks and Regards, Abhishek Kekane

1 0

[tacker] Cancelling weekly IRC meeting
by Yasufumi Ogawa 27 Oct '24

27 Oct '24

Hi tacker team, Thank you for joining the vPTG last week. Since I'm not available in the afternoon tomorrow, so going to cancel the meeting. Thanks, Yasufumi

1 0

Request for Scientific Resources on OpenStack Cost Efficiency Compared to Public Cloud Providers
by HAMMOUMI TARIK 27 Oct '24

27 Oct '24

Dear OpenStack Community, I am currently working on a case study as part of my software engineering course, where I aim to examine the cost efficiency of OpenStack as a private infrastructure solution in comparison to public cloud services such as AWS, Google Cloud, and Azure. My goal is to determine whether OpenStack can be more cost-effective in the long term, particularly for organizations that rely heavily on sustained workloads. In my initial research, I encountered challenges finding scientific studies or published articles that provide a clear, data-driven comparison between the Total Cost of Ownership (TCO) of OpenStack and that of leading public cloud providers. During a recent discussion with my professor, they emphasized the importance of using concrete, peer-reviewed sources and precise financial comparisons to validate claims about cost savings or trade-offs between owning physical resources and cloud-based infrastructure. I would appreciate it if anyone could point me to peer-reviewed research, published case studies, or white papers that analyze OpenStack’s TCO, especially in comparison to major public cloud providers. Insights on maintenance and operational costs for OpenStack in practical, high-utilization scenarios would be particularly helpful. Thank you in advance for any assistance or direction on reliable resources. I look forward to contributing to this ongoing discussion and hopefully sharing insights from my findings with the community. Best regards, Tarik HAMMOUMI (https://itstarik.tech)

1 0

The CFP for OpenInfra Days North America 2025 closes in ONE WEEK!
by Helena Spease 25 Oct '24

25 Oct '24

Hi everyone, OpenInfra Days North America[1] is returning in March 2025 and will be co-located with SCaLE in Pasadena, CA! SCaLE takes place March 6-9, 2025. OpenInfra Days North America will take place 2 days during that timeframe, but the exact timing has not been determined. The CFP[2] for OpenInfra Days North America 2025 is currently open until NOVEMBER 1, 2024! OpenInfra Days North America is being jointly organized by the OpenInfra communities from the United States, Mexico, and Canada. We are looking for sessions that will cover open-source projects used to build data center infrastructure including OpenStack, Kubernetes, Linux, Kata Containers, StarlingX, Ceph, and many more! A few hot topics we would love to hear about include: - VMWare Migration to OpenStack - OpenStack and Kubernetes - AI and HPC workloads - Zuul CI - Baremetal & Ironic - OpenStack 2024.2, Dalmatian - OpenStack upgrade/SLURP demos - OpenStack cost/labor savings - Government and healthcare compliance stories Registration[3] and sponsorship[4] opportunities are also now available! If you have any questions about the event, please feel free to let me know. Cheers, Helena helena(a)openinfra.dev Community Programs Coordinator The OpenInfra Foundation [1] https://www.socallinuxexpo.org/scale/22x/events/open-infra-days [2] https://www.socallinuxexpo.org/scale/22x/call-presenters [3] https://register.socallinuxexpo.org/reg6/ [4] https://openinfra.dev/events/sponsorship#OIDNA

1 0

[largescale-sig]scaling story
by Alex Song (宋文平) 25 Oct '24

25 Oct '24

Hi everyone, we want to share our scaling story. Please refer the link: https://etherpad.opendev.org/p/large-scale-inspur Thanks, Alex Song

6 8

[i18n] i18n SIG PTG Schedule & Key Topics for Discussion
by Seongsoo Cho 24 Oct '24

24 Oct '24

Hi i18n SIG Team I’m pleased to announce the i18n SIG schedule for the upcoming PTG. Here are the details: Dates: • October 23rd: 13:00 - 15:00 UTC • October 24th: 13:00 - 15:00 UTC We encourage you to check the meeting minutes and contribute to the discussion by accessing the following Etherpad: https://etherpad.opendev.org/p/oct2024-ptg-i18n Key Discussion Topics: 1. Weblate Migration: We will discuss the progress and steps needed for migrating to Weblate. 2. Enhancing Translation Engagement: We’ll focus on strategies to improve the engagement and translation rates for OpenStack projects. Potential ideas, such as incorporating LLM (Large Language Models), will also be explored. We look forward to your participation and contributions to these discussions. Best Regards Seongsoo -- Seongsoo Cho OpenStack Korea User Group / Community Leader IRC #seongsoocho

2 2

[barbican][vault]
by Andy Speagle 24 Oct '24

24 Oct '24

Hey Folks, I'm having some trouble getting barbican to generate and/or store keys in our hashicorp vault in our charmed openstack environment. I have issues with both TPM and SSL related keys. I get extremely generic errors back that I can't seem to figure out how to troubleshoot: 'Traceback (most recent call last):\n File "/usr/lib/python3/dist- packages/nova/conductor/manager.py", line 688, in build_instances\n scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist- packages/nova/scheduler/utils.py", line 998, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExce eded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 0c50b2d7-8ac5-41a2-90ab-1e33f7aa1c22. Last exception: Key manager error: Internal Server Error: Secret creation failure seen - please contact site administrator.\n' I'm using barbican 14.0.2 in yoga/stable ... anyone have an ideas how to troubleshoot this? -- Andy Speagle

2 1

[cloudkitty] 2025.01 PTG summary
by Rafael Weingärtner 23 Oct '24

23 Oct '24

Dear all, The CloudKitty project had a great PTG meeting [1]. Thanks to the OpenInfra Foundation for holding it. Also, thanks to all who have dedicated precious time to attend. We started the meeting by revisiting the latest release and discussing the review format we have adopted in the past years. Looking at the result of our process, we decided to maintain it, as it seems to bring results, and concentrate our efforts on getting CloudKitty better and evolving over time. Furthermore, as the current CloudKitty team is a bit small, we have decided to focus on the following topics for the next cycle. - State of Elasticsearch - We want to make sure the CloudKitty CI tests Elasticsearch v8, driver was last tested on v6/7. - OpenSearch/Elasticsearch datastreams -- It looks like this feature is present in both OpenSearch and Elasticsearch. Therefore, we will make an effort to concentrate on these two drivers, instead of replicating their codebase, and separate between them just the differences. - Epoxy (2025.1) supported runtimes -- we will drop the unsupported Python releases/versions. - oslo.policy RBAC new defaults -- We will need to address these new supports in the upcoming release. - Horizon dashboard improvements -- Borrow ideas from Catnip? https://github.com/ObjectifLibre/catnip, and maybe discuss with other agents from the community. In summary that is the outcome of our small, but efficient PTG meeting. Let's keep improving and making CloudKitty better. If I missed something, please do not hesitate to reply back. Also, if somebody needs any help, you can always ping me :) See you guys. Have a nice week! [1] https://etherpad.opendev.org/p/oct2024-ptg-cloudkitty -- Rafael Weingärtner

1 0

Openstack haproxy customization
by Gaurav Sharma 23 Oct '24

23 Oct '24

I have deployed openstack multinode using kolla-ansible ,you know when we run deploy command then haproxy configuration generated by default, now i want to customize the by default configuration by custom file(if create). Please provide better steps to customize haproxy. i have tried many steps but did't get success, one thing i have docker not systemctl .

1 0

[ironic] Core team split into reviewers/approvers
by Jay Faulkner 23 Oct '24

23 Oct '24

OpenStack & Ironic community, Ironic has not added a core reviewer in years. We've had new contributors during this time, but none who has, as yet, risen to the level of being given core review access -- which reflects how steep the learning/responsibility curve is to being a core on an OpenStack project. I'd like to propose, on behalf of many Ironic cores and our PTL Riccardo, a new core review structure for Ironic projects. The goal being to provide an on-ramp for newer cores into the community. The existing ironic-core group will be removed and replaced with two new groups: - ironic-reviewers will contain contributors permitted to perform core reviews (+2/-2) against patches in ironic projects. This group will be initially populated with two new code reviewers (should they accept), CID and cardoe. - ironic-approvers will contain contributors permitted to both perform core reviews and workflow (+A) patchsets. This group will be initially populated with the existing membership of ironic-core. I hope that by providing a middle point on the route to being an approver, we'll be able to extend trust to new contributors more quickly and give them incentive to code review more often -- since their reviews will be able to have a positive impact on landing code more quickly. I've proposed a change here: https://review.opendev.org/c/openstack/project-config/+/931991which makes the first step in adjusting ACLs to split the groups. I've also as part of this change unified more ironic-related projects under the ironic core review banner after auditing those core groups and seeing some inactive contributors in them. Ironic cores have already discussed this privately, and we have a lazy consensus to perform this change. This email serves as both notice to the larger community and as a formal way to gain consensus. If there are any questions or concerns please let us know. Thanks, Jay Faulkner

8 12

[kolla-ansible][haproxy][2024.1] HAProxy template error
by Satish Patel 22 Oct '24

22 Oct '24

I am deploying kolla-ansible latest release 2024.1 on Ubuntu 22.04 and getting strange error at following task When I am running kolla-ansible -i multinode deploy TASK [include_role : glance] ***************************************************************************************************************************************************************** task path: /usr/local/share/kolla-ansible/ansible/site.yml:168 redirecting (type: modules) ansible.builtin.firewalld to ansible.posix.firewalld TASK [haproxy-config : Copying over glance haproxy config] *********************************************************************************************************************************** task path: /usr/local/share/kolla-ansible/ansible/roles/haproxy-config/tasks/main.yml:2 The full traceback is: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/ansible/plugins/action/template.py", line 152, in run resultant = templar.do_template(template_data, preserve_trailing_newlines=True, escape_backslashes=False, overrides=overrides) File "/usr/local/lib/python3.10/dist-packages/ansible/template/__init__.py", line 1010, in do_template res = myenv.concat(rf) File "/usr/local/lib/python3.10/dist-packages/ansible/template/native_helpers.py", line 83, in ansible_concat return ''.join([to_text(v) for v in nodes]) File "/usr/local/lib/python3.10/dist-packages/ansible/template/native_helpers.py", line 83, in <listcomp> return ''.join([to_text(v) for v in nodes]) File "<template>", line 436, in root File "/usr/local/lib/python3.10/dist-packages/ansible/template/__init__.py", line 381, in call return super().call(obj, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/jinja2/runtime.py", line 303, in call return __obj(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/jinja2/runtime.py", line 768, in __call__ return self._invoke(arguments, autoescape) File "/usr/local/lib/python3.10/dist-packages/jinja2/runtime.py", line 782, in _invoke rv = self._func(*arguments) File "<template>", line 193, in macro File "/usr/local/lib/python3.10/dist-packages/ansible/template/__init__.py", line 295, in wrapper ret = func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/jinja2/filters.py", line 1030, in do_format return soft_str(value) % (kwargs or args) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 165, in __mod__ return self.__class__(super().__mod__(value)) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 370, in __str__ return str(self.escape(self.obj)) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 235, in escape rv = escape(s) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 45, in escape return Markup(_escape_inner(str(s))) SystemError: <built-in function _escape_inner> returned NULL without setting an exception failed: [os2-bos-ctrl01] (item={'key': 'glance-api', 'value': {'container_name': 'glance_api', 'group': 'glance-api', 'host_in_groups': True, 'enabled': True, 'image': ' registry.example.com/kolla/glance-api:2024.1-ubuntu-jammy', 'environment': {'http_proxy': '', 'https_proxy': '', 'no_proxy': 'localhost,127.0.0.1,10.0.52.11,10.0.52.10'}, 'privileged': False, 'volumes': ['/etc/kolla/glance-api/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'glance:/var/lib/glance/', '', 'kolla_logs:/var/log/kolla/', '', ''], 'dimensions': {}, 'healthcheck': {'interval': '30', 'retries': '3', 'start_period': '5', 'test': ['CMD-SHELL', 'healthcheck_curl http://10.0.52.11:9292'], 'timeout': '30'}, 'haproxy': {'glance_api': {'enabled': True, 'mode': 'http', 'external': False, 'port': '9292', 'frontend_http_extra': ['timeout client 6h'], 'backend_http_extra': ['timeout server 6h'], 'custom_member_list': ['server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5', '']}, 'glance_api_external': {'enabled': True, 'mode': 'http', 'external': True, 'external_fqdn': 'openstack-bos-2.example.com', 'port': '9292', 'frontend_http_extra': ['timeout client 6h'], 'backend_http_extra': ['timeout server 6h'], 'custom_member_list': ['server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5', '']}}}}) => { "ansible_loop_var": "item", "changed": false, "item": { "key": "glance-api", "value": { "container_name": "glance_api", "dimensions": {}, "enabled": true, "environment": { "http_proxy": "", "https_proxy": "", "no_proxy": "localhost,127.0.0.1,10.0.52.11,10.0.52.10" }, "group": "glance-api", "haproxy": { "glance_api": { "backend_http_extra": [ "timeout server 6h" ], "custom_member_list": [ "server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5", "" ], "enabled": true, "external": false, "frontend_http_extra": [ "timeout client 6h" ], "mode": "http", "port": "9292" }, "glance_api_external": { "backend_http_extra": [ "timeout server 6h" ], "custom_member_list": [ "server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5", "" ], "enabled": true, "external": true, "external_fqdn": "openstack-bos-2.example.com", "frontend_http_extra": [ "timeout client 6h" ], "mode": "http", "port": "9292" } }, "healthcheck": { "interval": "30", "retries": "3", "start_period": "5", "test": [ "CMD-SHELL", "healthcheck_curl http://10.0.52.11:9292" ], "timeout": "30" }, "host_in_groups": true, "image": " registry.example.com/kolla/glance-api:2024.1-ubuntu-jammy", "privileged": false, "volumes": [ "/etc/kolla/glance-api/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "/etc/timezone:/etc/timezone:ro", "glance:/var/lib/glance/", "", "kolla_logs:/var/log/kolla/", "", "" ] } }, "msg": "SystemError: <built-in function _escape_inner> returned NULL without setting an exception" }

1 1

[zaqar]Proposing Takashi Kajinami as Zaqar core member
by hao wang 22 Oct '24

22 Oct '24

Hello, everyone, I would like to propose Takashi Kajinami for the OpenStack Zaqar core member. Takashi has contributed to the Zaqar project since 2021 and has done a great job of improving this project. It is very important to Zaqar to keep it stable and reliable. I am so excited that we can have such a great contributor and I believe we can do more valuable work to the Zaqar project. Thank you for your contributions. Regards, Hao Wang -- Best Wishes For You!

1 0

[nova][ptg] Nova sessions on Wednesday
by Sylvain Bauza 22 Oct '24

22 Oct '24

Hi, Given most of the nova cores will attend the eventlet-removal session at 1300UTC, we'll defer the nova sessions to start at 1400UTC instead of 1300UTC. Please stay online in #openstack-nova IRC channel or please take a look at https://ptg.opendev.org/ptg.html to see the exact time when we spawn again the nova conversations. Thanks, -Sylvain (who just unbooked the austin room for Wed-B1)

1 0

[Horizon] Cancelling weekly meeting tomorrow
by Tatiana Ovchinnikova 22 Oct '24

22 Oct '24

Hello team, Since we have vPTG sessions, I'm cancelling our weekly meeting tomorrow. With kind regards, Tatiana

1 0

[dev][cinder] Bubbling up maintenance status on a volume from a third-party backend
by David Pineau 22 Oct '24

22 Oct '24

Hello Openstack and Cinder team, I'm currently working on the storage management layer of our company's Openstack platform. As we manage the storage ourselves, we may run background operations, which are akin to a maintenance, on individual volumes, or part of the infrastructure. As a bit of context, this infrastructure will be both exposed to external customers, as well as internal departments, and as such, we are looking for a refined user experience where an error is not a simple "an error occurred in service X", but something potentially actionable by whoever encounters it. At the moment in the Cinder documentation (and what I could find on the discuss archive), there seems to be no way for a Cinder Driver and its backing services to: - Tell Cinder that a specific Volume is undergoing a backend maintenance and should be considered unavailable - raise a relevant error to Cinder about ongoing Operations affecting a Volume within its backend, that cinder could properly react to As a first step, I wanted to check if you, the community, had ever considered this issue (or at least we consider it to be one). We'd be very happy if you had recipes or pieces of advice to share with us on how to handle this. If nothing is available or known, how could we help in bringing such an improvement to Cinder ? Kind regards, -- David Pineau (joa)

3 5

[Openstack-ansible]- Need to delete VM with its OS disk even if it has active snapshot available.
by Danish Khan 22 Oct '24

22 Oct '24

Dear All, When I delete a VM then the OS disk of the VM remains even when I have marked the Volume to get deleted on VM deletion. This is happening only when we have a snapshot of the OS disk or snapshot of the complete VM available. Can we configure it in a way then either that the VM should get deleted completely with OS disk even if we have the snapshot available or the VM deletion task should fail if the snapshot is available. Regards, Danish

1 0

[nova][libvirtd] internal error: unable to execute QEMU command 'query-named-block-nodes'
by Satish Patel 22 Oct '24

22 Oct '24

Folks, I have a ceph cluster and recently I configured rbd-mirror to replicate all data to remove ceph cluster for disaster recovery. Yesterday for POC we did a hard cutover on ceph and point openstack to the new cluster. All other vms came back up fine but 2 vms stuck in this error state in libvirt logs 2024-01-31 22:44:37.591+0000: 474597: error : qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied If it's a wider issue than it should impact all the vms but why only two VM stuck and not starting and libvirt giving me this error in logs nova-compute logs also showing same error 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] [instance: 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from powering-on on failure for instance. 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied

3 8

[swift][ptg] Ops Feedback Session - 23 October at 14:00 UTC
by Tim Burke 21 Oct '24

21 Oct '24

As we've done in PTGs past, we're getting devs and ops together to talk about Swift: what's working, what isn't, and what would be most helpful to improve. We're meeting on meetpad (https://meetpad.opendev.org/p/swift-epoxy-ops-feedback) on Wednesday at 14:00UTC -- if you run a Swift cluster, we hope to see you there! Even if you can't make it, we'd appreciate it if you can offer some feedback on the etherpad (https://etherpad.opendev.org/p/swift-epoxy-ops-feedback) This has always been a highlight at every PTG for the swift developers. Have your say and help make the next Swift the best one yet! Tim

1 0

[all][rbac] Canceling policy popup team tomorrow meeting
by Ghanshyam Mann 21 Oct '24

21 Oct '24

Hello Everyone, Due to vPTG weeks, I am cancelling the policy pop-up tomorrow meeting. https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… -gmann

1 0

[cinder] 2025.1 PTG reminder
by Jon Bernard 21 Oct '24

21 Oct '24

Hello all, We covered the schedule in last week's meeting but I wanted to post it here for reference. For Cinder, we are meeting Wednesday, Thursday, and Friday of this week from 14:00 UTC until 17:00 UTC. I have consolidated and updated our etherpad [1] to include both a tentative schedule that includes cross-project session commitments and the set of proposed topics. As we did in the previous PTG, we'll be using the same google meet link and the sessions will be recorded. Everything should be noted in the etherpad. If you notice something is missing please add it or reach out with any questions. See you all in a couple of days! [1] https://etherpad.opendev.org/p/epoxy-ptg-cinder -- Jon

1 0

[openstack-helm] 2025.1 PTG reminder
by Vladimir Kozhukalov 21 Oct '24

21 Oct '24

Hi dear helmers, This is a reminder that we have a PTG session scheduled at 15:00-18:00 UTC on Wednesday 2024-10-23 [1]. This time we have quite a lot to discuss (and this is good) so I booked a 3 hours session (instead of our usual 2 hours). If you have something to discuss please put topics to the Etherpad [2] [1] https://ptg.opendev.org/ptg.html [2] https://etherpad.opendev.org/p/oct2024-ptg-os-helm -- Best regards, Kozhukalov Vladimir

1 0

Spot Instances
by Mia 21 Oct '24

21 Oct '24

Hello everyone, I am working on a project in the OpenStack environment. More precisely, I am looking into the implementation of spot instances as a cost optimization strategy and better resource utilization. If any community members could share their experiences and insights related to the following areas, it would be greatly appreciated. Implementation Strategies: How did you go about implementing spot instances in your cluster? Challenges Faced: Mention some of the challenges or limitations. Best Practices: Do you have any recommendations or best practices that you would like to suggest for seamless deployment? Thank you very much for your time and support.

8 11

[all][designate][manila] Grenade job failures caused by GLOBAL_VENV=True
by Takashi Kajinami 21 Oct '24

21 Oct '24

Just in case you are not aware of the situation. The change to enable GLOBAL_VENV in grenade job was recently merged[1] in core grenade. As we saw when the same was introduced in devstack some time ago, some project got their (grenade) jobs broken and need a few adjustments. [1] https://review.opendev.org/c/openstack/grenade/+/930507 So far I'm aware that the change affected manila and designate, which I'm fixing by the changes below. Please be careful in case you see grenade job failing in your project. designate: - https://review.opendev.org/c/openstack/designate/+/932697 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2 - https://review.opendev.org/c/openstack/designate/+/932697 devstack: Fix manage.py execution with GLOBAL_VENV This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches manila: - https://review.opendev.org/c/openstack/manila/+/932671 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2 - https://review.opendev.org/c/openstack/manila/+/932665 devstack: Fix missing manila command This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches Thank you, Takashi -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

4 4

[ptg][sunbeam] Epoxy cycle PTG 2025.1 schedule
by Guillaume Boutry 21 Oct '24

21 Oct '24

Hi All, OpenStack PTG has been scheduled on Oct 22nd and Oct 24th 2024 [1] and I have scheduled the following time slots for the OpenStack Sunbeam project. Tuesday 22 October 2024 0600 UTC to 0800 UTC Thursday 24 October 2024 1400 UTC to 1600 UTC The meeting links and details are in ptg link [2] or etherpad [3]. Feel free to add the topics of interest to discuss in the etherpad [3]. [1] https://openinfra.dev/ptg/ [2] https://ptg.opendev.org/ptg.html [3] https://etherpad.opendev.org/p/oct2024-ptg-sunbeam Looking forward to seeing you next week! Regards, Guillaume Boutry

1 0

[tacker] Cancel IRC meeting this week
by Yasufumi Ogawa 20 Oct '24

20 Oct '24

Hi tacker team, Since we have vPTG sessions, I'm going to cancel weekly IRC meeting tomorrow. Thanks Yasufumi

1 0

[ptl][tc][all] Community Leaders meeting at the PTG
by Goutham Pacha Ravi 20 Oct '24

20 Oct '24

Hello PTLs, Project Liaisons, Core Reviewers, and invested maintainers, The TC is hosting a cross project session with all OpenStack project leaders at 1400 UTC today (2024-10-21). Please see the PTG schedule at https://ptg.opendev.org/ptg.html and the TC PTG etherpad for details: https://etherpad.opendev.org/p/oct2024-ptg-os-tc We look forward to seeing you all there! P.S: while you are there, you may also be interested in the cross project goal proposal/discussion for migrating away from Eventlet (1500 UTC); and other community-wide topics in the TC PTG room. Thanks, Goutham Pacha Ravi OpenStack TC Chair

1 0

[neutron] Bug report week 42th (14 Oct - 20 Oct)
by Rodolfo Alonso Hernandez 20 Oct '24

20 Oct '24

Hello Neutrinos: This is the bug report of the 42th week (14 Oct - 20 Oct). High: * https://bugs.launchpad.net/neutron/+bug/2083832: "[OVN] Host_id value on lsp for router gateway is not updated on failover". Assigned. In progress: https://review.opendev.org/c/openstack/neutron/+/931632. * https://bugs.launchpad.net/neutron/+bug/2084977: "Deleting a security group fails with "Cannot find Port_Group with name=pg_<id>" when neutron-api is run by httpd + mod_wsgi". *Not assigned*. Medium: * https://bugs.launchpad.net/neutron/+bug/2084446: "Scaling down neutron-openvswitch-agent don't remote it's tunnel endpoints". *Not assigned*. * https://bugs.launchpad.net/neutron/+bug/2084516: " "Tagging" extension POST for QoS policies doesn't work". *Not assigned*. * https://bugs.launchpad.net/neutron/+bug/2084536: "OVN: Connecting internal tenant VLAN networks to virtual routers when external (BM) ports are also present". *Not assigned*. * https://bugs.launchpad.net/neutron/+bug/2084782: "[OVN] VLAN/flat LRP should be updated when the router networks are updated". Assigned. * https://bugs.launchpad.net/neutron/+bug/2084906: " Hardcoded keepalived configuration parameters". Assigned. In progress: https://review.opendev.org/c/openstack/neutron/+/932716. RFE: * https://bugs.launchpad.net/neutron/+bug/2084894: " [RFE] Support injection of local NTP servers into subnets". * https://bugs.launchpad.net/neutron/+bug/2084905: "[RFE][OVN] Make use of the new flag 'always_tunnel' to tunnel the E/W traffic between logical switches". Regards.

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-23)
by Goutham Pacha Ravi 20 Oct '24

20 Oct '24

Hello Stackers, This week, the OpenStack community is convening virtually to set the agenda for the 2025.1 "Epoxy" release and beyond [1][2]. As of today, there are 33 different discussion tracks spread over five days. Now's as good a time as ever to review the Community Code of Conduct as we step into this event [3]. In the past week, the North American section of the OpenInfra community gathered in Indianapolis (and virtually) for a two-day conference. I had the privilege to attend in person and chat with numerous users, operators, and contributors. The community is as energetic as ever, as evidenced by the participation we saw in the leadership meet-and-greet event [4]. Amy Marrich (spotz) organized this event, and we had representation from the OpenInfra Board, Zuul, OpenStack TC, SIGs, and PTLs. The audience learned the essence of our "Open Community," how elections are conducted, what different community leadership roles are, and how one can participate in OpenInfra governance. We brainstormed how to revive projects with dwindling contributors and recapped takeaways from the "Bridging the Gap" and Security SIG-related discussions. Prof. Corey Leong (profcorey) from Valencia College volunteered to propose a "Higher Education" Special Interest Group focused on increasing awareness and creating learning material for university students interested in joining the OpenStack community. I'll share a link with video recordings of this conference in a future post. === Weekly Meeting === Dmitriy Rabotyagov (noonedeadpunk) chaired the weekly IRC meeting [5] of the OpenStack Technical Committee last week. We had fewer attendees than usual, but it was a productive discussion nevertheless. We prepped the TC's PTG etherpad [6] with topics and timings for the discussion. As is tradition, please allow flexibility in terms of the discussion schedule. Discussions may start a bit later (or earlier) than planned, and we'll have to be dynamic to accommodate scheduling conflicts, as many contributors are involved across multiple topics. In lieu of the PTG, the weekly IRC meeting scheduled for 2024-10-22 has been canceled [7]. We briefly discussed the leadership appointment patches concerning the Watcher and Mistral project teams. These are still being reviewed, and we'll try to get them closed out after the PTG week. We also discussed grenade job changes that Ghanshyam Mann (gmann) and other community members have been pursuing [8]. Concerns were raised about projects (like Kolla and Loci) publishing container images under the OpenStack name without a clear security policy. The need to clarify OpenStack's support and security posture on such deliverables was discussed. We have more to discuss on this topic at the PTG and beyond. The next weekly IRC meeting for the OpenStack Technical Committee is on 2024-10-29. The agenda is up early [9], but I'll write to you once again before it is formalized. I look forward to seeing you all at the PTG and possibly seeding more topics for this weekly meeting afterward. === Governance Proposals === ==== Open for Review ==== Add Cinder Huawei charm | https://review.opendev.org/c/openstack/governance/+/867588 Appoint Ke Chen as PTL for Watcher | https://review.opendev.org/c/openstack/governance/+/932419 Adding Axel Vanzaghi as PTL for Mistral | https://review.opendev.org/c/openstack/governance/+/927962 Retire kuryr-kubernetes and kuryr-tempest-plugin | https://review.opendev.org/c/openstack/governance/+/922507 Propose a pop-up team for eventlet-removal | https://review.opendev.org/c/openstack/governance/+/931978 Propose to select the eventlet-removal community goal | https://review.opendev.org/c/openstack/governance/+/931254 === Upcoming Events === 2024-10-21 to 2024-10-25: OpenInfra Project Teams Gathering: https://openinfra.dev/ptg/ 2024-10-27: All Things Open: https://allthingsopen.org/ 2024-11-04: OpenInfra Board Meeting: https://board.openinfra.dev/ 2024-11-15: KubeCon + CloudNativeCon: https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] "Epoxy" PTG Schedule: https://ptg.opendev.org/ptg.html [3] OpenInfra Community Code of Conduct: https://openinfra.dev/legal/code-of-conduct [4] OpenInfra Leadership Meet-and-Greet: https://etherpad.opendev.org/p/leadership-na [5] TC Meeting IRC Log, 2024-10-15: https://meetings.opendev.org/meetings/tc/2024/tc.2024-10-15-18.00.log.html [6] TC PTG Etherpad: https://etherpad.opendev.org/p/oct2024-ptg-os-tc [7] TC IRC meeting 2024-10-22 is canceled: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [8] Grenade Job changes: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [9] TC Meeting Agenda, 2024-10-29: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[ptg][tacker] Epoxy vPTG planning
by Yasufumi Ogawa 20 Oct '24

20 Oct '24

Hi tacker team, I've booked slots for our two days vPTG sessions this week as I've announced on the previous weekly meeting. Please find the etherpad [1] and register your proposals for Epoxy release. - 23th Oct. 6am-8am UTC (3pm-5pm JST/KST) - 24th Oct. 6am-8am UTC (3pm-5pm JST/KST) [1] https://etherpad.opendev.org/p/oct2024-ptg-tacker Thanks, Yasufumi

1 0

[ptg][tacker] Epoxy vPTG planning
by Yasufumi Ogawa 20 Oct '24

20 Oct '24

Hi tacker team, I've booked slots for our two days vPTG sessions this week as I've announced on the previous weekly meeting. Please find the etherpad [1] and register your proposals for Epoxy release. - 23th Oct. 6am-8am UTC (3pm-5pm JST/KST) - 24th Oct. 6am-8am UTC (3pm-5pm JST/KST) [1] https://etherpad.opendev.org/p/oct2024-ptg-tacker Thanks, Yasufumi

1 0

[manila] 2025.1/Epoxy PTG planning
by Carlos Silva 20 Oct '24

20 Oct '24

Hello Zorillas and Interested stackers, The OpenInfra PTG will take place from Oct 21st to 25th. I have reserved some time slots for the Manila discussions [1]. If you would like to bring up any topics during the PTG, please add them to the PTG planning etherpad [2] by next Tuesday (Oct 16th). I intend to have the tentative schedule ready next Thursday and share it with you. Additionally, in case you have any time slot preferences, please let me know and I will do my best to accommodate them. [1] https://ptg.opendev.org/ptg.html [2] https://etherpad.opendev.org/p/epoxy-ptg-manila-planning Looking forward to it! Regards, carloss

1 1

Invitation to Discuss Collaboration Opportunities Between OpenStack and OceanBase
by 汪鹏(酒晨) 19 Oct '24

19 Oct '24

Dear OpenStack Team, I hope this message finds you well. My name is Peng Wang, and I am the Global Technical Evangelist at OceanBase. We have been following the remarkable achievements of the OpenStack community and are excited about the potential opportunities for collaboration between OceanBase and OpenStack. OceanBase is a high-performance distributed database, now open-source, designed to meet the demands of large-scale applications with exceptional scalability and reliability. We are actively exploring opportunities to seamlessly integrate it into the OpenStack ecosystem, proposing OceanBase as an alternative backend database option to enhance the flexibility and resilience of OpenStack deployments. We would like to invite your team to a virtual meeting to discuss this collaboration in more detail, including the technical aspects, shared customers, and potential benefits for the OpenStack ecosystem. The proposed meeting details are as follows: Date: 10/25/2024 Time: 13:00 UTC Duration: 1 hour Location: PTG Opendev Meeting Agenda: 1. Introduction to OceanBase and its architecture 2. Overview of the proposed pull request to integrate OceanBase with OpenStack 3. Introduction to the mutual benefits and shared interests between OceanBase and OpenStack 4. Exploring potential collaboration and future opportunities 5. Q&A and open discussion We believe this integration will be a significant step toward further empowering OpenStack with a robust and scalable database solution. We look forward to the opportunity to work closely with your team to bring this vision to life. Please let us know your availability for the proposed meeting or suggest an alternative time that works for you. We would be happy to adjust accordingly. Thank you for your time, and we look forward to your response. Best Regards, Peng Wang Global Technical Evangelist, OceanBase Tel: 86+ 13585603170 Email: jiuchen.wp(a)oceanbase.com from 阿里邮箱 macOS

2 2

[Telemetry] Takashi for Telemetry core
by Erno Kuvaja 19 Oct '24

19 Oct '24

Hi all, I'd like to address massive oversight from my side regarding Telemetry core by adding Takashi Kajinami to the group. Honestly based on all his involvement in the various Telemetry projects I just assumed he was in the core group already and I was pretty shocked that was not the case. I've already talked with him and would like to add him ASAP. As in by the end of the week, so if anyone has objections, please raise them now. Thanks Takashi for all your invaluable contributions and reviews, please keep them coming. - Erno "jokke" Kuvaja

2 1

[ptg][publiccloud-sig] PTG meetings for tthe publiccloud-sig
by Felix Kronlage-Dammers 19 Oct '24

19 Oct '24

Ahoi, as all other teams the publiccloud-sig is getting ready for the upcoming PTG as well :) We’re very much looking forward to the next week. The publiccloud-sig will meet for the PTG on Tuesday at 1600 UTC and on Wednesday at 0700 UTC. The regular etherpad[1] contains a list of items that we collected and that we will likely address. The tuesday slot should also allow for participants not in one of the EU and NZ timezones to join - we would love to get a nice group together. The PTG also serves as a good starting point to become more active in the publiccloud-sig :) See ya next week! felix [1] https://etherpad.opendev.org/p/publiccloud-sig

1 0

[watcher]Interest in a session at upcoming PTG?
by Ronelle Landy 18 Oct '24

18 Oct '24

Hello, There has been some renewed interest in the Watcher project. We wanted to reach out to see if the project PTL (or anyone else) would be willing to run/attend a session on Watcher at the upcoming PTG. Would it make sense to hold a dedicated session or discuss plans at one of the TC sessions? Thank you!

3 3

[qa][ptg] Virtual 2025.1 vPTG Planning
by Martin Kopec 18 Oct '24

18 Oct '24

Hello everyone, here is [1] our etherpad for the Epoxy/2025.1 virtual PTG. Please, add your topics there if there is anything you would like to discuss / propose for discussion ... You can also vote for time slots of the sessions so that they fit your schedule [2]. The number of slots will depend on the number of topics proposed in [1]. [1] https://etherpad.opendev.org/p/oct2024-ptg-os-qa [2] https://framadate.org/juHJSryIYHMRwdHA Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 1

[ptg][sdk][cli] PTG meeting of the SDK team
by Artem Goncharov 18 Oct '24

18 Oct '24

I have booked a slot for Monday 14:00-15:59 UTC for the SDK team (Monday is the day with least conflicts). Etherpad: [1] Topics: * state of the OpenAPI * codegenerator: state and next steps [1]: https://etherpad.opendev.org/p/oct2024-ptg-os-sdkcli See you there, Artem

1 0

[kolla-ansible] Problems with my lab . Many bugs with python
by Franck VEDEL 18 Oct '24

18 Oct '24

hello. I'm going crazy. Last year I set up a lab for students. The platform: ubuntu 22.04, kolla-ansible, antelope. Everything works perfectly by following the link https://docs.openstack.org/kolla-ansible/2023.1/user/quickstart.html We start with all-in-one in a virtual machine, then 20 hours later, a cluster of 3 nodes on DELL servers, with TLS, Ceph. I was super happy. I'm coming back to the subject a year later. I'm thinking about doing an update and so I start, still on ubuntu 22.04, by following the link https://docs.openstack.org/kolla-ansible/2024.1/user/quickstart.html well, it doesn't work. after hours of research, I resume an installation with Antelope. It doesn't work either. New research... and I come across a bug. Someone explains that you have to downgrade MarKupSafe to version 2.1.5. Okay, I'll do it. I start again with Caracal... and now, in the Horizon interface, we no longer see multiple domains. It doesn't work. New bug. I come back with Antelope and MarkupSafe == 2.1.5... it works, and my two domains are OK in Horizon. So I continue with Antelope ..... and now, when I try to activate TLS, it's impossible to do the "deploy". New bug but on rabbitmq. I'm going crazy. New bug but no solutions... The only solution I found is to restart a virtual machine that worked last year, do a pip freeze and in this year's installation, use last year's pip freeze. I can't use Caracal because the second domain is not visible. I can (maybe I'm not done yet) use Antelope but with last year's requirements.txt. How do I get out of this situation? How would you do it in my place knowing that I want to keep Ubuntu 22.04? Bobcat? I've already spent days on the subject. I don't have time anymore. That's why I'm asking for your help. Thansk a lot Franck

1 0

[neutron] Canceling weekly meetings tomorrow and next week
by Brian Haley 17 Oct '24

17 Oct '24

Hello Neutrinos: I'm going to cancel tomorrow's drivers meeting as well as our meetings next week as we have the PTG sessions. Please add any items there if you wish to discuss them [0]. Have a nice weekend! -Brian [0] https://etherpad.opendev.org/p/oct2024-ptg-neutron

1 0

[kolla][neutron][ovn] High ovs-dbserver CPU Usage with OVN in Kolla-ansible Large-Scale Deployment
by 韩光宇 17 Oct '24

17 Oct '24

Hi, I have a Wallaby cluster with 5 controller and 130 compute node. Host OS: Rocky 8.7 Deploy tool: Kolla-ansbile kolla_base_distro(container images): centos Network Mode：**OVN** After completing the deployment, the functionality for creating a single instance works as expected, and instances are created successfully. However, the process is extremely slow. When attempting to create instances in bulk, the operation fails on a large scale. During this time, I observed that the ovsdb-server is consuming close to 100% CPU usage. Even without any tasks being processed, the CPU usage remains consistently around 90%, just from the cluster's regular operations. I redeployed the cluster and switched the network mode to Open vSwitch (OVS), and everything works as expected with normal performance. Could anyone offer guidance on what might be causing this issue? I’m aiming to use the OVN mode with Kolla in a large-scale OpenStack cluster (100+ nodes). What configuration or adjustments would I need to make? Thank you in advance for your help! Han Guangyu

2 2