September 2024 - openstack-discuss

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 08 Dec '25

08 Dec '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 16

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 29 Oct '25

29 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

Problems with the Placement service
by Ron Gage 24 Jun '25

24 Jun '25

5 7

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

[oslo] python-consul2 is unmaintained
by Takashi Kajinami 21 Apr '25

21 Apr '25

Hello, I recently noticed that python-consul2 library[1], which is used by the consul coordination driver in tooz, hasn't been updated for 3+ years. Looking at issues in the project, there is an indication that the library is not compatible with ACL API in consul >= 1.11[2]. There is another fork created[3] and this includes fixes to support new API endpoints. Although the driver in tooz does not directly use the feature, I think it's better to discuss our plan to mitigate risks caused by the unmaintained library. As far as I can think of, there are three options now 1. Deprecate consul driver and remove it 2. Keep using python-consul2 now. At least basic functionality is proven to work by our functional tests (Note that the tests do not cover all configuration patterns) 3. Replace python-consul2 by py-consul Before we start the overall discussions I'd like to ask a few questions to the wider audience, as our inputs. - Is anyone using the consul coordination driver now ? - Does anyone have experience with python-consul2 and/or py-consul ? Note that this discussion affects the http messaging driver[4] currently proposed, because it also requires client library to communicate with Consul. Actually I noticed the situation when I noticed that the requirement check job is failing because the library is not part of global requirements list... Thank you, Takashi Kajinami [1] https://github.com/poppyred/python-consul2 [2] https://github.com/poppyred/python-consul2/issues/42 [3] https://github.com/criteo-forks/py-consul [4] https://review.opendev.org/c/openstack/oslo.messaging/+/912499 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 3

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

[tc][all][monasca ] Retiring the Monasca project
by Ghanshyam Mann 16 Feb '25

16 Feb '25

Hello everyone, As you might know, the Monasca project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Monasca retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

6 17

[ironic]About the OS supported by Diskimage-builder
by Junya Noguchi (Fujitsu) 03 Feb '25

03 Feb '25

To whom it may concern. My name is Junya Noguchi. I have some questions about the supported OS for Diskimage-builder operation. Please respond. References https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html Does it mean that the OS that says "work but receive only limited testing and support" works but is not supported? References https://docs.openstack.org/diskimage-builder/latest/ This document describes RHEL and other operating systems that are not supported in the previous question. Are these operating systems supported? Also, are there any plans to support more operating systems in the future? Best regards, Junya Noguchi

5 7

[glance][ops] proposal to remove the glance-scrubber
by Brian Rosmaita 20 Jan '25

20 Jan '25

Hello Glance users, The Glance project team wants to determine whether anyone is using the glance-scrubber utility [0]. If not, we are thinking of deprecating it this cycle to be removed in the appropriate future cycle. If you have not previously heard of the glance-scrubber, that's most likely because it's only useful for particular deployment configurations of which yours is not one. See the doc [0] for details if you are curious. Our suspicion is that the glance-scrubber may have outlived its usefulness for modern OpenStack configurations. (The last time I remember it being discussed at a PTG was during the snowstorm in Dublin, which I believe was the Rocky development cycle). On the other hand, the team also has some ideas for improving the scrubber, but there's no point doing that if it's not actually being used in the first place. So if removal of the glance-scrubber would negatively affect you, please respond to this email so we can get some data points. Thanks! [0] https://docs.openstack.org/glance/latest/cli/glancescrubber.html

2 2

[cinder][kubernetes] pvc stuck in reserved state
by Eugen Block 10 Jan '25

10 Jan '25

Hi, we have a couple of k8s clusters in our openstack environment (Victoria) and are facing some problems when pods get moved to different worker nodes. The PVCs get stuck in "reserved" state, and they only get attached after resetting the volumes to "available" state. First I thought it was due to the missing multiattach volume type which seems to work fine in our customer's cluster. I also found the "duplicate connector" error messages in the nova-compute.log: > Unable to update attachment.(Invalid volume: duplicate connectors > detected on volume But after adding the multiattach volume type and retyping the affected volumes, the PVCs still get stuck in "reserved" state while I don't see any error messages on the hypervisors anymore. So it seems that something has changed/improved, but now I don't have a trace where to look next. Could this be a cinder-csi driver issue or something? Our cinder backend is ceph (Pacific). Not sure which information could help so please ask for anything I can provide. Any pointers are highly appreciated! Thanks! Eugen

1 2

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

[horizon][ceph] object storage container storage policy missing
by Satish Patel 25 Nov '24

25 Nov '24

Folks, Openstack Release: Zed Recently I have integrated ceph RGW service with openstack/keystone and testing with command line tool "openstack container create foo1" and it works but when I go to Horizon UI I am seeing it doesn't have storage policy dropdown list and because of that I am not able to create container or upload file from Horizon GUI https://ibb.co/KhYW6Rd It works with commands like but not horizon. $ openstack container create foo1 +---------------------------------------+-----------+------------------------------------------------+ | account | container | x-trans-id | +---------------------------------------+-----------+------------------------------------------------+ | AUTH_d4d82849885e4126aef402c408ad2c5c | foo1 | tx00000c057454841b13497-0065b558b4-42ff2eb-ahm | +---------------------------------------+-----------+------------------------------------------------+ $ openstack container list +------+ | Name | +------+ | foo1 | +------+

2 2

[ironic] Having issue with deploying ISO image to node
by James Leong 18 Nov '24

18 Nov '24

Hi all, I am trying to play around with ironic, and I am having an issue with deploying an Ubuntu 20.04 ISO image onto my bare metal node. Since I am quite new to bare metal, I am not entirely sure what went wrong as I couldn't find any error messages shown on my log file (ironic-conductor, ironic-api, ironic_neutron_agent, and nova-compute-ironic). This might be my guess; the ironic inspection is hitting timeout, which is causing the deployment to timeout. However, I have no idea why the inspection is failing at timeout. Any help would be appreciated. Best, James

6 7

[horizon] SCSS compilers
by Radomir Dopieralski 15 Nov '24

15 Nov '24

The openstack dashboard uses scss for styling, and so far we have used a python scss library for compiling those files to css for use in the browser. However, that library (pyscss) had in recent years some problems, and is not developed or supported by the authors anymore. So it's time to switch to something with a better support. The django-compressor plugin that we use for handling the compilation of files allows us to use any command-line tool to do the compilation, so we can switch to an scss compiler that is supported and used by the wider community. However, there are several options we can choose from: sassc python-libsass pysassc ruby-sass ruby-sassc node-node-sass Each of those would introduce a non-python dependency on the dashboard. The question for the packagers and distro maintainer is which of those tools would be preferable in terms of support, packaging, and maintenance for the long term? Do you have opinions? Maybe there is another tool that would be even better that I didn't find?

2 3

OpenStack Cinder - Volume-ID in cinder DB
by Harsh Ailani 12 Nov '24

12 Nov '24

Hello contributors, I am working on clone operation for a volumegroup and in that the volume ID auto-generated by OpenStack is not the one with which the clone volumes are being created on my IBM storage backend. This is as per design that IBM Storage backend will take the source-volume ID in the source-volumegroup and add a suffix to it while cloning. Now since there is a mismatch in the IDs between cinder and storage backend, because of which I am unable to perform any volume operations on this cloned volume. So, I was wondering if I can change the volume ID in cinder DB? I know that it’s a unique key and there is a limit of 36 characters for this field in cinder DB, but since I cannot change the volume in storage backend, I wanted to check if there is a way I can alter the volume ID in cinder DB? Kindly share your thoughts/suggestions on this matter. Thanks and Regards, Harsh Ailani

2 4

[largescale-sig]scaling story
by Alex Song (宋文平) 25 Oct '24

25 Oct '24

Hi everyone, we want to share our scaling story. Please refer the link: https://etherpad.opendev.org/p/large-scale-inspur Thanks, Alex Song

6 8

[nova][libvirtd] internal error: unable to execute QEMU command 'query-named-block-nodes'
by Satish Patel 22 Oct '24

22 Oct '24

Folks, I have a ceph cluster and recently I configured rbd-mirror to replicate all data to remove ceph cluster for disaster recovery. Yesterday for POC we did a hard cutover on ceph and point openstack to the new cluster. All other vms came back up fine but 2 vms stuck in this error state in libvirt logs 2024-01-31 22:44:37.591+0000: 474597: error : qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied If it's a wider issue than it should impact all the vms but why only two VM stuck and not starting and libvirt giving me this error in logs nova-compute logs also showing same error 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] [instance: 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from powering-on on failure for instance. 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied

3 8

[kuryr][tc] Final call for Kuryr PTL/DPL liaisons
by Ghanshyam Mann 15 Oct '24

15 Oct '24

Hello Everyone, Kuryr project ends up being a leaderless project for the 2024.2 Dalmatian cycle[1]. There is no volunteer to serve as PTL or DPL liaisons yet. As the next step, I have proposed to mark this project Inactive: - https://review.opendev.org/c/openstack/governance/+/916945 If you would like to serve as a leader and help maintain this project, this is the right time to step up by replying to this email or commenting in gerrit change 916945. [1] https://etherpad.opendev.org/p/2024.2-leaderless#L99 Ref: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -gmann

4 10

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Improving stestr... or switching to a (better) compatible test runner?
by Thomas Goirand 09 Oct '24

09 Oct '24

Hi, This is the 2nd time I'm opening a thread about it. Sorry for this, but it really bothers me. Today, running tempest, I got these stats from stestr: - Worker 0 (132 tests) => 0:24:49.209885 - Worker 1 (100 tests) => 0:17:14.791845 - Worker 2 (124 tests) => 0:42:52.690906 - Worker 3 (189 tests) => 0:41:21.307241 - Worker 4 (159 tests) => 0:45:49.503031 - Worker 5 (143 tests) => 0:28:13.282371 - Worker 6 (156 tests) => 3:16:52.364976 - Worker 7 (103 tests) => 0:46:05.366089 So, thread #1 ran 17 minutes, sitting idle for the rest of the 3 hours run of thread #6. While I thought stestr was deviding the number of tests by the number of thread, I don't get why worker #1 only had 100 tests assigned. All together, all tests could have been ran within maybe less than an hour of time (rather than 3h16 above), if not-yet-ran-tests were reassigned to idle treads. I'm currently spending a lot of time on running tempest. Indeed, I'm running tempest on each OpenStack upgrade, from Victoria up to Dalmatian. With the current way stestr run, it may take 2 full days to do that (and that's not even counting when upgrade will fail and will need fixing...), when it could be done in maybe 8 hours (if I count 1h per OpenStack release). So, knowing the above, it might be a good use of my time to dig into stestr and see if I can fix this... or not! Does anyone have a suggestion for another test runner, that's compatible with stestr, at least for the tests selection with a regular expression? As much as I know, pytest cannot take a regexp for test selection, can it? Or is there maybe a plugin for it? If there's no compatible test runner, where should I dig in the stestr code to rewrite things in a smarter way? Cheers, Thomas Goirand (zigo) P.S: On Caracal today, I just had this: ============== Worker Balance ============== - Worker 0 (117 tests) => 0:59:32.291385 - Worker 1 (147 tests) => 1:22:28.228980 - Worker 2 (125 tests) => 0:45:20.969397 - Worker 3 (114 tests) => 1:46:21.667579 - Worker 4 (170 tests) => 0:45:27.577738 - Worker 5 (190 tests) => 2:28:39.744920 - Worker 6 (182 tests) => 2:29:01.402255 - Worker 7 (152 tests) => 2:29:10.183359 this looks better, but that's probably 1/ random 2/ still not perfect, with worker #0, #2 and #4 doing nothing 2/3rd of the time.

2 1

[tc][watcher] No leaders for project team, heading to retirement
by Goutham Pacha Ravi 09 Oct '24

09 Oct '24

Hello Stackers, The "watcher" project [1] did not have any volunteers to lead the project in the 2025.1 election cycle. In the past few years, the project's team has slowly disbanded and there have been no significant changes proposed/merged in the project's repositories [2]. We noticed that there were three deployers that noted their use of Watcher in production in the past user survey [3]. It's possible that the project failed to gather any significant adoption, and hence there's less interest in contributing to, and maintaining the project. I'm sending this notification pursuant to the OpenStack governance recommendations [4] to retire project teams. We can mark a project team "inactive", however, an "inactive" project must have some contributors aiming to build-back a maintainers team around it - and it must pick a project team lead, or adopt a distributed project leadership model by nominating liaisons. Since there are no PTL nominees, we think marking this project "inactive" serves no purpose. Is there any objection to retiring the "openstack/watcher" project (and associated deliverables: python-watcherclient, watcher-dashboard and watcher-tempest-plugin)? Once the project is retired, configuration management and deployment projects can drop their integrations as well. Thanks, On behalf of the OpenStack TC, Goutham Pacha Ravi [1] https://wiki.openstack.org/wiki/Watcher [2] https://review.opendev.org/q/(project:%5Eopenstack/watcher.*+OR+project:ope… [3] https://governance.openstack.org/tc/user_survey/analysis-2023.html [4] https://governance.openstack.org/tc/reference/dropping-projects.html

5 7

Stop bumping major versions when dropping EOL Python versions
by Stephen Finucane 04 Oct '24

04 Oct '24

o/ Python 3.8 is going EOL (upstream, anyway) next month. We've been working to drop support for this from various SDK deliverables, and I'm guessing (though don't yet know for sure) that this will happen across other project deliverables before long (if it hasn't started happening already). One side-effect of this change is that we will be expected to raise the major version of the affected dependencies on our next release. This is annoying for projects like openstacksdk that use the release-with-intermediary release model because it forces us to have a major version release earlier than we would like to. To give a concrete example, we have a RemovedInSDK50Warning deprecation warning that we are using to indicate (as you might guess) things that we plan to remove in openstacksdk 5.0.0. We need to wait 6 months/1 OpenStack release before removing anything flagged with this warning, so this has been mainly applied to legacy deprecation warnings to put a concrete timeline on their removal. However, because we just dropped Python 3.8 support, it is currently expected that our next release be 5.0.0. We last deprecated something new ~2 months ago, so this means we either cannot release a new version of openstacksdk for 3-4 months (to allow 6 months to pass) or we need to either rename RemovedInSDK50Warning to RemovedInSDK60Warning or add a new RemovedInSDK60Warning and move the offending deprecations to this. None of these are nice options and to be honest, I don't think the major version bump really provides all that much value to end-users: they already have a much better signal for the removal of a Python version in the 'python_requires' marker in setup.py/setup.cfg/pyproject.toml/(other build system file). By using this, we ensure that a user simply can't install a version of a package that doesn't support their chosen Python version. I'd be in favour of getting rid of this particular constraint, and instead focusing on API changes when deciding whether to bump the major version. Is this a reasonable suggestion, and if so what would we need to do to tweak our policy to allow it? Cheers, Stephen

4 3

[Kolla-Ansible] [Ironic] Node Deploy Failed with MissingAuthPlugin
by Ali Hussain 01 Oct '24

01 Oct '24

Hi Kolla and Ironic folks, While playing with stable/kolla release to fix the No VIF found error. Looks like adding Readers role to ironic user fixed it. But run into Missing Auth plugin [2] issue while image is being deployed to ironic node. Openstack endpoint list command [1] shows the correct IP of the controller. My question is: 1. Kolla-Ansible needs explicite Ironic keystone configuration in Ironic.conf? Following this link <https://docs.openstack.org/kolla-ansible/yoga/reference/bare-metal/ironic-g…> which does not seems to need any keystone configurations. This issue is observed when 1. --boot-interface redfish-virtual-media 2. --deploy-interface ramdisk 3. --driver-info deploy_iso= 4. -instance-info kernel & ramdisk 2. My passwords.yaml has 1. ironic_database_password: 2. ironic_inspector_database_password: 3. ironic_inspector_keystone_password: 4. ironic_keystone_password: Any suggestions please. Thank you. Ali [1] openstack endpoint list | grep ironic | <ID> | <Region> | ironic-inspector | baremetal-introspection | True | internal | http://<IP>:5050 | | <ID> | <Region> | ironic | baremetal | True | public | http://<IP>:6385 | | <ID> | <Region> | ironic-inspector | baremetal-introspection | True | public | http://<IP>:5050 | | <ID> | <Region> | ironic | baremetal | True | internal | http://<IP>:6385 [2] *Error: *Detected change in error condition on node Node1. Deploy step deploy.deploy failed with MissingAuthPlugin: An auth plugin is required to determine endpoint URL.. An unhandled exception was encountered while aborting. More information may be found in the log file.

2 6