January 2025 - openstack-discuss - lists.openstack.org

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 08 Dec '25

08 Dec '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 14

[openstack helm] [Glance] Glance showing OSError: unable to receive chunked part when uploading image
by daniel890723＠gmail.com 08 Nov '25

08 Nov '25

Hello i'm testing using Openstack-helm to install all component but after the installation complete i can't upload the OS image through both openstack CLI or Horizon it stuck at Queueing and when i checking Glance Api log it shows 2024-11-07 07:31:19.362 7 INFO glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2024-11-07 07:31:19.400 7 ERROR glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Failed to upload image data due to internal error: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Caught error: unable to receive chunked part: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Traceback (most recent call last): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1302, in __call__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1345, in dispatch 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return method(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 415, in wrapped 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 299, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 162, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 492, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 443, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/quota/__init__.py", line 322, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 596, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 487, in _upload_to_store 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 397, in add_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 479, in store_add_to_backend_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/driver.py", line 294, in add_adapter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/capabilities.py", line 176, in op_checker 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 764, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise errors.get(e.errno, e) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 746, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi for buf in utils.chunkreadable(image_file, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/common/utils.py", line 73, in chunkiter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = fp.read(chunk_size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 294, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = self.data.read(i) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 121, in readfn 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = fd.read(*args) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/format_inspector.py", line 956, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = self._source.read(size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1045, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi data = uwsgi.chunked_read() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Ceph Cluster Status: NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID ceph /var/lib/rook 3 82m Ready Cluster created successfully HEALTH_OK 5cf8b9bb-13e1-4c31-b47d-9e7d92c3c05d Ceph osd status: ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS TYPE NAME -1 0.35156 - 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 1.00 - root default -5 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 - host instance-lu-1 2 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 195 up osd.2 -3 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-2 0 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 195 up osd.0 -7 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-3 1 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 197 up osd.1 TOTAL 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 MIN/MAX VAR: 1.00/1.01 STDDEV: 0.01 now i'm able to create Empty Volume using horizon(can't create volume using default cirros image ) and i can start an Instance without volume and mount the empty volume in(Ceph also can detect when i use that volume) is there something i need to set when starting glance? because i'm following the Openstack-helm installation document to install openstack with helm

5 8

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 30 Oct '25

30 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

[openstack][octavia]Resize a Load Balancer
by Nguyễn Hữu Khôi 22 Oct '25

22 Oct '25

Hello. I see that patch https://review.opendev.org/c/openstack/octavia/+/890215 Could I ask why we cannot merge it? Nguyen Huu Khoi

3 5

[i18n][infra][horizon] Zanata translation job failures after Python 3.8 deprecation
by Ian Y. Choi 09 Jul '25

09 Jul '25

Hi all, During last week's PTG, the I18n SIG identified Zanata translation job failures for Horizon (and likely related plugins). The root cause appears to be Python 3.8 deprecation on Horizon [1]. Failed jobs have been observed on Zuul [2], and translation proposals were made on Gerrit [3]. Since translation sync on Horizon stable branches is functioning correctly, the error message seems to be related to Python 3.8 deprecation: ERROR: Cannot install -r doc/requirements.txt (line 3) due to conflicting package version dependencies. To address this, I'd like to raise two points: 1/ To resolve the issue, would you support updating translation sync jobs with Bionic-based Zuul jobs by either upgrading to Jammy or modifying the Bionic image to support Python >3.8? 2/ At least me from I18n SIG acknowledges that recognizing these translation job errors was delayed. Moving forward, would it be beneficial to implement periodic (e.g., monthly) reporting on translation job successes and failures? Thank you, /Ian [1] https://docs.openstack.org/releasenotes/horizon/2024.2.html#upgrade-notes [2] https://zuul.opendev.org/t/openstack/builds?job_name=propose-translation-up… [3] https://review.opendev.org/q/owner:proposal-bot+project:openstack/horizon+b…

2 3

Problems with the Placement service
by Ron Gage 25 Jun '25

25 Jun '25

5 7

[ops][neutron] Neutron-db-manage upgrade from Antelope to Bobcat fails with "foreign key constraint 'porthints_ibfk_1' are incompatible" error
by federica fanzago 20 May '25

20 May '25

Dear all, we are running an OpenStack deployment Yoga on AlmaLinux 9.5 and we are attempting the upgrade Openstack up to Caracal release, one release at time (Zed--Antelope--Bobcat--Caracal) The procedure we are following is to shut down all openstack services, to remove the "old" openstack release and to install the new one, upgrading all service databases. This procedure works well for Zed and Antelope, but during the installation of the Bobcat release, the Neutron database upgrade fails when creating the new "porthints" table. The command we are using is su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron and the error message we receive is ... File "/usr/lib/python3.9/site-packages/pymysql/err.py", line 107, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (3780, "Referencing column 'port_id' and referenced column 'id' in foreign key constraint 'porthints_ibfk_1' are incompatible.") [SQL: CREATE TABLE porthints ( port_id VARCHAR(36) NOT NULL, hints VARCHAR(4095) NOT NULL, PRIMARY KEY (port_id), FOREIGN KEY(port_id) REFERENCES ports (id) ON DELETE CASCADE )ENGINE=InnoDB ... Our "ports" table uses the collation utf8mb3_general_ci mysql> SHOW FULL COLUMNS FROM ports; +------------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+ | Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment | +------------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+ | project_id | varchar(255) | utf8mb3_general_ci | YES | MUL | NULL | | select,insert,update,references | | | id | varchar(36) | utf8mb3_general_ci | NO | PRI | NULL | | select,insert,update,references | | | name | varchar(255) | utf8mb3_general_ci | YES | | NULL | | select,insert,update,references | | | network_id | varchar(36) | utf8mb3_general_ci | NO | MUL | NULL | | select,insert,update,references | | | mac_address | varchar(32) | utf8mb3_general_ci | NO | | NULL | | select,insert,update,references | | | admin_state_up | tinyint(1) | NULL | NO | | NULL | | select,insert,update,references | | | status | varchar(16) | utf8mb3_general_ci | NO | | NULL | | select,insert,update,references | | | device_id | varchar(255) | utf8mb3_general_ci | NO | MUL | NULL | | select,insert,update,references | | | device_owner | varchar(255) | utf8mb3_general_ci | NO | | NULL | | select,insert,update,references | | | standard_attr_id | bigint | NULL | NO | UNI | NULL | | select,insert,update,references | | | ip_allocation | varchar(16) | utf8mb3_general_ci | YES | | NULL | | select,insert,update,references | | +------------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+ in Bobcat the collation required for the new "porthints" table is latin1_swedish_ci MariaDB [neutron]> show create table porthints; +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | porthints | CREATE TABLE `porthints` ( `port_id` varchar(36) NOT NULL, `hints` varchar(4095) NOT NULL, PRIMARY KEY (`port_id`), CONSTRAINT `porthints_ibfk_1` FOREIGN KEY (`port_id`) REFERENCES `ports` (`id`) ON DELETE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci | So, we have tried to create the porthints table in mysql, forcing the collation to utf8mb3_general_ci mysql> CREATE TABLE porthints ( port_id VARCHAR(36) NOT NULL, hints VARCHAR(4095) NOT NULL, PRIMARY KEY (port_id), FOREIGN KEY(port_id) REFERENCES ports (id) ON DELETE CASCADE )ENGINE=InnoDB DEFAULT CHARSET=utf8 DEFAULT COLLATE utf8mb3_general_ci; and commenting out the creation of the new table in the neutron code /usr/lib/python3.9/site-packages/neutron/db/migration/alembic_migrations/versions/2023.2/expand/6f1145bff34c_port_hints.py. In this way, the upgrade works, but we encounter the same kind of problem during the upgrade to Caracal, specifically with the creation of the new table porthardwareoffloadtype. Have you already encountered this issue during the Neutron upgrade? Do you have any suggestions on how to upgrade the neutron DB, without manually creating new tables? Thanks a lot, cheers, Federica -- Federica Fanzago INFN Sezione di Padova Via Marzolo, 8 35131 Padova - Italy Tel: +39 049.967.7367 --

4 8

[manila] Removal of the Standalone CephFS NFS protocol helper in Epoxy
by Ashley Rodriguez 02 May '25

02 May '25

Hi zorillas! As per subject line, we will be deprecating the standalone CephFS NFS protocol helper in this cycle. This is following our deprecation warning we put out in 2024.1 / Caracal [1] [2]. We've supported using a clustered Ceph NFS service since the Zed release. We've also added upgrade helpers to migrate from a standalone NFS-Ganesha service to the ceph-orch-deployed clustered NFS service; this code was backported to stable/2023.1 (Antelope). Although the upgrade path is disruptive, there is a UX improvement in Manila to ease this migration through surfacing “preferred” export paths. Moving forward, deployments must have “cephfs_nfs_cluster_id” in your configuration, otherwise the driver will fail to start. Manila will not make any export changes to NFS-Ganesha, so upgrading manila without an NFS cluster will mean that only the control plane is not going to work; but, data plane will remain unaffected. To assist migration, admins can also configure "cephfs_ganesha_export_ips" (or alternatively, "cephfs_ganesha_server_ip") alongside "cephfs_nfs_cluster_id". Setting these options will allow the CephFS driver to report additional export paths. These additional export paths will have the "preferred" metadata key set to False. Best, Manila team [1] https://review.opendev.org/c/openstack/manila/+/912648 [2] https://docs.openstack.org/manila/2024.2/configuration/shared-file-systems/…

5 7

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

[oslo] python-consul2 is unmaintained
by Takashi Kajinami 21 Apr '25

21 Apr '25

Hello, I recently noticed that python-consul2 library[1], which is used by the consul coordination driver in tooz, hasn't been updated for 3+ years. Looking at issues in the project, there is an indication that the library is not compatible with ACL API in consul >= 1.11[2]. There is another fork created[3] and this includes fixes to support new API endpoints. Although the driver in tooz does not directly use the feature, I think it's better to discuss our plan to mitigate risks caused by the unmaintained library. As far as I can think of, there are three options now 1. Deprecate consul driver and remove it 2. Keep using python-consul2 now. At least basic functionality is proven to work by our functional tests (Note that the tests do not cover all configuration patterns) 3. Replace python-consul2 by py-consul Before we start the overall discussions I'd like to ask a few questions to the wider audience, as our inputs. - Is anyone using the consul coordination driver now ? - Does anyone have experience with python-consul2 and/or py-consul ? Note that this discussion affects the http messaging driver[4] currently proposed, because it also requires client library to communicate with Consul. Actually I noticed the situation when I noticed that the requirement check job is failing because the library is not part of global requirements list... Thank you, Takashi Kajinami [1] https://github.com/poppyred/python-consul2 [2] https://github.com/poppyred/python-consul2/issues/42 [3] https://github.com/criteo-forks/py-consul [4] https://review.opendev.org/c/openstack/oslo.messaging/+/912499 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 3

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

GPU PCI passthrough woes.
by Andy Speagle 09 Apr '25

09 Apr '25

Hey Folks, I could use a little assistance getting GPU passthrough working. I had this working already for one flavor of nvidia gpu... and I've added some hosts with a much newer gpu... I've updated the pci_alias and pci_passthrough variables and those seem to be getting set properly in nova.conf passthrough_whitelist = [{"vendor_id":"10de", "product_id":"1b06"},{"vendor_id":"10de", "product_id":"26b9"}] alias = {"name": "gpu", "product_id": "1b06", "vendor_id": "10de"} alias = {"name": "gpu-l40s", "product_id": "26b9", "vendor_id": "10de"} I believe I have all of the iommu stuff configured and have the pci- stub module entries... dmesg output shows that the GPUs are being claimed by the stub module. $ openstack flavor show t1.small_gpu_l40s +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 0 | | id | af70c94e-0026-4a39-bc1e-dfb93b286a54 | | name | t1.small_gpu_l40s | | os-flavor-access:is_public | True | | properties | pci_passthrough:alias='gpu-l40s:1' | | ram | 2048 | | rxtx_factor | 1.0 | | swap | 0 | | vcpus | 1 | +----------------------------+--------------------------------------+ Yet... I can't seem to get an instance to run using that new flavor... keeps complaining about there not being enough hosts available. Fault: code: 500 created: 2024-10-30T02:04:33Z message: "No valid host was found. There are not enough hosts available." details: | Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 1580, in schedule_and_build_instances host_lists = self._schedule_instances(context, request_specs[0], File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 940, in _schedule_instances host_lists = self.query_client.select_destinations( File "/usr/lib/python3/dist- packages/nova/scheduler/client/query.py", line 41, in select_destinations return self.scheduler_rpcapi.select_destinations(context, spec_obj, File "/usr/lib/python3/dist-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/usr/lib/python3/dist- packages/oslo_messaging/rpc/client.py", line 189, in call result = self.transport._send( File "/usr/lib/python3/dist- packages/oslo_messaging/transport.py", line 123, in _send return self._driver.send(target, ctxt, message, File "/usr/lib/python3/dist- packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send return self._send(target, ctxt, message, wait_for_reply, timeout, File "/usr/lib/python3/dist- packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send raise result nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. Traceback (most recent call last): File "/usr/lib/python3/dist- packages/oslo_messaging/rpc/server.py", line 241, in inner return func(*args, **kwargs) File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 223, in select_destinations selections = self._select_destinations( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 250, in _select_destinations selections = self._schedule( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 416, in _schedule self._ensure_sufficient_hosts( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 455, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason) nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. Any clues on how/where to dig into this more to see what might be missing? Thanks. -- Andy Speagle

3 2

[Devstack] Trying to setup development environment on VM
by Ishan Shanware (ishanwar) 17 Mar '25

17 Mar '25

Hello Everyone, I am currently trying to setup in all-in-one development environment for OpenStack on a Debian VM. The VM has 16 GB RAM and 4 CPUs. However, I face an error while trying to run FORCE=yes ./stack.sh. I get an error that keystone does not start. This causes the entire process to fail. Requesting assistance in solving this issue. I am following the this guide: https://docs.openstack.org/devstack/latest/guides/single-machine.html PFA the error log and error message on the terminal. [Call Trace] ./stack.sh:1126:start_keystone /opt/stack/devstack/lib/keystone:549:die [ERROR] /opt/stack/devstack/lib/keystone:549 keystone did not start Error on exit World dumping... see /opt/stack/logs/worlddump-2024-11-18-150808.txt for details ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) Thanks in advance, Ishan

3 5

[all][tc][ptl][qa] Migrating upstream CI/CD jobs to Ubuntu Noble (24.04)
by Ghanshyam Mann 14 Mar '25

14 Mar '25

Hello Everyone, As per the 2025.2 cycle testing runtime, we need to run upstream CI/CD jobs on Ubuntu Noble (24.04) [1]. For better tracking and speeding up this work, it is the community-wide goal for this cycle[2]. This is the etherpad which has the migration plan and will be used for the tracking: - https://etherpad.opendev.org/p/migrate-to-noble I have prepared the change(WIP) for base jobs migration on Noble (openstack tox jobs, devstack, and tempest) which will migrate most of the projects jobs. Tempest change[3] include dependency on openstack tox, devstack base jobs changes so testing project gate using the the tempest change will test the complete set. Refer to the etherpad for the complete migration plan/steps. * Deadline: Nov 29th 2024 (R18) is the deadline to test/fix the projects jobs and to merge the base jobs running on Noble (openstck tox, devstack, and tempest). [1] https://governance.openstack.org/tc/reference/runtimes/2025.1.html [2] https://governance.openstack.org/tc/goals/selected/migrate-ci-jobs-to-ubunt… [3] https://review.opendev.org/c/openstack/tempest/+/932156 -gmann

2 9

[kolla] [nova] API flags ignored in Wallaby
by Albert Braden 12 Mar '25

12 Mar '25

After rebuilding our Train clusters on Wallaby we are seeing some API issues. It appears that flags that work in Train are still documented in Wallaby, but are ignored. Do we have something setup wrong, or are we encountering bugs? The first thing is the “locked” list: Train: openstack server list --all-projects --locked --os-compute-api-version 2.73 <empty list because nothing is locked> openstack server list --all-projects --unlocked --os-compute-api-version 2.73 <list of every VM in the cluster> Wallaby (locked flag is ignored): openstack server list --all-projects --locked --os-compute-api-version 2.73 <list of every VM in the cluster> openstack server list --all-projects --unlocked --os-compute-api-version 2.73 <list of every VM in the cluster> The second problem is the hypervisor list. We can’t get “long” output in Wallaby: Train: openstack hypervisor list --long | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | vCPUs Used | vCPUs | Memory MB Used | Memory MB | Wallaby (long flag is ignored): openstack hypervisor list --long | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |

6 14

[nova]Intel RDT CAT Support Missing from OpenStack
by Junya Noguchi (Fujitsu) 25 Feb '25

25 Feb '25

To whom it may concern I have a question regarding the lack of Intel RDT CAT implementation in OpenStack We are planning to commit Resource Monitoring functionality to Openstack for ARM, and are exploring the implementation of Intel RDT on Openstack. We found the following blueprint for the Intel RDT CAT implementation in nova. https://blueprints.launchpad.net/nova/+spec/cat-support However, the Intel RDT CAT feature is not currently implemented in OpenStack. Has the implementation of Intel RDT CAT functionality in Openstack been rejected? Or is it just a development delay? Best regards, Junya Noguchi.

5 9

[tc][all][monasca ] Retiring the Monasca project
by Ghanshyam Mann 16 Feb '25

16 Feb '25

Hello everyone, As you might know, the Monasca project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Monasca retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

6 17

[nova] How to gain approval for a new feature on unavailable hardware
by taketani.ryo＠fujitsu.com 14 Feb '25

14 Feb '25

To Whom It May Concern, We are considering to propose and develop a new feature on OpenStack Nova for building Arm Confidential Computing Architecture (Arm CCA)-encrypted instances. I have some questions for doing it. * While specification of Arm CCA is already open(https://developer.arm.com/documentation/den0125/0300/Overview), No Arm CCA-enabled hardware is currently available to our knowledge. Therefore, we plan to conduct testing in an internal environment. * Would a proposal for a feature utilizing publicly unavailable hardware be acceptable to OpenStack? Alternatively, in this case, what steps or preparations are required for approval? * Should the blueprint and specification be submitted simultaneously? * The specification is currently incomplete. Building Arm CCA instances requires libvirt APIs, similar to AMD SEV instances(https://blueprints.launchpad.net/nova/+spec/amd-sev-libvirt-suppo…, but specification of these are not yet determined publicly. We are considering submitting the blueprint first to inform the community of our ongoing development efforts. Regards

3 6

openstack instances unable to reach the internet
by collinl＠churchofjesuschrist.org 12 Feb '25

12 Feb '25

I have been banging my head on this issue for quite some time now. I am hoping that somebody can help me see what I have misconfigured. Here is the situation. I have tried setting up several openstack installs, and in none of them am I able to reach the internet from the instances running inside of openstack. All of them are on Rocky Linux 9.5, running the dalmatian version of openstack, installed using packstack. (I have tried an all-in-one, with everything on a single node, and a cluster with the controller/networking on one node, and the compute running on separate nodes). I can get the cirros instances running on the backend network within openstack, and can ssh to them via the floating ip from the host without issues. The cirros instance can ping other cirros instances in the openstack controlled network (both on the backend network(subnet is in the 192.168.100.0/24 range, or on the external public network(subnet is in the 172.24.4.024 range) Even trying the simplest configuration, where everything is all on a single node (compute/networking/controller/etc), Everything from instance creation to VM migration/etc all seems to work just fine. However, the cirros instances cannot ping anything on the internet -- for example, I am unable to ping 8.8.8.8 or even the gateway that the host can reach (The host has an ip used to reach horizon on the 10.61.157.0/24 subnet on our internal lan) I can ping all interfaces from the cirros interface through the backend_network (192.168.100.0/24) through the public interface (172.24.4.0/24), and can even ping the host ip address (10.61.157.59) . . .However, I can't ping anything beyond that (i.e. the 10.61.147.1 gateway does not ping from the cirros instance) From the host commandline, it is able to reach everything external just fine (i.e. it can ping it's gateway 10.61.147.1 and 8.8.8.8 or anything else external) Here is the ip addr output from the host: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000 link/ether 00:1d:d8:b7:1e:df brd ff:ff:ff:ff:ff:ff inet6 fe80::21d:d8ff:feb7:1edf/64 scope link valid_lft forever preferred_lft forever 3: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 56:d8:51:53:10:b0 brd ff:ff:ff:ff:ff:ff 4: br-int: <BROADCAST,MULTICAST> mtu 1442 qdisc noop state DOWN group default qlen 1000 link/ether 62:8c:b0:c2:3e:5a brd ff:ff:ff:ff:ff:ff 5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 00:1d:d8:b7:1e:df brd ff:ff:ff:ff:ff:ff inet 10.61.157.59/24 brd 10.61.157.255 scope global br-ex valid_lft forever preferred_lft forever inet 172.24.4.1/24 brd 172.24.4.255 scope global br-ex valid_lft forever preferred_lft forever inet6 fe80::2446:29ff:fe58:6541/64 scope link valid_lft forever preferred_lft forever 6: tapeffe7cc1-56: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000 link/ether fe:16:3e:e2:66:5d brd ff:ff:ff:ff:ff:ff inet6 fe80::fc16:3eff:fee2:665d/64 scope link valid_lft forever preferred_lft forever 7: tap65170f1f-e0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000 link/ether 3a:26:09:cf:3d:4f brd ff:ff:ff:ff:ff:ff link-netns ovnmeta-65170f1f-e837-4b5e-a471-13e783e6b48e inet6 fe80::3826:9ff:fecf:3d4f/64 scope link valid_lft forever preferred_lft forever And the output from the ovs-vsctl show command: 46294eda-8a39-4775-875d-ef6ec9929346 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-int fail_mode: secure datapath_type: system Port tap65170f1f-e0 Interface tap65170f1f-e0 Port patch-br-int-to-provnet-371bdb7c-d2f3-4e70-aca9-1c4838c57e24 Interface patch-br-int-to-provnet-371bdb7c-d2f3-4e70-aca9-1c4838c57e24 type: patch options: {peer=patch-provnet-371bdb7c-d2f3-4e70-aca9-1c4838c57e24-to-br-int} Port tapeffe7cc1-56 Interface tapeffe7cc1-56 Port br-int Interface br-int type: internal Bridge br-ex fail_mode: standalone Port patch-provnet-371bdb7c-d2f3-4e70-aca9-1c4838c57e24-to-br-int Interface patch-provnet-371bdb7c-d2f3-4e70-aca9-1c4838c57e24-to-br-int type: patch options: {peer=patch-br-int-to-provnet-371bdb7c-d2f3-4e70-aca9-1c4838c57e24} Port br-ex Interface br-ex type: internal Port eth0 Interface eth0 ovs_version: "3.3.1" Here is the info for the networks/subnets currently configured: From "openstack network list --long" +------------------------+-----------------+--------+------------------------+-------+--------+------------------------+--------------+-------------+--------------------+------+ | ID | Name | Status | Project | State | Shared | Subnets | Network Type | Router Type | Availability Zones | Tags | +------------------------+-----------------+--------+------------------------+-------+--------+------------------------+--------------+-------------+--------------------+------+ | 65170f1f-e837-4b5e-a47 | backend_network | ACTIVE | c5f408c4daed4fb2afefa4 | UP | False | b1e5aef4-77da-4b0b-9b9 | geneve | Internal | | | | 1-13e783e6b48e | | | 4fa039fe26 | | | 4-5aaf395484e0 | | | | | | b8e7a305-65f0-44ea-93a | public | ACTIVE | c5f408c4daed4fb2afefa4 | UP | False | 6cff8f70-9dc8-4f17-b78 | flat | External | | | | a-cc6ec741d408 | | | 4fa039fe26 | | | 0-32cf2aedb017 | | | | | | c0c67da4-22bf-4dd2-b4c | private | ACTIVE | 2967b1fe70794fa88cbb3d | UP | False | caed16ae-22ff-427a-a37 | geneve | Internal | | | | 2-2edcb41ffb97 | | | e7227cec77 | | | 0-6b891755bfa7 | | | | | +------------------------+-----------------+--------+------------------------+-------+--------+------------------------+--------------+-------------+--------------------+------+ From "openstack subnet list --long" +---------------+---------------+---------------+---------------+---------------+-------+--------------+------------------+-------------+------------+---------------+---------------+------+ | ID | Name | Network | Subnet | Project | DHCP | Name Servers | Allocation Pools | Host Routes | IP Version | Gateway | Service Types | Tags | +---------------+---------------+---------------+---------------+---------------+-------+--------------+------------------+-------------+------------+---------------+---------------+------+ | 6cff8f70-9dc8 | public_subnet | b8e7a305-65f0 | 172.24.4.0/24 | c5f408c4daed4 | False | | 172.24.4.2-172.2 | | 4 | 172.24.4.1 | | | | -4f17-b780-32 | | -44ea-93aa- | | fb2afefa44fa0 | | | 4.4.254 | | | | | | | cf2aedb017 | | cc6ec741d408 | | 39fe26 | | | | | | | | | | b1e5aef4-77da | backend_subne | 65170f1f-e837 | 192.168.100.0 | c5f408c4daed4 | True | | 192.168.100.100- | | 4 | 192.168.100.1 | | | | -4b0b-9b94-5a | t | -4b5e-a471-13 | /24 | fb2afefa44fa0 | | | 192.168.100.250 | | | | | | | af395484e0 | | e783e6b48e | | 39fe26 | | | | | | | | | | caed16ae-22ff | private_subne | c0c67da4-22bf | 10.0.0.0/24 | 2967b1fe70794 | True | | 10.0.0.2-10.0.0. | | 4 | 10.0.0.1 | | | | -427a-a370-6b | t | -4dd2-b4c2-2e | | fa88cbb3de722 | | | 254 | | | | | | | 891755bfa7 | | dcb41ffb97 | | 7cec77 | | | | | | | | | +---------------+---------------+---------------+---------------+---------------+-------+--------------+------------------+-------------+------------+---------------+---------------+------+ From "openstack floating ip list" +------------------------------------+---------------------+------------------+------------------------------------+------------------------------------+----------------------------------+ | ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project | +------------------------------------+---------------------+------------------+------------------------------------+------------------------------------+----------------------------------+ | 61d98a1c-f19a-451f-9353-6cb8305a0c | 172.24.4.175 | 192.168.100.128 | effe7cc1-566d-4574-b4e3-d9f440b776 | b8e7a305-65f0-44ea-93aa- | c5f408c4daed4fb2afefa44fa039fe26 | | 2f | | | 5e | cc6ec741d408 | | +------------------------------------+---------------------+------------------+------------------------------------+------------------------------------+----------------------------------+ From "openstack server list" +--------------------------------------+-------+--------+-----------------------------------------------+--------------------------+---------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------+--------+-----------------------------------------------+--------------------------+---------+ | e42372d2-946d-4c90-bf2d-1e36cda80e2f | test3 | ACTIVE | backend_network=172.24.4.175, 192.168.100.128 | N/A (booted from volume) | m1.tiny | +--------------------------------------+-------+--------+-----------------------------------------------+--------------------------+---------+ Any assistance in helping me figure out what I am missing to get external access would be greatly appreciated. Thank you!

4 14

[openstack][manila][cephfs]About access rule
by Nguyễn Hữu Khôi 11 Feb '25

11 Feb '25

Hello. I setup Manila with Cephfs driver-CephFS NFS shares and I use NFS-Ganesha based “ceph nfs” service. I can create a share and can mount after creating access rule: openstack share access create cephnfsshare ip 10.10.11.76 But I can mount my share from any ip. Is it a bug, or do I understand it wrong? My env: Openstack 2024 with Kolla-Ansible deployment Ceph Quincy Thank you. Regards Nguyen Huu Khoi

2 12

[freezer] Moving project bug tracker to launchpad
by Dmitriy Rabotyagov 06 Feb '25

06 Feb '25

Hey there, In order to simplify and align with most OpenStack projects on bug tracker I aim to move Freezer from different Storyboard [1] projects back to unified Launchpad [2] tracker. However, historically the OpenStack Administrators [3] group was not added as Freezer project administrator/owner on Launchpad, which means that no current maintainer of the project has the necessary rights to manage the mentioned Launchpad project. I've added all current Launchpad admins for Freezer project to CC of this email, to either of you to add "openstack-admins" [3] to the Freezer Drivers [4] team, so that OpenStack Infra team could assist with further recovering/managing access to the Launchpad project. That will be very nice if some of you could spend couple of minutes on making this change and help us avoid cumbersome process of recovering access to the project Thanks in advance! [1] https://storyboard.openstack.org/#!/project/list?q=freezer [2] https://launchpad.net/freezer [3] https://launchpad.net/~openstack-admins [4] https://launchpad.net/~freezer-drivers

4 3

Discussion of OpenStack development getting Error.
by Kiran Raja 05 Feb '25

05 Feb '25

Hi Team, We have deployed Open stack and when we are trying to access Neutron Gateway from the Controller node we are unable to access, but It is access able from Namespace Created in neutron , please help me on neutron configuration we are using Self-provision Network Option. root@controller:~# openstack router list +--------------------------------------+--------------+--------+-------+----------------------------------+-------------+-------+ | ID | Name | Status | State | Project | Distributed | HA | +--------------------------------------+--------------+--------+-------+----------------------------------+-------------+-------+ | 16d92543-e9a3-4e97-bdb0-ccb3a8b8dd99 | Admin_Router | ACTIVE | UP | 523711770d5243dd9af5b2356b540b07 | False | False | +--------------------------------------+--------------+--------+-------+----------------------------------+-------------+-------+ root@controller:~# ip netns exec qrouter-16d92543-e9a3-4e97-bdb0-ccb3a8b8dd99 ip route default via 172.16.86.1 dev qg-afb1772f-2f proto static 172.16.86.0/24 dev qg-afb1772f-2f proto kernel scope link src 172.16.86.49 root@controller:~# ip netns exec qrouter-16d92543-e9a3-4e97-bdb0-ccb3a8b8dd99 route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default 172.16.86.1 0.0.0.0 UG 0 0 0 qg-afb1772f-2f 172.16.86.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-afb1772f-2f root@controller:~# ip netns exec qrouter-16d92543-e9a3-4e97-bdb0-ccb3a8b8dd99 ping 172.16.86.1 PING 172.16.86.1 (172.16.86.1) 56(84) bytes of data. ^C --- 172.16.86.1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2042ms root@controller:~# ip netns exec qrouter-16d92543-e9a3-4e97-bdb0-ccb3a8b8dd99 ip address 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 13: qg-afb1772f-2f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:d1:58:50 brd ff:ff:ff:ff:ff:ff inet 172.16.86.49/24 brd 172.16.86.255 scope global qg-afb1772f-2f valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fed1:5850/64 scope link valid_lft forever preferred_lft forever root@controller:~# ip netns exec qrouter-16d92543-e9a3-4e97-bdb0-ccb3a8b8dd99 ping 172.16.86.49 PING 172.16.86.49 (172.16.86.49) 56(84) bytes of data. 64 bytes from 172.16.86.49: icmp_seq=1 ttl=64 time=0.073 ms 64 bytes from 172.16.86.49: icmp_seq=2 ttl=64 time=0.113 ms ^C --- 172.16.86.49 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1003ms rtt min/avg/max/mdev = 0.073/0.093/0.113/0.020 ms root@controller:~# ping 172.16.86.49 PING 172.16.86.49 (172.16.86.49) 56(84) bytes of data. ^C --- 172.16.86.49 ping statistics --- 2 packets transmitted, 0 received, 100% packet loss, time 1048ms root@controller:~# ip netns exec qrouter-16d92543-e9a3-4e97-bdb0-ccb3a8b8dd99 ip address 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 13: qg-afb1772f-2f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:d1:58:50 brd ff:ff:ff:ff:ff:ff inet 172.16.86.49/24 brd 172.16.86.255 scope global qg-afb1772f-2f valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fed1:5850/64 scope link valid_lft forever preferred_lft forever 15: qr-6be4aabd-b9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:df:a9:ad brd ff:ff:ff:ff:ff:ff inet 172.16.100.1/24 brd 172.16.100.255 scope global qr-6be4aabd-b9 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fedf:a9ad/64 scope link valid_lft forever preferred_lft forever root@controller:~# ping 172.16.86.1 PING 172.16.86.1 (172.16.86.1) 56(84) bytes of data. 64 bytes from 172.16.86.1: icmp_seq=1 ttl=64 time=0.204 ms ^C --- 172.16.86.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.204/0.204/0.204/0.000 ms root@controller:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever 2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:25:f8:21 brd ff:ff:ff:ff:ff:ff altname enp3s0 inet 172.16.85.11/24 brd 172.16.85.255 scope global ens160 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe25:f821/64 scope link valid_lft forever preferred_lft forever 3: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000 link/ether 00:0c:29:25:f8:2b brd ff:ff:ff:ff:ff:ff altname enp11s0 inet6 fe80::20c:29ff:fe25:f82b/64 scope link valid_lft forever preferred_lft forever 4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 2e:14:3b:10:5d:42 brd ff:ff:ff:ff:ff:ff 5: br-tun: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 02:fb:f2:93:65:46 brd ff:ff:ff:ff:ff:ff inet6 fe80::fb:f2ff:fe93:6546/64 scope link valid_lft forever preferred_lft forever 6: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 00:0c:29:25:f8:2b brd ff:ff:ff:ff:ff:ff inet6 fe80::20c:29ff:fe25:f82b/64 scope link valid_lft forever preferred_lft forever 7: br-int: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 72:cb:66:8a:21:4c brd ff:ff:ff:ff:ff:ff inet6 fe80::70cb:66ff:fe8a:214c/64 scope link valid_lft forever preferred_lft forever root@controller:~# ovs-vsctl show 77dc413c-c4a0-4c7a-b3e2-3ba8ecf5bee7 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-int Interface br-int type: internal Port qg-afb1772f-2f tag: 2 Interface qg-afb1772f-2f type: internal Port qr-6be4aabd-b9 tag: 3 Interface qr-6be4aabd-b9 type: internal Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port tap93e5731e-d9 tag: 2 Interface tap93e5731e-d9 type: internal Port tap52dcef2e-c9 tag: 3 Interface tap52dcef2e-c9 type: internal Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} Bridge br-tun Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port br-tun Interface br-tun type: internal Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Port br-ex Interface br-ex type: internal Port ens192 Interface ens192 ovs_version: "3.4.0" Thanks and Regards CH Kiran Raja M: +91 9959503256 [cid:1df6874c-4456-494d-9e7c-aa3aed63e7af]<https://pidatacenters.com/> [cid:693f6ef0-1ee9-4ea9-a345-8488a3aa8c2b]<https://harbour1.in/> kiran.raja(a)pidatacenters.com<mailto:kiran.raja@pidatacenters.com> [cid:0770d139-f2b6-424b-a451-6ad2a391a96b]

3 3

VPNaaS
by Vincent Godin 04 Feb '25

04 Feb '25

Hello, The VPNaaS provided with Openstack is a Site-to-Site VPN made to interconnect two tenants or an external gateway with a tenant network. I would like to know if there are plans to offer a VPNaaS more like a Remote VPN. This would allow Openstack users to administer their instances securely. A user could mount this VPN from a roaming device, be assigned a local IP address, and then administer the instances of his tenant. Regards Vincent

3 2

[watcher] core team expansion
by Sean Mooney 04 Feb '25

04 Feb '25

Hi everyone, As you may be aware a number of us from Redhat have been trying to revitalize the Watcher project. To enable that one of the changes that we previously discussed on the mailing list and ptg was the expansion of the core team. https://etherpad.opendev.org/p/r.79644f8f3cd04686c4f45c10aa35d8ac#L46 https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… In November the initial phase of that proposal was completed and I took an action to follow up in Q1 to see what steps are needed to rebuild a healthy core team. based on the activity of the last 90 days and the progress of work I think it is time to expand the core teams once again to enable effective maintenance and development of watcher https://openstack.biterg.io/app/dashboards#/view/95487340-6762-11e9-a198-67…:'Gerrit%20Approvals%20panel%20by%20Bitergia',filters:!(('$state':(store:appState),meta:(alias:Bots,disabled:!f,index:gerrit,key:author_bot,negate:!t,params:(query:!t),type:phrase),query:(match:(author_bot:(query:!t,type:phrase)))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:gerrit,key:project,negate:!f,params:(query:watcher),type:phrase),query:(match_phrase:(project:watcher)))),fullScreenMode:!f,options:(darkTheme:!f,hidePanelTitles:!f,useMargins:!t),query:(language:lucene,query:''),timeRestore:!f,title:'Gerrit%20Approvals',viewMode:view) tiny url version of ^ https://tinyurl.com/439wzbdk https://www.stackalytics.io/?module=watcher-group the current core teams are as follows * watcher-core https://review.opendev.org/admin/groups/77cd139bd57514e98e5e694a82f7e261b47… * watcher-dashboard-core https://review.opendev.org/admin/groups/bcc8588732437ceea3533249313e6e96148… * watcher-specs-core https://review.opendev.org/admin/groups/747af616eb8748aee5b17d9493f9ebbc49e… * watcher-tempest-core https://review.opendev.org/admin/groups/09a91d8e24af9ce44b80062c4851a1d2fa3… starting in order watcher-core: https://tinyurl.com/2p8sr7u4 https://www.stackalytics.io/?module=opendev.org/openstack/watcher - add Marios Andreou - add Douglas Viroel Marios and Doug both have past experience as core reviews in other OpenStack projects (triple-o and Manilla), over the last 90 days, they have engaged in reviews of bug fixes and the development of the new Prometheus data source. watcher-dashboard-core: https://tinyurl.com/4259h8nz https://www.stackalytics.io/?module=opendev.org/openstack/watcher-dashboard - add Marios Andreou - add Douglas Viroel as with Watcher itself Doug and Marios have spent time looking at the open review for the watcher-dashboard. watcher-spec-core: is tricky as we have effective only had one spec this cycle that was written by Marios with some minor patches for ci. As watcher core is a member of watcher-spec-core, Marios and Doug will inherit review rights on the specs repo. At this time I do not see a need to make any other changes to the group membership. watcher-tempest-core: https://tinyurl.com/4nbctef8 https://www.stackalytics.io/?module=opendev.org/openstack/watcher-tempest-p… - add Chandan Kumar - add Douglas Viroel - remove Martin Kopec As many of you may know Martin has taken up a new role in Redhat outside of OpenStack and will no longer be working directly on OpenStack. Over the last 90 days, Martin and Gmann have modernized the tempest plugin significantly paying down much of the tech debt. I want to thank them for the efforts they put in to help revive Watcher this cycle and if they return to the OpenStack community in the future I will gladly welcome them back. In their stead, I am proposing adding Chandan and Doug to augment the existing core group. As agreed in the Watcher team meeting earlier today we will review this mail thread in the next Watcher IRC meeting next week. I would like to invite others to provide feedback on this proposal over the next week. In the next PTG, I will also raise the composition of the core teams for discussion. It is my hope that with these additions we will have at least 3 active reviews on all deliverables of the watcher project mitigating the current bus factor and accelerating development. Regards Sean.

1 1

[ironic]About the OS supported by Diskimage-builder
by Junya Noguchi (Fujitsu) 03 Feb '25

03 Feb '25

To whom it may concern. My name is Junya Noguchi. I have some questions about the supported OS for Diskimage-builder operation. Please respond. References https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html Does it mean that the OS that says "work but receive only limited testing and support" works but is not supported? References https://docs.openstack.org/diskimage-builder/latest/ This document describes RHEL and other operating systems that are not supported in the previous question. Are these operating systems supported? Also, are there any plans to support more operating systems in the future? Best regards, Junya Noguchi

5 7

[neutron] neutron-core/networking-ovn-core consolidation
by Brian Haley 03 Feb '25

03 Feb '25

Hello Neutrinos: I noticed the other day we still have the networking-ovn-core group [0] included in the neutron-core group [1], a left-over from when we merged the OVN code into the Neutron repo in Ussuri. Most people in the networking-ovn-core group no longer contribute to Neutron, except for one - Terry Wilson (otherwiseguy). In order to cleanup a little I'm going to consolidate things and add Terry as a member of the neutron-core group, and remove the import. Since he has basically been operating as a Neutron core reviewer already for years, I don't think we need to vote on this. So welcome officially Terry! -Brian, Neutron PTL [0] https://review.opendev.org/admin/groups/ecdeda6cb5657af51f20b282cd0525cb7e6… [1] https://review.opendev.org/admin/groups/38

6 5

OpenStack BoF at FOSDEM
by Amy Marrich 03 Feb '25

03 Feb '25

Hey all, Sorry for the late promotion but we will be having an OpenStack BoF at FOSDEM[0] on Saturday at 16:00 in room AW1.121. Hope to see you there! Amy 0 - https://fosdem.org/2025/schedule/event/fosdem-2025-4885-openstack-community…

3 3

[glance][security][vmt] propose Cyril Roelandt for glance coresec
by Brian Rosmaita 31 Jan '25

31 Jan '25

I'd like to propose an update to the Glance Core Security Contacts team [0], which handles embargoed security issues for Glance project deliverables. The OpenStack Vulnerability Management Team prefers to keep the project core security teams small (because otherwise it kind of defeats the point of having embargoed private security issues). Thus, the changes I propose will keep the glance-coresec team at its current size: Add: Cyril Roelandt (croelandt on IRC). Cyril has been a Glance core since June 2021 and is a thorough and respected reviewer and developer; he also organizes the recurring Glance Review Party. Remove: Erno Kuvaja (jokke on IRC). Erno has been focusing his time away from Glance. On behalf of the Glance project team, I thank Erno for his past service on glance-coresec, and note that Erno continues to participate in the Glance project as a "regular" glance core. If you have any concerns about this change, please communicate them to me or Pranali Deore (Glance PTL) before 23:59 UTC on Wednesday, 29 January 2025. cheers, brian [0] https://launchpad.net/~glance-coresec

1 1

[manila] driver_handles_share_servers not recognized as boolean
by Francesco Di Nucci 31 Jan '25

31 Jan '25

Hi all, I'm running OpenStack 2024.1 Caracal, configured through Puppet, and I'm having an issue with Manila. /etc/manila/manila.conf contains the line "driver_handles_share_servers = True", but setup fails, "/var/log/manila/share.log" reports that "Config opt 'driver_handles_share_servers' has improper value - 'None'. Please define it as boolean." Has anyone encountered this? Where else should I define this? Thanks in advance -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email: francesco.dinucci(a)na.infn.it

3 9

[all] Is oslo.vmware still relevant?
by Herve Beraud 31 Jan '25

31 Jan '25

Hey folks, I observed that oslo.vmware has not received any significant modifications since almost 4 years ago [1]. The only changes proposed to oslo.vmware since a while are just related to infra changes (python versions, tox versions, etc...). Due to the recent news about vmware [2], I wonder if oslo.vmware is still a relevant library in our context and I wonder if it is worth spending time on maintaining it. Do you think we should deprecate and retire this library? Thanks in advance for your opinions. [1] https://opendev.org/openstack/oslo.vmware/commits/branch/master [2] the broadcom story -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

5 6

[neutron] Drivers meeting agenda for Friday, January 31st
by Brian Haley 31 Jan '25

31 Jan '25

Hello Neutrinos: Sorry for the late notice but this is the agenda for tomorrow's drivers meeting: * [RFE] Allow rebalancing of OVN LRPs * https://bugs.launchpad.net/neutron/+bug/2096704 * We quickly discussed this in our weekly meeting and it seemed like a good idea See you then, -Brian

1 0

[manila] Feb 5th and 6th mid cycle
by Carlos Silva 30 Jan '25

30 Jan '25

Hello, zorillas and interested stackers, Manila's 2025.1 Epoxy mid cycle is right around the corner. We will meet next on February 5th and 6th, from 14 to 16 UTC on meetpad [1]. Please add your topics to the mid cycle etherpad [2]. We will use the second day of the mid cycle for collaborative review sessions, so in case you are proposing a feature or would like to have reviewers attention in a specific change, please bring it up and walk us through your change. [1] https://meetpad.opendev.org/manila-epoxy-mid-cycle [2] https://etherpad.opendev.org/p/manila-epoxy-mid-cycle Looking forward to it, carloss

1 0

OS-EXT-SRV-ATTR incomplete in compute api
by Jan 30 Jan '25

30 Jan '25

Hi, is there a reason why some member variables of the nova instance object are returned by the api and some not. For example the "os_type" value is not returned but the "kernel_id" (OS-EXT-SRV-ATTR:kernel_id) is. I have a use case where it would be very helpful to get the "os_type" via the api and python sdk. Is there a chance that a patch implementing this would be accepted? Regards, Jan

3 4

Skip upgrades
by engineer2024 30 Jan '25

30 Jan '25

Is there a way to upgrade openstack older versions like ussuri , victoria, by migrating just the nova, neutron dbs, schema to the latest version by skipping one level at once and then install the latest venvs against the latest migrated galera dbs. On the compute nodes, if we preserve just the var lib nova instances and on top of it reinstall the compute and neutron agents, will it work ? I am asking becos, I don't know what r the dependencies among the major nova versions. Are the db schema and data, the only diffs from one version to another, or is there any thing else ?

5 5

[keystone] Weekly meeting canceled 29-Jan-25
by Dave Wilde 29 Jan '25

29 Jan '25

Hi folks, Apologies for the late notice but I need to cancel the Keystone weekly meeting today due to a scheduling conflict. Please reach out either here or via IRC if you need anything. Thanks!! /Dave

1 0

[rabbitmq] Need help about RabbitMQ quorum HA mode
by İzzettin Erdem 29 Jan '25

29 Jan '25

Hi, In kolla-ansible yoga environment, rabbitmq queues are set to quorum by default instead of classical mirroring, as it seems they are not replicated (per queue has +0 quorum). We tried to find a solution to handle it in configs, but we couldn't find it and we set queues to be replicated by using CLI commands (rabbitmq-queues grow). Now, my questions are: 1- Why do all rabbitmq queues come as non-replicated by default? 2- How can we set HA mode to quorum queues?

5 5

Nova
by engineer2024 29 Jan '25

29 Jan '25

Does nova support flavors where vcpus of a single flavor can be in mixed modes where some of them are pinned and the rest dedicated ? Say, a flavor has 5 vcpus, I want 2 of them to be of dedicated type and the remaining 3 vcpus to be shared

2 2

[neutron][upgrades] IPv6 Prefix Delegation 'dibbler' driver removal
by Brian Haley 29 Jan '25

29 Jan '25

Hi again Stackers, In 2023, during the Bobcat cycle, the Neutron team changed the IPv6 Prefix Delegation (IPv6 PD) 'dibbler' driver to be an experimental feature [0]. The reason was due to it (dibbler) becoming unmaintained/retired upstream, as well as the low level of testing support we had in Neutron for the feature. So what does this mean? We (the neutron cores) have come to the conclusion that it is time to take the next step and remove the driver from the tree [1]. The framework to support IPv6 PD is *not* being removed, but as this is the only in-tree driver the functionality will no longer be available to end-users. We plan on doing this in the current Epoxy cycle. We realize that there may be deployments using IPv6 PD, although we have not to my knowledge ever received a bug against it, so we think any usage is very small. If there is desire to add an in-tree driver back to the code base, there is an open OVN-related bug for that at [2], we have just not had the interest from downstream customers to warrant the investment in this area. Thanks for reading, -Brian, Neutron PTL [0] https://review.opendev.org/c/openstack/neutron/+/879030 [1] https://review.opendev.org/c/openstack/neutron/+/934283 [2] https://bugs.launchpad.net/neutron/+bug/1895972

1 0

[Octavia][magnum] change timeout value
by Satish Patel 28 Jan '25

28 Jan '25

Folks, I am running magnum-cluster-api (mcapi) with openstack to spin up k8s but now I want to change octavia LB timeout. Is there a way to tell magnum to change the timeout value of LB? loadbalancer.openstack.org/timeout-client-data: "120000" loadbalancer.openstack.org/timeout-member-data: "120000"

1 0

[Nova] Nova-compute service flapping on Antelope
by Jakub Darmach 28 Jan '25

28 Jan '25

Hello, Recently I encountered an interesting issue - nova-compute service started to temporarily lose Rabbit connectivity to regain it after a few seconds on Antelope. The issue was replicated on the test environment, without specific steps to replicate though - it just starts to happen after some time. I pasted a log example in the bug I opened [1]. First we can see rabbit logging closing AMQP connection, followed by nova-compute reporting rabbit server being unreachable, with next message being successful reconnection. Restarting nova-compute services helps temporarily, the issue starts to manifest after a few days. Initially disconnects once every few hours, each time in shorter intervals, to the point it happens every minute or so. Did anyone encounter something similar? [1] https://bugs.launchpad.net/nova/+bug/2092297 Pozdrawiam / Best regards, *Jakub Darmach*

3 2

[administrivia] Bounce processing disabled for now
by Jeremy Stanley 28 Jan '25

28 Jan '25

Due to repeated challenges with random messages being rejected by subscribers' mail providers (from DMARC invalidations, lines too long to be forwarded, content misidentified as spam, et cetera), the time required for list administrators to manage the resulting extra work is greater than the benefit we get from Mailman being able to figure out when subscriber addresses become defunct. As such, I've disabled the "bounce processing" feature for the time being, until we can work out solutions to make it less of a burden on our list admins and less confusing/failure prone for the list subscribers. If we do decide to give it another try at some point in the future, after some of the current challenges with it get ironed out, I'll let the list know. -- Jeremy Stanley

1 0

openstack-discuss-owner mailbox
by Albert Braden 28 Jan '25

28 Jan '25

Is anyone manning the openstack-discuss-owner mailbox? I've been writing to that address about the recent glitch that deleted my list subscription and account (and according to the error message, hundreds of others), but I'm not getting a response.

2 3

[publiccloud-sig] Public Cloud SIG Meeting this week
by Felix Kronlage-Dammers 28 Jan '25

28 Jan '25

Hi everyone, This upcoming Wednesday (January 29th) the next meeting of the publiccloud-sig is going to happen or at least would happen: However I’ll not be around tomorrow to facilitate and I did not get any feedback from anyone else on it. Usually we would be meeting at 0800 UTC here: https://meetpad.opendev.org/publiccloud-sig - so give it a shot ;) See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG felix -- Felix Kronlage-Dammers Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[nova] Performance Issues with OpenStack Server List on os-compute-api-version 2.96
by 黒羽正徳 [Kuroha Masanori] 28 Jan '25

28 Jan '25

Hello, We have recently encountered a situation where the response time for the openstack server list command is significantly slower when specifying os-compute-api-version 2.96 for certain projects. In comparison with version 2.95, some projects experience a response time that is 2-3 times longer, while other projects show little to no difference. We understand that in version 2.96, there was a change to obtain the availability zone from the request_spec [1]. However, upon checking the instance records in request_spec for the projects, we have not identified any differences that would account for this slow response behavior. Additionally, we have observed that when certain ERROR VM is present, the response time for the server list can become considerably slower for specific projects. In one case, we resolved the slow response issue by deleting instances that did not have corresponding fault records in the database. However, the exact reason why this action resolved the issue remains unclear. If anyone has more detailed information or potential solutions regarding these issues, or if you have encountered similar situations, we would greatly appreciate your insights. [1] https://git.linecorp.com/Flava-IaaS/nova/commit/c98c8d84eec6e65f1b67ae1ab82… Best regards, Masanori Kuroha

1 1

[largescale-sig] Next meeting: January 15, 9utc
by Thierry Carrez 28 Jan '25

28 Jan '25

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 9UTC. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20250115T09 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

3 3

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-9)
by Goutham Pacha Ravi 28 Jan '25

28 Jan '25

Hello Stackers, We're 9 weeks away from OpenStack's 2025.1 coordinated "Epoxy" release [1]. These weeks leading up to feature freeze (R-5/Feb 24-28) will be about "getting things done" for project teams. As Slawek Kaplonski (slaweq) wrote to you [2], we'll be kicking off combined elections for Project Team Leads for the 2025.2 "Flamingo" release cycle and five seats on the OpenStack Technical Committee next week. In light of this, please confirm your eligibility to vote in these elections by navigating to the CIVS website and entering your Gerrit email address: https://civs1.civs.us/cgi-bin/opt_in.pl In each of the past three elections, we've had roughly a fifth of eligible voters cast their votes. It would make a great difference to our community if we could improve this statistic with your cooperation! In the past week, the OpenStack Technical Committee approved a resolution to welcome the "Freezer" (Disaster-Recovery-as-a-Service) project back into the fold of "active" projects [3]. I'd like to extend a shout-out to Dmitriy Rabotyagov (noonedeadpunk) and Álvaro Soto (khyr0n) for their work to revive the project team. The team is looking to pursue the release of several project deliverables during this release. === Weekly Meeting === The TC weekly IRC meeting on 2025-01-21 [4] was well attended. We were in "getting things done" mode too, as much of the meeting followed up on action items from past meetings. We discussed getting more active review participation for changes to the "openstackdocstheme" repository by working with the Oslo team. We also deliberated on election nomination windows. We are in favor of early nominations from folks who are unable to self-nominate due to holidays or other personal reasons to be away from their computers during the nomination window. However, there were concerns expressed that the election officials may be burdened by the work necessary to shepherd such nominations. Early nominations would fail several CI checks, require re-checks, and potentially create some confusion. We'll attempt to watch this closely for the next election cycle and recalibrate. Our goal is to avoid leaderless projects due to holiday cycles during elections. The TC also discussed the next steps for the newly active Freezer project to release artifacts during the current release cycle. The next meeting of the OpenStack TC is today, 2025-01-28, at 1800 UTC in the #openstack-tc IRC channel on OFTC. Please find the meeting's agenda on the meeting wiki [5]. I hope you'll be able to join us. === Governance Proposals === ==== Merged ==== - Retire Freezer DR | https://review.opendev.org/c/openstack/governance/+/938183 - Remove Freezer from inactive state | https://review.opendev.org/c/openstack/governance/+/938938 ==== Open for Review ==== - Retire qdrouterd role | https://review.opendev.org/c/openstack/governance/+/938193 - Reset the DPL model for Oslo project | https://review.opendev.org/c/openstack/governance/+/939485 - Reset the DPL model for Release project | https://review.opendev.org/c/openstack/governance/+/939486 - Add devstack-plugin-prometheus repo to Quality Assurance repos | https://review.opendev.org/c/openstack/governance/+/939953 - Reset the DPL model for Watcher project | https://review.opendev.org/c/openstack/governance/+/939488 - Add Watcher project update liaison | https://review.opendev.org/c/openstack/governance/+/940168 - Adopt DPL model for Watcher 2025.2 | https://review.opendev.org/c/openstack/governance/+/940167 - Define 2025 upstream investment opportunities | https://review.opendev.org/c/openstack/governance/+/939507 - Add resolution for late Freezer release | https://review.opendev.org/c/openstack/governance/+/939792 - Deprecate ironic-lib | https://review.opendev.org/c/openstack/governance/+/939278 - Resolve to adhere to non-biased language | https://review.opendev.org/c/openstack/governance/+/934907 === Upcoming Events === - 2025-01-28: OpenInfra Board Monthly Meeting: https://board.openinfra.org/ - 2025-02-01: FOSDEM 2025 (https://fosdem.org/2025/) OpenStack's 15th Birthday Celebration - 2025-02-05: Nominations open for 2025.2 TC/PTL Elections: https://governance.openstack.org/election/ - 2025-02-28: 2025.1 ("Epoxy") Feature Freeze and release milestone 3 [1] - 2025-03-06: SCALE 2025 + OpenInfra Days NA (https://www.socallinuxexpo.org/scale/22x) Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] 2025.2 Election Kickoff: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] Remove Freezer from inactive state: https://review.opendev.org/c/openstack/governance/+/938938 [4] TC Meeting IRC Log 2025-01-21: https://meetings.opendev.org/meetings/tc/2025/tc.2025-01-21-18.00.log.html [5] TC Meeting Agenda, 2025-01-28: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[mistral] meetings during next PTG
by Arnaud Morin 28 Jan '25

28 Jan '25

Hey all, Next PTG will take place between April 7 to 10. Is there anyone willing to talk about mistral developement during this PTG? So far, mistral has not evolved that much, but maybe someone is having ideas about it? Also, there is this eventlet-removal topic that could be discussed. Let me know, Cheers, Arnaud.

1 0

[neutron] Bug Deputy Report Jan 20 - Jan 26
by Yatin Karel 27 Jan '25

27 Jan '25

Hi Neutrinos, Below is the bug deputy report for last week:- High ------ - https://bugs.launchpad.net/neutron/+bug/2095807 - OVN loadbalancer via FIP does not work for DVR + VLAN networks Unassigned OVN octavia provider, Asked Fernando to have a look as could be known one - https://bugs.launchpad.net/neutron/+bug/2095598 - test_async_process_respawns functional test failing in fips functional job Unassigned We thought it would get fixed with https://review.opendev.org/c/openstack/neutron/+/939627 but it's not, so have to be checked again - https://bugs.launchpad.net/neutron/+bug/2095595 - The neutron-dhcp-agent resync throttling mechanism doesn't work Assigned to Anton Kurbatov - https://bugs.launchpad.net/neutron/+bug/2095590 - [OVN] Neutron API stops refreshing the hash ring node Assigned to Rodolfo - https://bugs.launchpad.net/neutron/+bug/2095566 - [OVN] Router creation timeouts randomly Unassigned Rodolfo mentioned he has some fixes in flight to cover ovn wsgi related issue, so to confirm if this is also covered. Medium ---------- - https://bugs.launchpad.net/neutron/+bug/2095576 - [OVS] False ERROR log message every time when creating VM Unassigned - https://bugs.launchpad.net/neutron/+bug/2095322 - ADMIN_OR_NET_OWNER_{MEMBER|READER} rules don't work for subnets Unassigned Incomplete --------------- - https://bugs.launchpad.net/neutron/+bug/2096704 - [RFE] Allow rebalancing of OVN LRPs Asked for more context -- Thanks and Regards Yatin Karel

1 0

[all][elections][ptl][tc] Combined PTL/TC 2025.2 cycle Election Season
by Sławek Kapłoński 27 Jan '25

27 Jan '25

Hi, Election details: https://governance.openstack.org/election/ The nomination period officially begins Feb 05, 2025 23:45 UTC. Please read the stipulations and timelines for candidates and electorate contained in these governance documents[1][2][3]. Due to circumstances of timing, PTL and TC elections for the coming cycle will run concurrently; deadlines for their nomination and voting activities are synchronized but will still use separate ballots. Please note, if only one candidate is nominated as PTL for a project team during the PTL nomination period, that candidate will win by acclaim, and there will be no poll. There will only be a poll if there is more than one candidate stepping forward for a project team's PTL position. Similarly, there will only be a TC poll if there are more than 5 candidates stepping forward the available TC positions. All members of the community are encouraged to confirm their email address as described at: https://governance.openstack.org/election/#electorate There will be further announcements posted to the mailing list as action is required from the electorate or candidates. This email is for information purposes only. If you have any questions which you feel affect others please reply to this email thread. If you have any questions that you wish to discuss in private please email any of the election officials[4] so that we may address your concerns. Thank you, [1] https://governance.openstack.org/tc/reference/charter.html [2] https://governance.openstack.org/tc/reference/role-of-the-tc.html [3] https://docs.openstack.org/project-team-guide/ptl.html [4] https://governance.openstack.org/election/#election-officials -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[openstack-helm] Use Loci images by default
by Vladimir Kozhukalov 27 Jan '25

27 Jan '25

Hi, This is to announce multiple things: 1) OpenStack/Loci (a project for building OpenStack images) is currently equipped with periodic jobs that are run nightly. These jobs build images for 3 most recent OpenStack releases: - 2023.2 Bobcat - 2024.1 Caracal - 2024.2 Dalmatian Images are published to these two destinations: - docker.io/loci/* (which is known for its strict request rate limits) - quay.io/airshipit/* (which is much more tolerant to the number of requests) Images are tested before publishing with two basic deployment configurations: - compute-kit (Deploy minimal set of OpenStack components to run VMs) - cinder (Deploy Cinder with Ceph backend) 2) By default we are going to use Loci images published to the Quay registry [1]. All test jobs will be run with these images and this is also a recommendation for users to consume images from quay to mitigate possible rate limit issues. 3) OpenStack/OpenStack-Helm-Images project will build only non-OpenStack images. Primarily the motivation is to deduplicate the code base. Consequently, OpenStack images published nightly to docker.io/openstackhelm/* will not be updated after 2024.2. [1] https://review.opendev.org/c/openstack/openstack-helm/+/939426 -- Best regards, Kozhukalov Vladimir

2 2

Guidance Required for OVN-BGP-Agent Configuration with EVPN Mode
by AkashMadan Talebailkar 26 Jan '25

26 Jan '25

Dear Community, I hope this email finds you well. I came across this insightful discussion on the OpenStack forum regarding the OVN-BGP-Agent Discussio<https://lists.openstack.org/pipermail/openstack-discuss/2022-August/030080.…>n. We are doing research on openstack capability and would greatly appreciate your guidance. We are utilizing the OVN mechanism driver along with the OVN-BGP-Agent<https://github.com/openstack/ovn-bgp-agent/tree/master> in EVPN mode. Our objective is to establish BGP-EVPN between our leaf switch and the OpenStack controller. The OVN-BGP-Agent should advertise the IP and MAC addresses of VMs to its BGP peers. Please find attached the following details for your reference: bgp-agent.conf Agent configuration OVN_DB configuration Result We installed the OVN-BGP-Agent and FRR on the controller. Below are the steps we followed: Network Setup: Created a tenant network (NET) with a subnet of 121.168.0.0/24. Created a provider network (EXNET) with a subnet of 192.27.5.0/24. Router Configuration: Set up a router (newrouter) with an external network (EXNET) and unchecked SNAT. Added the NET interface to the router. OVN Database Configuration: Configured the OVN Northbound DB (detailed in the attachment). VM Creation: Deployed a VM in NET with the VM's IP 121.168.0.202 PFA for the agent configuration and FRR configuration done once agent VM has been created. Despite following the steps above, the VM IP is not visible in FRR, PFA for the result. Consequently, we are unable to advertise its route via BGP-EVPN. We kindly request your guidance to identify any potential misconfigurations or missing steps. Thank you for your time and support. Best regards, Akash Talebailkar Palc Networks akashmadan.t(a)palcnetworks.com +91 9035376555

2 1

[manila][cephfs][kollla-ansible]cannot create access rule on share
by Nguyễn Hữu Khôi 26 Jan '25

26 Jan '25

Hello. I setup manila with cephfs driver-CephFS NFS shares and I use NFS-Ganesha based “ceph nfs” service. This is my configuration: [DEFAULT] debug = true enabled_share_protocols = NFS,CIFS,CEPHFS [cephfsnfs1] driver_handles_share_servers = False share_backend_name = CEPHFSNFS1 share_driver = manila.share.drivers.cephfs.driver.CephFSDriver cephfs_protocol_helper_type = NFS cephfs_conf_path = /etc/ceph/ceph.conf cephfs_auth_id = manila cephfs_cluster_name = az1 cephfs_filesystem_name = cephfs I can create a share but cannot set access rule for it and I encountered this error in manila-share.log: 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.10/site-packages/manila/share/drivers/ganesha/manager.py", line 472, in add_export 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server self._dbus_send_ganesha("AddExport", "string:" + path, 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.10/site-packages/manila/share/drivers/ganesha/manager.py", line 410, in _dbus_send_ganesha 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server self.execute("dbus-send", "--print-reply", "--system", 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.10/site-packages/manila/share/drivers/ganesha/manager.py", line 240, in _execute 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server raise exception.GaneshaCommandFailure( 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server manila.exception.GaneshaCommandFailure: Ganesha management command failed. 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server Command: sudo manila-rootwrap /etc/manila/rootwrap.conf dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/etc/ganesha/export.d/share-5d49f7c6-27a0-43c3-931d-56486b8f3d51.conf string:EXPORT(Export_Id=106) 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server Exit code: 96 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server Stdout: '' 2025-01-25 22:00:11.895 774 ERROR oslo_messaging.rpc.server Stderr: '/var/lib/kolla/venv/bin/manila-rootwrap: Executable not found: dbus-send (filter match = dbus-addexport)\n' One more thing, when I run openstack share export location list myshare I see lab-control01:/volumes/_nogroup/5d49f7c6-27a0-43c3-931d-56486b8f3d51/fb094ad8-9200-4c93-b800-304178f51136 lab-control01 is my openstack controller, is it right because nfs service is on ceph cluster. Thank you. Nguyen Huu Khoi

1 1

[all][PTLs][release] OpenStack 2025.2 Flamingo release schedule plan
by Elõd Illés 24 Jan '25

24 Jan '25

Hi everyone, Let me inform you that 2025.2 Flamingo release schedule plan can be reviewed: https://review.opendev.org/c/openstack/releases/+/939528 (or for better reading you can take a look at the generated preview for Patch Set 1 [1] but note that the link **only** works for Patch Set 1!) [1] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/… Cheers, Előd Illés irc: elodilles @ #openstack-release

1 0

[kolla-ansible] Multipath unsuccesfull in nova
by Kamil Madac 24 Jan '25

24 Jan '25

Hello, I have a openstack deployed with kolla-ansible 2024.2. I'm trying to configure cinder and nova to use multipath when attaching volumes from the Hitachi SAN array. Without multipathing everything works good. When I enabled multipath in kolla-ansible (enable_multipathd: "yes"), deployment is without errors, volume can be created, but attachment of volume ends up with error in nova_compute container: ... 2025-01-22 12:16:02.962 2 ERROR nova.virt.block_device [instance: 714bd6f8-49e5-406f-b608-b7aaded2e728] raise exc_type(*result[2]) 2025-01-22 12:16:02.962 2 ERROR nova.virt.block_device [instance: 714bd6f8-49e5-406f-b608-b7aaded2e728] oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2025-01-22 12:16:02.962 2 ERROR nova.virt.block_device [instance: 714bd6f8-49e5-406f-b608-b7aaded2e728] Command: multipath -C dm-0 2025-01-22 12:16:02.962 2 ERROR nova.virt.block_device [instance: 714bd6f8-49e5-406f-b608-b7aaded2e728] Exit code: 1 2025-01-22 12:16:02.962 2 ERROR nova.virt.block_device [instance: 714bd6f8-49e5-406f-b608-b7aaded2e728] Stdout: '' 2025-01-22 12:16:02.962 2 ERROR nova.virt.block_device [instance: 714bd6f8-49e5-406f-b608-b7aaded2e728] Stderr: '3601817.750747 | /etc/multipath.conf does not exist, blacklisting all devices.\n3601817.750753 | You can run "/sbin/mpathconf --enable" to create\n3601817.750754 | /etc/multipath.conf. See man mpathconf(8) for more details\n3601817.751393 | sdd: error 2 in pathinfo, discarding path\n3601817.751501 | sde: error 2 in pathinfo, discarding path\n3601817.751504 | 360060e8012b142005040b14200000ee2: removing empty pathgroup 0\n3601817.751505 | dm-0: no usable paths found\n' When I manually added /etc/multipath.conf to nova_compute container, the attachment passed without any issues. I do not want to build special image only because of that single config file. My question is, how to make kolla-ansible to copy/add /etc/multipath.conf file into nova_compute service? Is there any mechanism for that? Thanks. Kamil Madac

3 4

[nova][neutron][ops] disabling the heal instance info cache periodic
by Sean Mooney 24 Jan '25

24 Jan '25

hi folks o/ This is a short PSA and request for comments The nova and neutron teams have discussed the existence and importantly the default behavior of nova's "heal instance info cache periodic" on an off for the better part of the last 4+ years but we never had time to take any action on it until now. Context for those who are not aware of it, nova has an instance info cache which we use to cache networking information such as ip address related to ports associated with an instance. This was introduced in the Essex release when nova networks was being replaced with quantum/neutron. Its rater fitting but over due that in the Epoxy release we are now turning this off. for more context i have captured some of the important point in the release note of the proposed change https://review.opendev.org/c/openstack/nova/+/939476/2/releasenotes/notes/d… But ill provide a TL;DR here. By default, as of essex, [compute]heal_instance_info_cache_interval defaulted to 60. it has been possible to disable this by setting the value to <=0 (recommended to use -1 by convention). when the periodic is enabled, every nova compute agent loop over the instance on the current host, retrieves all there ports form neutron and fully rebuilds the cache(as of stein). on a small cloud (less then 50 computes) this has minimal impact on the performance of the neutron server, on moderate to large clouds this creates 10s of MB of constant db read load. This has several side-effects: - first it creates unnecessary load on the neutron api and neutron db/network backend in creasing latency and reducing overall performance. - second, it create unnecessary load on rabbitmq as each compute service need to do an RPC to update the info cache(every 60 seconds). - third both of the first two waste power doing extra work that is not needed. - finally as of Icehouse this periodic has been optional because we added the os-server-external-events api so that neutron can tell nova if there has been a change to refresh the cache as needed. With that in mind hte nova team is proposing that we disable this going forward, for this release i have not deprecated the periodic so operators can re-enable it if they find its required for any reason. in the 2026.1 release we will likely revisit this based on operator feedback/bug reports. In this weeks team meeting we agree to wait 2 more weeks to get feedback from OPS if they had any concerns with this change hence this email. If anyone knows of a reason why we should not proceed with this default change feel free to let us know via this email thread or the gerrit review https://review.opendev.org/c/openstack/nova/+/939476 regards sean

2 1

April 2025 PTG Dates & Team Signup Kickoff
by Kendall Nelson 24 Jan '25

24 Jan '25

Hello Everyone, I am pleased to announce the next OpenInfra PTG[1] will be taking place April 7-11, 2025! Registration will open soon! To sign up your team, you must complete the survey[2] by February 21st at 7:00 UTC. We NEED accurate contact information (email and and IRC nickname ideally) for the moderator of your team’s sessions. This is because the survey information will be used to organize the schedule signups which will be done via the PTGBot. If you are not on IRC, please get setup[3] on the OFTC network and join #openinfra-events. You are also encouraged to familiarize yourself with the PTGBot documentation[4] as well. If you have any questions, please reach out to ptg(a)openinfra.dev! Information about signing up for time slots will be sent to moderators after the team signup deadline. Continue to visit openinfra.org/ptg for updates. -Kendall (diablo_rojo) [1] PTG Site: https://openinfra.org/ptg/ [2] Team Survey: https://openinfrafoundation.formstack.com/forms/ptg_april_2025_survey [3] Setup IRC: https://docs.openstack.org/contributors/common/irc.html [4] PTGBot README: https://opendev.org/openstack/ptgbot/src/branch/master/README.rst

1 0

[neutron] Drivers meeting canceled Friday, January 24th
by Brian Haley 24 Jan '25

24 Jan '25

Hello Neutrinos: I'm going to cancel tomorrow's drivers meeting as there is no agenda. Just a reminder there are still open specs that need review, list is at [0]. Have a nice weekend! -Brian [0] https://review.opendev.org/q/status:open+project:openstack/neutron-specs

1 0

[cinder] 2025.1 Midcycle reminder
by Jon Bernard 24 Jan '25

24 Jan '25

Hello all, this is a reminder that our midcycle meeting will take the place of next week's upstream meeting on 29 Jan. I've allocated 2 hours (as opposed to our usual 1) in case additional time is needed. The etherpad we'll be using is here [1]. See you all then! [1]: https://etherpad.opendev.org/p/cinder-epoxy-midcycles -- Jon

1 0

[Neutron][operators] LinuxBridge to OVN Outreach
by Daniel Failing 24 Jan '25

24 Jan '25

Dear operators of Neutron deployments using LinuxBridge, This mail is to target operators who plan to move away from neutron LinuxBridge as well. In our setup we are currently planing to move away from LinuxBridge and use OVN instead. Based on the last PTGs survey with quite a few deployments using LinuxBridge and it being marked experimental for some time, we feel there could be a common ground and pain points. We were wondering how many other operators have the same problems and want to share knowledge and if necessary consolidate code written for a potential migration. Knowing there are quite some different deployments, we most likely will not be able to find the golden solution that works for everyone, but we could strive for not making the same mistakes multiple times in different deployments. We are located in European timezones, if there are others interested in a zoom/jitsi call, please let me know your timezones/time preferences, so we find a good slot. Otherwise I can propose one of the following dates already: Friday, Jan 24st 2 or 3pm UTC Tuesday, Jan 28st 10am UTC Tuesday, Jan 28st 3:30pm UTC Wednesday, Jan 29st 10am UTC Friday, Jan 31st 2 or 3pm UTC Best, Daniel CERN Cloud Infrastructure [1] original thread: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

3 3

[openstack-ansible]:Wallaby to Xena Upgrade
by Rambo Rambo 23 Jan '25

23 Jan '25

Hi Team I am planning openstack-ansible wallaby to Xena upgrade. Distribution: openstack-ansible Release: Wallaby (current) ==> Xena (Planned) I was checking the release notes of Xena and could see that there is introduction of SSL in several services with nova for live migrations one of them. https://docs.openstack.org/releasenotes/openstack-ansible/xena.html As per the release notes: - Nova now defaults to to using the “QEMU-native TLS” feature for live migrations, rather than the deprecated SSH method. A pre-existing PKI (Public Key Infrastructure) setup is required. QEMU-native TLS requires all compute hosts to accept TCP connections on port 16514 and port range 49152 to 49261. More information can be found here: https://docs.openstack.org/nova/latest/admin/secure-live-migration-with-qem… Can you please confirm what steps are required to setup this PKI before doing the upgrade or it will be taken care by internal-pki using openstack-ansible upgrade method? Is there any option to skip this QEMU-native TLS for live-migrations and keep the orginal ssh based live-migrations? Regards

2 5

[all][oslo] revisiting the proposal to deprecate eventlet from oslo.service
by Mike Bayer 23 Jan '25

23 Jan '25

Hi - We've come to an impasse regarding our efforts to create a new backend selection system for oslo.service which will allow applications based on eventlet to have an incremental migration path where they can select for threads instead of eventlet [1] [2]. Briefly, the change organizes oslo.service into two different "backends", known as "eventlet" and "threaded". A calling application may choose to use either backend exclusively. Applications that use oslo.service right now are essentially using the "eventlet" backend. As these applications seek to migrate away from eventlet, they'd select the "threaded" backend and tune their applications to work under this new mode, where eventually the "eventlet" backend would be removed from oslo.service and ultimately from all of openstack entirely. All of that falls under things that have been agreed upon. The current impasse is among three options of how this "backend" system in oslo.service may be informed of which backend should be used: 1. being selected at runtime via CONF from the .cfg file , which is what was originally agreed upon 2. at start up time via environment variable 3. as a fixed setting in source code. Personally, as someone who has worked for decades on multiple switchable backend-oriented systems, I'm fully in the camp of choice 3, which is that switching the backend is just the beginning of a whole series of changes and tuning that a codebase would need in order to accommodate this backend change, and having the ability to "switch the backend back to eventlet" in CONF or runtime is simply not something anyone would want to do, unless a considerable amount of additional effort were made to support the codebase being reliable under both idioms. Which I would argue is a waste of effort since eventlet is going away. More concretely, if you took nova, which were using oslo.service with eventlet and decided to try the "threaded" backend, that's naturally going to have a lot of implications for other parts of the code. For example, if your code is setting eventlet monkey patching, that has to be turned off when this backend selection is made. If your code has other explicit use of eventlet, this also needs to be changed. Finally, threaded concurrency is naturally going to "feel" different than coroutine-based concurrency - context switches happen at different places and with different frequencies. Typically in a large codebase, the places that are prone to race conditions can shift; areas that had no races under coroutines now have threaded concurrency problems, areas that were sensitive to coroutine issues like being CPU bound are suddenly smooth with threading. The overall number of "working routines" can change, whereas eventlet can spin up 1000 coroutines like popcorn, under threads we now have to work with pools of threads typically a few dozen or so threads large. Overall, if I had my whole application passing under tempest under "threading", I would have had to make changes to get there, which may very well be mutually exclusive vs. test resiliency under eventlet. More generally, getting my application to run under threads represents a change in implementation that I expect to be permanent. If there are continued issues running under threaded mode, those are bugs that have to be fixed in any case. In the discussion at [1], Takashi Kajinami writes that a PTG discussion (for which I dont have any link or background) explicitly decided against the above idea: > In the initial design we considered that developer may make the decision to migrate from eventlet to the new mechanism, and didn't intend to provide any options to switch back to the original eventlet model. > However there was some strong push back during the previous PTG and people were concerned with having no mechanism to switch back to "previously worked well" eventlet model and to address the concern we agreed to introduce the option for operators to switch the backend. > If we don't allow users to configure the option then IMO we should not expose the option and we should use an internal flag instead (though that's basically what was disagreed with during the PTG, AFAIR) So first off, we have already realized that going with option 1. , use CONF, is not tenable, because the CONF collection is not typically in its final populated state when imports occur, and since the "backends" are an import time selection, there's a chicken/egg problem that cannot be resolved unless applications either highly modify the way in which their applications consume CONF vs. do their basic imports, or the way we are writing our backends in [1] needs to be modified so that imports can proceed using proxy objects which then change their loaded implementation when CONF is set up. Both of these options seem deeply complex and overarchitected in order to support the feature of "I want to change the backend in CONF" which I maintain is not going to be a real world use case. As a compromise, I suggested option 2. using an environment variable. This allows the backend selection to remain something that can be theoretically changed from the outside without modifying source code. However, current deployment practices suggest this is not really useful, as again per Takashi the way in which we deploy mod_wsgi does not allow environment variables to be local per virtual host; per [3] it looks like you need a hardcoded python script anyway that sets the env the way you want which makes it not too useful. As we were proceeding on option 2. we received some more pushback on the "env" idea from Arnaud Morin which I don't disagree with, and still more pushback and arguments leading in favor of 3, here from Jay Faulkner: > Using environment variables to set backends means that operators can change what backend is used. This is an easy thing for them to screw up and will end up creating a large amount of operator pain and bug churn. I am -1 in the strongest terms to the use of operator-adjustable values to enable/disable backends. The above was surprising considering that this was supposed to be a decision that had already been made at PTG, yet there still seems to be disagreement as though the decision were not actually made in any final way. So as I am tasked along with Daniel Bengtsson and Herve Beraud with getting [1] merged and moving onto building out the threaded backend, I would like to ask the group here to give me some background on the concerns raised at PTG and if we can all just here revisit the whole issue and hopefully decide that at least to start, let's get this merged without any CONF/env variable process (option 3); an application that's gone through the effort to transition to threads with the help of this backend selector should be assumed to be moving forward with that implementation, and if it has problems, that's just an ordinary bug like any other. thanks for reading! [1] https://review.opendev.org/c/openstack/oslo.service/+/935783 [2] https://review.opendev.org/c/openstack/oslo-specs/+/927503 [3] https://gist.github.com/GrahamDumpleton/b380652b768e81a7f60c

10 11

[kolla] Deprecating swift support
by Michał Nasiadka 23 Jan '25

23 Jan '25

Hello, Kolla and Kolla-Ansible is deprecating Swift deployment support, due to absence of active maintainers in the community for swift Kolla images and swift Kolla-Ansible role. If there are any volunteers to pick up that - please reach out either on #openstack-kolla or in the patch [1]. [1]: https://review.opendev.org/c/openstack/kolla-ansible/+/939968 Best regards, Michal

1 0

[watcher] Bug Triage Meeting - January 14, 2025, 12pm UTC
by Ronelle Landy 23 Jan '25

23 Jan '25

Hello All,We will be holding a IRC/Google Meet meeting on Tuesday, January 14, 2025 at 12pm UTC to start the triage process for the list of current Watcher bugs: <https://bugs.launchpad.net/watcher> <https://bugs.launchpad.net/watcher> - <https://bugs.launchpad.net/watcher>https://bugs.launchpad.net/watcher - https://bugs.launchpad.net/watcher-dashboard - https://bugs.launchpad.net/watcher-tempest-plugin Anyone interested in the Watcher project is invited to join the meeting on: - IRC channel #openstack-watcher - Google Meet link: meet.google.com/drk-bgbx-uyz Notes from the meeting will be captured on the IRC channel.Thank you

1 3

[Glance]Cancelling Today's Weekly Meeting
by Mridula Joshi 23 Jan '25

23 Jan '25

Hello, As PTL and I are not around today , hence skipping today's weekly meeting . If someone has anything to discuss, ping Abhishek or Cyril on the IRC channel . Thanks, Mridula Joshi

1 0

[security-sig][ops][neutron] OVN Security issue can impact OpenStack users
by Jay Faulkner 22 Jan '25

22 Jan '25

Hi all, A recent security issue with OVN recently came to light and was fixed in a recent release. Full details are here: https://www.openwall.com/lists/oss-security/2025/01/22/5 . OpenStack deployers running OVN are strongly recommended to upgrade to the new fixed version. No CVE number has been issued yet for this resolved vulnerability. As a note, this was originally reported to the VMT as a neutron bug, but after triage it was responsibly reported to OVN and has been held under embargo. Thank you to the folks who participated in this triage process, including the OVN community for fixing the issue. Please see https://bugs.launchpad.net/ossa/+bug/2088280 for full details on OpenStack handling of this security report. Thanks, Jay Faulkner OpenStack VMT

1 0

[openstack-ansible]: GPU passthrough
by Rambo Rambo 22 Jan '25

22 Jan '25

Hi Experts Release: Wallaby Distribution: Openstack-ansible I am working on enabling GPU computes through PCIpassthrough: I managed to test the GPU PCI passthrough with the direct configuration below. *GPU compute:* /etc/nova/nova.conf [pci] device_spec = { "vendor_id": "10de", "product_id": "25b6" } passthrough_whitelist = { "vendor_id": "10de", "product_id": "25b6" } alias = { "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PCI", "name": "nvidia" } *Nova API container:* /etc/nova/nova.conf enabled_filters = AvailabilityZoneFilter,ComputeFilter,AggregateNumInstancesFilter,AggregateIoOpsFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,NUMATopologyFilter, *PciPassthroughFilter* [pci] alias = { "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PF", "name": "nvidia" } Now I need to put the configuration through openstack-ansible. So I need support for the same. *compute:* I intend to use hostvar for the compute configuration: /etc/openstack_deploy/host_vars/compute-<>.yml Can someone please confirm the parameters I need to set in this file and how to implement it? device_spec = ? passthrough_whitelist = ? nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }' =====> how can I add *device_type field* here as per the mail.yaml it only have name,product_id and vendor_id field? Run openstack-ansible os-nova-install.yml --tags nova-config --limit <compute name> *For nova_api configuration,* I intend to use below: Below parameter in the user_variable file nova_scheduler_default_filters: - AvailabilityZoneFilter - ComputeFilter - AggregateInstanceExtraSpecsFilter - AggregateNumInstancesFilter - AggregateIoOpsFilter - ComputeCapabilitiesFilter - ImagePropertiesFilter - ServerGroupAntiAffinityFilter - ServerGroupAffinityFilter - NUMATopologyFilter - PciPassthroughFilter nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }' =====> how can I add *device_type field here* as per the mail.yaml it only have name,product_id and vendor_id field? Execute: openstack-ansible os-nova-install.yml --tags nova-config --limit <nova api containers> Regards

2 2

[skyline-console] Issue with Skyline Console in Dalmatian Version of OpenStack
by Shubham Kumar Yadav 22 Jan '25

22 Jan '25

Dear Support Team, I hope this message finds you well. I am reaching out to report an issue I am experiencing with the Skyline console in the Dalmatian version of OpenStack. Issue Description: I am unable to list volumes and instances, as well as access volume-related services like Manila. This functionality is critical for my operations. Version Details: I have successfully set up the Skyline console for the Zed version of OpenStack, where everything is functioning as expected. However, the same cannot be said for the Dalmatian version. Troubleshooting Steps Taken: I have checked the service status and confirmed that all necessary services are running. I have attempted to refresh the console and clear the cache, but the issue persists. Request for Assistance: I would appreciate any guidance or troubleshooting steps you could provide to resolve this issue. If there are any logs or additional information you need from my end, please let me know. Also im using same image for both the openstack version - *99cloud/skyline:latest * These are the versions of nova-api & cinder-api *Zed version nova-manage version 26.2.2 cinder-manage version list 21.1.0* *Dalmation versionnova-manage version30.0.1cinder-manage version list* *25.0.0* Thank you for your attention to this matter. I look forward to your prompt response. Best regards, Shubham

1 0

reg trove - issues in deployment
by Rajendar K 22 Jan '25

22 Jan '25

Dear All, I have the following issues while trying trove service. (i) I have installed using kolla ansible and my trove container version is (trove-conductor:2024.1-debian-bookworm) and all my service is running in healthy state. (ii) trove-taskmanger.log as follows 2025-01-22 10:41:54.605 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.11/site-packages/trove/common/utils.py", line 195, in poll_and_check 2025-01-22 10:41:54.605 7 ERROR oslo.service.loopingcall obj = retriever() 2025-01-22 10:41:54.605 7 ERROR oslo.service.loopingcall ^^^^^^^^^^^ 2025-01-22 10:41:54.605 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.11/site-packages/trove/taskmanager/models.py", line 839, in _service_is_active 2025-01-22 10:41:54.605 7 ERROR oslo.service.loopingcall raise TroveError(_("Service not active, status: %s") % status) 2025-01-22 10:41:54.605 7 ERROR oslo.service.loopingcall trove.common.exception.TroveError: Service not active, status: failed to spawn 2025-01-22 10:41:54.605 7 ERROR oslo.service.loopingcall 2025-01-22 10:41:54.624 7 ERROR trove.taskmanager.models [None req-9aede034-a46c-47ec-a149-d7e164fa82e2 cad362f5da0d4f5b8aa87b0650379ed7 4e4b5450f3514da4af3c282d510d9f71 - - - -] Failed to create instance b53d9ebb-2fc6-4927-91c2-f82379b98cfa, error: Service not active, status: failed to spawn.: trove.common.exception.TroveError: Service not active, status: failed to spawn 2025-01-22 10:41:54.636 7 ERROR trove.taskmanager.models [None req-9aede034-a46c-47ec-a149-d7e164fa82e2 cad362f5da0d4f5b8aa87b0650379ed7 4e4b5450f3514da4af3c282d510d9f71 - - - -] Service status: ERROR, service error description: guestagent error: trove.common.exception.TroveError: Service not active, status: failed to spawn 2025-01-22 10:41:54.645 7 ERROR trove.taskmanager.models [None req-9aede034-a46c-47ec-a149-d7e164fa82e2 cad362f5da0d4f5b8aa87b0650379ed7 4e4b5450f3514da4af3c282d510d9f71 - - - -] Trove instance status: ERROR, Trove instance status description: Build error: guestagent timeout.: trove.common.exception.TroveError: Service not active, status: failed to spawn (iii) Does docker required for the configuration of DB or not need in case I use the image trove-master-guest-ubuntu-jammy.qcow2 directly Kindly help me in this regard, thanks Raj

1 0

[magnum] Proposing Jakub Darmach as core reviewer
by Michał Nasiadka 22 Jan '25

22 Jan '25

Hello, I’d like to propose Jakub Darmach (jakub(a)stackhpc.com) as a core reviewer. He has spent quite a good deal of time to get unmaintained branches CI in a green state. He has also contributed some Heat driver patches in the past and is actively reviewing patches (also going the extra mile of updating these patches to be mergable when the author is not available/responsive). Michał Nasiadka mnasiadka(a)gmail.com

3 3

[telemetry] Stepping down from core team
by Yadnesh Kulkarni 22 Jan '25

22 Jan '25

Dear Team, Due to my role change I will no longer be able to contribute to the Telemetry projects. I’ve decided to step down from the core team. Working on the Telemetry projects has been a truly rewarding and enriching learning experience. I’ve thoroughly enjoyed being part of this incredible journey and have gained a lot from collaborating with such talented and supportive individuals. Thank you all for your guidance, collaboration, and support throughout this time. Wishing the team continued success and growth! -- Yadnesh Kulkarni

2 1

[tc][freezer][watcher][release][requirement][oslo] projects need to refresh the DPL model for the next cycle
by Ghanshyam Mann 21 Jan '25

21 Jan '25

Hi All, As you know, we have five projects (listed below) in the DPL model. As per process[1], we need to reset the DPL model every cycle. The project team either needs to opt-in to the DPL model (refer to the document for exact steps[1]) or plan for the PTL election. 1. Freezer: https://review.opendev.org/c/openstack/governance/+/939484 2. Watcher: https://review.opendev.org/c/openstack/governance/+/939488 3. Release: https://review.opendev.org/c/openstack/governance/+/939486 4. Requirement: https://review.opendev.org/c/openstack/governance/+/939487 5. Oslo: https://review.opendev.org/c/openstack/governance/+/939485 [1] https://governance.openstack.org/tc/reference/distributed-project-leadershi… -gmann

1 1

[nova] Seeking Advice on Adding Multiple CPU Models in One Aggregate for OpenStack
by mina.mohammadi367＠gmail.com 21 Jan '25

21 Jan '25

Hello friends, I am currently working on a deployment scenario in OpenStack where I need to group multiple CPU models within a single aggregate. I’m reaching out to the community to ask if anyone has experience with this type of setup or has any advice on best practices. I would greatly appreciate any input from the community regarding potential challenges or recommendations for managing multiple CPU models within a single aggregate. In particular, any advice on ensuring compatibility and optimizing instance scheduling based on traits would be extremely valuable. For reference, I am considering the solution outlined in the CPU Model Selection documentation: https://specs.openstack.org/openstack/nova-specs/specs/train/implemented/cp… Thank you in advance for your time and assistance.

1 0

Assistance with Configuring fdb_age_threshold in OVN with Kolla-Ansible
by Jan Wasilewski 21 Jan '25

21 Jan '25

Hi Stackers, Over the past few days, we encountered an issue during the migration of VMs from a customer environment to our OpenStack cloud. Both environments were connected via a cross-connect network (a VLAN-type network spanning the cloud and the customer environment), which resulted in duplicate MAC addresses in our OVN database. This occurred because the migrated VM retained the same MAC address as previously used in the customer's environment(so "ovn-sbctl list fdb" returned a port which was no-longer used and leads to such duplication) . To address the issue, we used the following command to clear the FDB database: ovn-sbctl --all destroy fdb While researching further, we came across the `fdb_age_threshold` configuration [1], which seemed to be a useful feature to prevent similar issues in the future. We attempted to implement it in our Kolla-Ansible environment by adding the following configuration to ` */etc/kolla/config/neutron/ml2_conf.ini*`: [ovn] fdb_age_threshold = 920 After deploying the changes, however, we noticed that the setting did not take effect in OVN. The configuration only applied successfully after we executed a manual script to propagate the setting directly to OVN [2]. Anyway such change is applicable only for exsisting networks, so it is far from perfect solution. Could you provide guidance on whether this behavior is due to incorrect documentation, an error in our configuration process, or an additional requirement needed to enable this feature? Any insights would be greatly appreciated. Thank you in advance for your support! /Jan Wasilewski **References:** [1] https://docs.openstack.org/neutron/latest/configuration/ml2-conf.html#ovn.f… [2] https://paste.openstack.org/show/b9xUcnG6GXmCdSbSReG0/

2 2

[openstack-ansible] issue
by Danish Khan 21 Jan '25

21 Jan '25

Dear All, Whenever I do any activity like Operating system upgrade of compute nodes or openstack major version update then the name of compute host gets updated. I have my compute node with name compute1,2,3.... but after the activity they sometimes become compute1.openstack.local or compute2.mgmt.local. Due to this I get duplicate resource providers. But only the latest provider works properly. and I had manually unset all the vm from old provider and added then into new provider. Is there a way I can avoid this? means I don't want my compute nodes to get renamed with domain. And is there a way I can unset a vm from all provider and heal them automatically instead adding them into new resource proivder? I tried below docs but the healing part is not working so I go with manual process for all vm which is too time consuming. https://docs.openstack.org/nova/yoga/admin/troubleshooting/orphaned-allocat… Regards, Danish Khan

3 2

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-10)
by Goutham Pacha Ravi 21 Jan '25

21 Jan '25

Hello Stackers, We're 10 weeks away from the 2025.1 coordinated "Epoxy" release [1]. Starting this week, there are several deadlines that project teams have adopted to ensure that code, specifications, and documentation changes are adequately peer-reviewed. As Slawek Kaplonski (slaweq) shared to this mailing list [2], the nomination period for the upcoming Technical Committee and Project Team Leads elections will begin on 2025-02-05 and remain open until 2025-02-19. Early nominations are highly encouraged in case you'll be unavailable during this period. In the last week, the OpenStack Technical Committee selected a community goal to migrate away from the eventlet library across its codebase [3]. As with many community goals, this will be a multi-release goal and needs contributors. Please join the #openstack-eventlet-removal IRC channel on OFTC to participate in the discussion. === Weekly Meeting === Dmitriy Rabotyagov (noonedeadpunk) chaired the TC's weekly IRC meeting on 2025-01-14 [4]. We discussed the maintenance of the "openstackdocstheme" Sphinx plugin and called out some early attempts to switch documentation to a stock theme that's more maintainable and sustainable. We also reviewed a proposal from Freezer project maintainers to remove the project team from the "inactive" teams list [5]. The TC also considered requesting an exception for Freezer to release within 2025.1. In this vein, we also highlighted the need for consistent criteria to move projects out of inactive status and better tooling for indicating project health and activity. The next meeting of the OpenStack Technical Committee is today, 2025-01-21. This meeting will be hosted via IRC on OFTC's #openstack-tc channel. Please find the meeting agenda on the wiki [6]. I hope you'll be able to join us there. === Governance Proposals === ==== Merged ==== - Propose to select the eventlet-removal community goal | https://review.opendev.org/c/openstack/governance/+/934936 - Rework the eventlet-removal goal proposal | https://review.opendev.org/c/openstack/governance/+/931254 - Add ansible-role-httpd repo to OSA-owned projects | https://review.opendev.org/c/openstack/governance/+/935694 ==== Open for Review ==== - Resolve to adhere to non-biased language | https://review.opendev.org/c/openstack/governance/+/934907 - Retire Freezer DR | https://review.opendev.org/c/openstack/governance/+/938183 - Retire qdrouterd role | https://review.opendev.org/c/openstack/governance/+/938193 - Remove Freezer from inactive state | https://review.opendev.org/c/openstack/governance/+/938938 - Reset the DPL model for oslo project | https://review.opendev.org/c/openstack/governance/+/939485 - Reset the DPL model for Release project | https://review.opendev.org/c/openstack/governance/+/939486 - Reset the DPL model for Requirement project | https://review.opendev.org/c/openstack/governance/+/939487 - Reset the DPL model for Watcher project | https://review.opendev.org/c/openstack/governance/+/939488 - Define 2025 upstream investment opportunities | https://review.opendev.org/c/openstack/governance/+/939507 === Upcoming Events === - 2025-01-28: OpenInfra Board Monthly Meeting: https://board.openinfra.org/ - 2025-02-01: FOSDEM 2025 (https://fosdem.org/2025/) OpenStack's 15th Birthday Celebration - 2025-02-05: Nominations open for 2025.2 TC/PTL Elections: https://governance.openstack.org/election/ - 2025-02-28: 2025.1 ("Epoxy") Feature Freeze and release milestone 3 [1] - 2025-03-06: SCALE 2025 + OpenInfra Days NA (https://www.socallinuxexpo.org/scale/22x) Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] TC/PTL Election dates for the 2025.2 cycle: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] Goal to Remove Eventlet from OpenStack: https://governance.openstack.org/tc/goals/selected/remove-eventlet.html [4] TC Meeting IRC Log 2025-01-14: https://meetings.opendev.org/meetings/tc/2025/tc.2025-01-14-18.00.log.html [5] Remove Freezer from inactive state: https://review.opendev.org/c/openstack/governance/+/938938 [6] TC Meeting Agenda, 2025-01-21: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[Horizon] Need Help - Error with Compress
by info＠patrick-omland.de 20 Jan '25

20 Jan '25

I need help with Horizon; when I run the following command ./manage.py compress, I get the following error: (venv) root@fsn1-controller01 /var/lib/horizon # ./manage.py compress /var/lib/horizon/venv/lib/python3.11/site-packages/debreach/__init__.py:6: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. version_info = version.StrictVersion(__version__).version CommandError: An error occurred during rendering _stylesheets.html: Error: File to import not found or unreadable: /horizon/lib/font_awesome/scss/font-awesome.scss. on line 12:1 of dashboard/scss/horizon.scss from line 6:9 of stdin >> @import "/horizon/lib/font_awesome/scss/font-awesome.scss"; ^ (venv) root@fsn1-controller01 /var/lib/horizon # ll total 124 -rw-r--r-- 1 root root 77 Jan 20 19:13 babel-django.cfg -rw-r--r-- 1 root root 258 Jan 20 19:13 babel-djangojs.cfg -rw-r--r-- 1 root root 233 Jan 20 19:13 bindep.txt -rw-r--r-- 1 root root 600 Jan 20 19:13 CONTRIBUTING.rst drwxr-xr-x 3 root root 4096 Jan 20 19:13 doc -rw-r--r-- 1 root root 579 Jan 20 19:13 HACKING.rst drwxr-xr-x 19 root root 4096 Jan 20 19:15 horizon -rw-r--r-- 1 root root 10143 Jan 20 19:13 LICENSE -rwxr-xr-x 1 root root 831 Jan 20 19:13 manage.py -rw-r--r-- 1 root root 634 Jan 20 19:13 MANIFEST.in drwxr-xr-x 6 root root 4096 Jan 20 19:15 openstack_auth drwxr-xr-x 18 root root 4096 Jan 20 19:15 openstack_dashboard -rw-r--r-- 1 root root 1023 Jan 20 19:13 package.json drwxr-xr-x 8 root root 4096 Jan 20 19:13 playbooks -rw-r--r-- 1 root root 2188 Jan 20 19:13 plugin-registry.csv -rw-r--r-- 1 root root 1920 Jan 20 19:13 README.rst drwxr-xr-x 4 root root 4096 Jan 20 19:13 releasenotes -rw-r--r-- 1 root root 120 Jan 20 19:13 reno.yaml -rw-r--r-- 1 root root 2664 Jan 20 19:13 requirements.txt drwxr-xr-x 5 root root 4096 Jan 20 19:13 roles -rw-r--r-- 1 root root 1937 Jan 20 19:13 setup.cfg -rw-r--r-- 1 root root 698 Jan 20 19:13 setup.py drwxr-xr-x 9 root root 4096 Jan 20 19:15 static -rw-r--r-- 1 root root 379 Jan 20 19:13 test-requirements.txt -rw-r--r-- 1 root root 3198 Jan 20 19:13 test-shim.js drwxr-xr-x 2 root root 4096 Jan 20 19:13 tools -rw-r--r-- 1 root root 7537 Jan 20 19:13 tox.ini drwxr-xr-x 6 root root 4096 Jan 20 19:13 venv (venv) root@fsn1-controller01 /var/lib/horizon # I have installed Horizon in a virtual Python 3 environment (Debian Bookworm) and I also clone the Horizon Github Repo, then I installed all requirements using pip install -r requirements.txt. Anyone have an idea?

1 0

[Horizon] Need Help - Error with Compress
by info＠patrick-omland.de 20 Jan '25

20 Jan '25

I need help with Horizon; when I run the following command ./manage.py compress, I get the following error: (venv) root@fsn1-controller01 /var/lib/horizon # ./manage.py compress /var/lib/horizon/venv/lib/python3.11/site-packages/debreach/__init__.py:6: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. version_info = version.StrictVersion(__version__).version CommandError: An error occurred during rendering _stylesheets.html: Error: File to import not found or unreadable: /horizon/lib/font_awesome/scss/font-awesome.scss. on line 12:1 of dashboard/scss/horizon.scss from line 6:9 of stdin >> @import "/horizon/lib/font_awesome/scss/font-awesome.scss"; ^ (venv) root@fsn1-controller01 /var/lib/horizon # ll total 124 -rw-r--r-- 1 root root 77 Jan 20 19:13 babel-django.cfg -rw-r--r-- 1 root root 258 Jan 20 19:13 babel-djangojs.cfg -rw-r--r-- 1 root root 233 Jan 20 19:13 bindep.txt -rw-r--r-- 1 root root 600 Jan 20 19:13 CONTRIBUTING.rst drwxr-xr-x 3 root root 4096 Jan 20 19:13 doc -rw-r--r-- 1 root root 579 Jan 20 19:13 HACKING.rst drwxr-xr-x 19 root root 4096 Jan 20 19:15 horizon -rw-r--r-- 1 root root 10143 Jan 20 19:13 LICENSE -rwxr-xr-x 1 root root 831 Jan 20 19:13 manage.py -rw-r--r-- 1 root root 634 Jan 20 19:13 MANIFEST.in drwxr-xr-x 6 root root 4096 Jan 20 19:15 openstack_auth drwxr-xr-x 18 root root 4096 Jan 20 19:15 openstack_dashboard -rw-r--r-- 1 root root 1023 Jan 20 19:13 package.json drwxr-xr-x 8 root root 4096 Jan 20 19:13 playbooks -rw-r--r-- 1 root root 2188 Jan 20 19:13 plugin-registry.csv -rw-r--r-- 1 root root 1920 Jan 20 19:13 README.rst drwxr-xr-x 4 root root 4096 Jan 20 19:13 releasenotes -rw-r--r-- 1 root root 120 Jan 20 19:13 reno.yaml -rw-r--r-- 1 root root 2664 Jan 20 19:13 requirements.txt drwxr-xr-x 5 root root 4096 Jan 20 19:13 roles -rw-r--r-- 1 root root 1937 Jan 20 19:13 setup.cfg -rw-r--r-- 1 root root 698 Jan 20 19:13 setup.py drwxr-xr-x 9 root root 4096 Jan 20 19:15 static -rw-r--r-- 1 root root 379 Jan 20 19:13 test-requirements.txt -rw-r--r-- 1 root root 3198 Jan 20 19:13 test-shim.js drwxr-xr-x 2 root root 4096 Jan 20 19:13 tools -rw-r--r-- 1 root root 7537 Jan 20 19:13 tox.ini drwxr-xr-x 6 root root 4096 Jan 20 19:13 venv (venv) root@fsn1-controller01 /var/lib/horizon # I have installed Horizon in a virtual Python 3 environment (Debian Bookworm) and I also clone the Horizon Github Repo, then I installed all requirements using pip install -r requirements.txt. Anyone have an idea?

1 0

How to know the openstack client's default microversion?
by berndbausch＠gmail.com 20 Jan '25

20 Jan '25

Without the version option e.g. --os-compute-api-version, the openstack client submits API requests with certain default microversions. The precise microversions vary with the openstack client's version and the specific task. For example, the --debug option reveals that openstack client 7.1.3 uses Nova microversion 2.89 when attaching a volume to an instance. Is there a simpler way than --debug to find out which microversions the client uses? Is there a command, or perhaps a website where this is documented? Thanks for your ideas! Bernd.

2 1

[openstack-ansible] : libvirtd debug logs after Wallaby upgrade
by Rambo Rambo 20 Jan '25

20 Jan '25

Hi Team I have recently done Victoria to Wallaby upgrade on my openstack-ansible installation. After the upgrade we observe that /var/log/libvirt/ size on all the computes was growing due to libvirtd.log file. Further checking this file it can be seen that there are debug logs. 2025-01-19 11:15:20.817+0000: 2943565: *debug *: virNetlinkEventCallback:862 : dispatching to max 0 clients, called from event watch 7 2025-01-19 11:15:20.817+0000: 2943565:* debug* : virNetlinkEventCallback:876 : event not handled. 2025-01-19 11:15:20.821+0000: 2943605: *debug :* udevHandleOneDevice:1492 : udev action: 'add' ... libvirtd configuration file on compute shows log_level=1 cat libvirtd.conf # Ansible managed # enabling debug mode according to http://wiki.libvirt.org/page/DebugLogs log_level = 1 log_filters="3:remote 4:event 3:json 3:rpc" log_outputs="1:file:/var/log/libvirt/libvirtd.log" listen_tls = 0 listen_tcp = 0 unix_sock_group = "libvirt" unix_sock_ro_perms = "0777" unix_sock_rw_perms = "0770" auth_unix_ro = "none" auth_unix_rw = "none" auth_tcp = "sasl" No related parameter was changed/updated in user_variable file during the upgrade? Any information what could have enabled the debug logs for libvirtd on all the computes? Also how can I turn-off this using the openstack-ansible and what could be impact during the procedure? Regards

2 3

Openstack Ansible
by engineer2024 20 Jan '25

20 Jan '25

It seems that the ssh to containers from the deploy node, is removed in the caracal release by default in OSA. The new containers don't have the ssh service inside them. Is there a way to enable it via a variable ?

2 2

[neutron] Bug deputy week 3 2025 (2025-01-13 - 2025-01-19)
by Lajos Katona 20 Jan '25

20 Jan '25

Hi Neutrinos, The bug list for week 3 of 2025. * Critical ** https://bugs.launchpad.net/neutron/+bug/2094970 (rally ovn job timing out with eventlet 0.38.2) *** https://review.opendev.org/c/openstack/neutron/+/939315 Merged * High ** https://bugs.launchpad.net/neutron/+bug/2094736 ([OVN] Missing ``LogicalSwitchPortUpdateUpEvent`` when a port is bound) *** WIP: https://review.opendev.org/c/openstack/neutron/+/939345 ** https://bugs.launchpad.net/neutron/+bug/2094810 (neutron-status upgrade check fails on checking if ovn_alembic_version table exists) *** https://review.opendev.org/c/openstack/neutron/+/939248 ** https://bugs.launchpad.net/neutron/+bug/2094842 ("ovs.db.error.Error: ovsdb error: 0 values when type requires between 1 and 9223372036854775807" when router is attached to network without subnet ) *** Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095000 (ovs-agent: Failed to store metadata for trunk ...: Cannot find Bridge with name=tbr-... ) Assigned ** https://bugs.launchpad.net/neutron/+bug/2095044 (test_ip_lib calls to sudo and fails (sometimes?)) *** https://review.opendev.org/c/openstack/neutron/+/939435 Merged ** https://bugs.launchpad.net/neutron/+bug/2095167 (BW limit not applied to external/provider network) *** https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/939539 ** https://bugs.launchpad.net/neutron/+bug/2095180 (test_agent_metadata_port_dhcp_reenable_event fails when ovsdb-server combines create and later update into a single create event for monitor) *** https://review.opendev.org/c/openstack/neutron/+/939542 ** https://bugs.launchpad.net/neutron/+bug/2095195 ( VM fails to receive IPv4 from DHCP agent because the agent has not updated the network config for dnsmasq after port was created) ** https://bugs.launchpad.net/neutron/+bug/2095196 (functional: test_virtual_port_host_update failed due to ovsdb SB transaction timeout, "Unreasonably long 25512ms poll interval (0ms user, 26ms system)" in ovsdb-server log) *** https://review.opendev.org/c/openstack/neutron/+/939558 ** https://bugs.launchpad.net/neutron/+bug/2095224 (Functional jobs randomly timed out, possible load issues on the host ) *** perhaps duplicate of bug/2095196 ( https://bugs.launchpad.net/neutron/+bug/209519) * Medium ** https://bugs.launchpad.net/neutron/+bug/2094846 (neutron.tests.functional.services.trunk.drivers.ovn.test_trunk_driver.TestOVNTrunkDriver.test_subport_delete fails because remove_subports fails to update OVN db due to revision number inconsistency) *** Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095084 ([OVN] Error in ``_delete_router_remove_external_gateways``: "'NoneType' object has no attribute 'network_id'") *** https://review.opendev.org/c/openstack/neutron/+/939451 ** https://bugs.launchpad.net/neutron/+bug/2095097 (500 error creating resource with emoji for name) Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095152 (ovs-agent: Leftover tpi/spi interfaces after VM boot/delete with trunk port(s)) Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095185 test_legacy_router_conntrack_helper failed as "neutron_lib.exceptions.BridgeDoesNotExist: Bridge test-bre0e2da16 does not exist" when router attempted to plug interface *** Unassigned, Was seen on stable, but can happen on master * Low ** https://bugs.launchpad.net/neutron/+bug/2095091 (test_metadata_ipv6_only_network failed looking for QUERY_MSG) Unassigned Lajos Katona (lajoskatona)

1 0

[glance][ops] proposal to remove the glance-scrubber
by Brian Rosmaita 20 Jan '25

20 Jan '25

Hello Glance users, The Glance project team wants to determine whether anyone is using the glance-scrubber utility [0]. If not, we are thinking of deprecating it this cycle to be removed in the appropriate future cycle. If you have not previously heard of the glance-scrubber, that's most likely because it's only useful for particular deployment configurations of which yours is not one. See the doc [0] for details if you are curious. Our suspicion is that the glance-scrubber may have outlived its usefulness for modern OpenStack configurations. (The last time I remember it being discussed at a PTG was during the snowstorm in Dublin, which I believe was the Rocky development cycle). On the other hand, the team also has some ideas for improving the scrubber, but there's no point doing that if it's not actually being used in the first place. So if removal of the glance-scrubber would negatively affect you, please respond to this email so we can get some data points. Thanks! [0] https://docs.openstack.org/glance/latest/cli/glancescrubber.html

2 2

[magnum] GPU support with magnum
by Satish Patel 19 Jan '25

19 Jan '25

Folks, We are running magnum with openstack to spin up k8s clusters and everything works as expected. Now I want GPU support so we can do some AI/ML stuff with k8s. Does magnum support GPU and how does it work? In the past I used GPU in passthrough mode to expose hardware to VMs but not sure how it works for k8s and what are the challenges?

5 16

[neutron][upgrades] Linuxbridge driver removal and migration
by Brian Haley 17 Jan '25

17 Jan '25

Hi Stackers, In 2022, during the Zed cycle, the Neutron team took a step at marking some items as experimental. One of those was the Linuxbridge driver. The reason was due to it being unmaintained for a number of cycles, and no one was able to take on its ownership. The implication of this decision was that it was consider deprecated, even if not given an official warning message. In doing this, all gate jobs were moved into the experimental queue, feature-parity with other drivers was deemed unnecessary, and bug fixing was considered best-effort. Since then, both major distros have stopped supporting new deployments using Linuxbridge, and have been mainly focused on OVN. So what does this mean? We (the neutron cores) have come to the conclusion that it is time to take the next step and remove the code from the tree [1]. After 5 cycles of being considered experimental without any investment it just makes sense. We plan on doing this in the current Epoxy cycle. We realize that there are still deployments using Linuxbridge, and hopefully they have been looking at migration strategies since we originally marked the code experimental. This email is another push to the user community to start looking at this, especially if you are planning any new deployments. There has been at least one migration guide written at [2] (thanks Jim!), and at this point I would encourage those still using Linuxbridge to look at it and start asking any questions they have so we have the chance to make it better and can make that process easier. Thanks for reading, -Brian, Neutron PTL [0] https://review.opendev.org/c/openstack/neutron/+/845181 [1] https://review.opendev.org/c/openstack/neutron/+/927216 [2] https://www.jimmdenton.com/migrating-lxb-to-ovn/

11 14

openstack ssl
by engineer2024 17 Jan '25

17 Jan '25

The present model for openstack API services uses keystone for authentication mechanism for the REST API. is it possible to test the api calls directly using the server/port of a API service, bypassing the keystone URI ? does this require any modification in the configuration of the openstack service ? Thanks

1 0

[neutron] Using address scopes in a DVR environment breaks floating IPs
by Lukas Steiner 17 Jan '25

17 Jan '25

Hi all, I hope to get some advice here if we’re doing something wrong or ran into a bug. We have an OpenStack environment (wallaby) with Distributed Virtual Router (DVR) enabled (compute nodes use DVR_NO_EXTERNAL agent mode, network nodes use DVR_SNAT) and we would like to onboard our provider network to a subnet pool part of an address scope to allow routed self-service networks. But as soon as the provider network is part of an address scope, the floating IP feature stops working. After debugging we found that adding the network to an address scope results in an additional set of firewall rules in the router and snat network namespaces which blocks cross-scope traffic. But floating IPs should be the exception to that and there are indeed rules that explicitly allow them by setting a mark in the iptables mangle table: -A neutron-l3-agent-floatingip -d <floating-ip>/32 -i qg-<random-id> -j MARK --set-xmark 0x4000000/0xffff0000 We can find these rules in the qrouter namespaces, but they’re missing in the snat namespaces. If we add the rule manually to the snat namespace, the floating IP is working again. If we switch the compute nodes to DVR agent mode, it also works, as the snat namespace is not used. I have already searched for bugs and changes in the DVR router code since wallaby but can’t find anything which is not already patched in wallaby. So, I’m wondering if we have an unsupported setup with combining dynamic routing with DVR and centralized network nodes or facing a long running neutron bug. Maybe someone has a similar setup or can confirm that this use case should be supported? Regards, Lukas

1 0

Re: Dalmatian Devstack: Empty Port Binding section in Horizon
by Lajos Katona 17 Jan '25

17 Jan '25

Hi, I checked and I hoped that this SDK patch : https://review.opendev.org/c/openstack/openstacksdk/+/915902 or SDK 4.2.0 is enough (for binding details at least) but something else is missing. Could you please open a bug report for this issue, I try to check where to fix it. Lajos Katona (lajoskatona) <berndbausch(a)gmail.com> ezt írta (időpont: 2025. jan. 14., K, 13:25): > Thanks much, Lajos! > > > > It was installed by Devstack with git_clone https://opendev.org/openstack/horizon.git > /opt/stack/horizon stable/2024.2. Furthermore, under /opt/stack/horizon/ > horizon.egg-info I find a file named pbr.json, which contains > > > > {"git_version": "4bcc3dcea", "is_release": false} > > > > I hope that makes it clear what I am using! Or is there another way of > obtaining the Horizon release number? > > > > Of the list of patches you shared, the ones that seem relevant (SDK for > neutron ports, SDK for neutron networks) were merged in August and June, > respectively, which makes me think that should be included in my release. > > > > And since this is an HTML message now, I am adding two screenshots. Not > only port binding info, but also network provider info is missing. I can > confirm that this is the admin user. > > > > Kind regards, > > > > Bernd > > > > > > *From:* Lajos Katona <katonalala(a)gmail.com> > *Sent:* Tuesday, January 14, 2025 5:22 PM > *To:* berndbausch(a)gmail.com > *Cc:* openstack-discuss <openstack-discuss(a)lists.openstack.org> > *Subject:* Re: Dalmatian Devstack: Empty Port Binding section in Horizon > > > > Hi Bernd, > > Shall I ask which version of Horizon do you use? Recently some patches > were merged for changing to SDK in the background: > > > https://review.opendev.org/q/project:openstack/horizon+topic:%22bug/1999774… > > And to be sure if you hit an issue from that change series or something > else. > > > > Best wishes > > Lajos Katona (lajoskatona) > > > > <berndbausch(a)gmail.com> ezt írta (időpont: 2025. jan. 14., K, 4:43): > > Is there a known bug, either in Devstack or Horizon, that prevents Neutron > port binding details from being displayed? My googling did not find > anything. > > As admin, I open a port's details, and in the Binding section at the end of > the page, all I get is a bold text "VNIC Type". The CLI command "openstack > port show" does provide the details. Also on a Xena Devstack, Horizon > displays them[1]. Horizon's setup.py seems to be identical on both > releases, > as far as Neutron configuration is concerned. > > Thanks for any input. > > Bernd. > > [1] The Binding details I expect: > > Binding > > VNIC Type Normal > Host xena-202 > Profile None > VIF Type ovs > VIF Details > connectivity l2 > port_filter True > ovs_hybrid_plug False > datapath_type system > bridge_name br-int > >

2 1

[qa] QA office hours new time
by Ghanshyam Mann 16 Jan '25

16 Jan '25

Hi All, As informed by Martin[1], we are changing the QA office hours to Wed biweekly at 18.00 UTC. I have updated the meeting info[2] and wiki page[3]. This time might not suit many of you, so feel free to ping me on IRC if any topic/review needs urgent attention. [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [2] https://review.opendev.org/c/opendev/irc-meetings/+/939480 [3] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee -gmann

1 0

Adding a compute node using Openstack ansible
by Derek O keeffe 16 Jan '25

16 Jan '25

Hi all, I have something weird happening when trying to add a compute node to a working cluster. I added "compute3" to my openstack_user_config.yml compute_hosts stanza and ran the setup_hosts playbook. This created (re-created) the openstack_inventory.json file which I have reviewed thoroughly and I can see compute3 added everywhere compute1 and compute2 had been added from the deploy. When I run the second playbook setup_infrastructure, all the plays skip saying "no hosts matched" but the tasks all touch compute3.... I ran /opt/openstack-ansible/scripts/inventory-manage.py -G and I can see my compute3 listed in the output. The commands I run for the playbooks are as follows: openstack-ansible setup-hosts.yml --limit ,compute3openstack-ansible setup-infrastructure.yml --limit ,compute3openstack-ansible setup-openstack.yml --limit ,compute3 I've been trying to solve this since before Christmas with no luck, could anyone please shed some light on what the issue could be? Output of first playbook: https://paste.openstack.org/show/blArG04q6wgUCJy9JPh3/Output of second playbook: Paste #bSiabwqOk7qdns2iBYDA | LodgeIt! | | | | Paste #bSiabwqOk7qdns2iBYDA | LodgeIt! | | | Any advice or info would be greatly appreciated. Regards,Derek

2 3

[all] 2024 OpenInfra Foundation annual report
by Jeremy Stanley 15 Jan '25

15 Jan '25

The OpenInfra Foundation staff have published 2024's annual report: https://openinfra.org/annual-report/2024 As usual, it includes a section on OpenStack, covering some highlights of our community's achievements throughout the year. There's an accompanying post on the OpenStack Blog too: https://www.openstack.org/blog/openstack-2024-community-progress-highlights/ Please feel free to share these anywhere you think people may be interested in seeing our year in review, or that of our sibling communities and other activities supported by the foundation. -- Jeremy Stanley

1 0

[kolla-ansible][keystone]problem using LDAP and TLS
by Franck VEDEL 15 Jan '25

15 Jan '25

Hello, with all previous versions, to authenticate users from an Active Directory domain, using an LDAP connector worked without any problem on my labs, with or without using TLS. With TLS the file used is the following: /etc/kolla/config/keystone/domains/keystone.DOM1.conf [identity] driver = ldap [ldap] url = ldap://172.16.60.41 user = CN=ldapconnect,OU=openstack,DC=dom1,DC=loc password = XXXXXXXXXX suffix = DC=dom1,DC=loc user_tree_dn = OU=openstack,DC=dom1,DC=loc user_objectclass = person user_filter = (|(memberOf=CN=openstack-users,OU=openstack,DC=dom1,DC=loc)(memberOf=CN=openstack-admins,OU=openstack,DC=dom1,DC=loc)) user_id_attribute = sAMAccountName user_name_attribute = sAMAccountName user_mail_attribute = mail user_pass_attribute = user_enabled_attribute = userAccountControl user_enabled_mask = 2 user_enabled_default = 512 user_attribute_ignore = password,tenant_id,tenants group_objectclass = group group_tree_dn = OU=openstack,DC=dom1,DC=loc group_filter = (|(CN=openstack-users)(CN=openstack-admins)) group_id_attribute = cn group_name_attribute = name query_scope = sub chase_referrals = false use_tls = True tls_certificate = /etc/kolla/certificates/adserv.crt tls_req_cert = never I want to see if it works with Dalmatian (to update my lab, which I can't do with the Bobcat and Caracal versions because bugs block me). Without TLS (use_tls = False) , no problem, it works, my AD users can use Horizon and have access to their project. If I put "use_tls = True" and tls_certificate = /etc/kolla/certificates/adserv.crt .... It doesn't work with Dalmatian, certificate problem. Indeed, the certificate is not copied into the keystone container, so it can't use it. In the keystone logs (/var/kolla/keystone/keystone.log) there is an error message: 2025-01-15 08:36:04.477 1084 ERROR keystone.server.flask.request_processing.middleware.auth_context OSError: tls_cacertfile /etc/kolla/certificates/adserv.crt not found or is not a file However, my adserv.crt file has not changed, is valid, and is in /etc/kolla/certificates directory. What could have changed with the Dalmatian version? I tried to change parameters, but I can't find the solution. How should I proceed to make it work with TLS. I specify again that it works without TLS so the problem is really to use this certificate correctly, so to put it in the right place with the right parameter: tls_certificate or a new one? My lab use ubuntu 2204. Thanks a lot for your help Franck VEDEL

1 0

Facing issue while creating image in Open stack using Barbican
by Shalini Srivastava 15 Jan '25

15 Jan '25

Hi Sir, I have created openstack image with the help of Barbican key-manager but it is not coming in active mode from past two days. It is still showing in queue mode only. Since i have searched a lot reagarding the same issue but could not find any solution to fix the same. Hope to hear from you soon. Thanks & Regards Shalini Srivastava Project Associate /प्रोजेक्ट एसोसिएट Centre for Development of Advanced Computing(C-DAC) / प्रगत संगणन विकास केन्द्र(सी-डैक) Tidel Park”, 8th Floor, “D” Block, (North &South) / “टाइडल पार्क”,8वीं मंजिल, “डी” ब्लॉक, (उत्तर और दक्षिण) No.4, Rajiv Gandhi Salai / नं.4, राजीव गांधी सलाई Taramani / तारामणि Chennai / चेन्नई – 600113 Ph.No.:044-22542226/27 Fax No.: 044-22542294 ------------------------------------------------------------------------------------------------------------ [ C-DAC is on Social-Media too. Kindly follow us at: Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ] This e-mail is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email is strictly prohibited and appropriate legal action will be taken. ------------------------------------------------------------------------------------------------------------

1 0

[dev] Invitation to participate in a survey on flaky tests spanning multiple OpenStack projects
by xiao＠ait.kyushu-u.ac.jp 15 Jan '25

15 Jan '25

Hello there, My name is Tao Xiao, Assistant Professor at Kyushu University (Japan), and togather with a team from Canada and China. Our research specifically investigates cross-project flakiness by analyzing information available on the Gerrit Code Review platform. Through our study, we have observed a significant number of flaky tests spanning multiple OpenStack projects and categorized several potential causes behind this phenomenon. Our next step is to validate our empirical findings and gather real developers' perceptions to improve the OpenStack ecosystem through a developer survey. If you have at most 10 minutes, we have designed a simple survey to capture your perceptions. The survey link is here: https://forms.gle/dUMWRL8MNALQE6kG8 This questionnaire is totally anonymized, and the survey was approved by the Kyushu University, Japan ethics board (2024-26). Again, thank you so much for your time, and please feel free to contact us for more information. Tao Xiao on Behalf of Our Team Assistant Professor Faculty of Information Science and Electrical Engineering Kyushu University, Japan xiao(a)ait.kyushu-u.ac.jp <mailto:xiao@ait.kyushu-u.ac.jp> Team Members and their homepages: Tao Xiao – https://tao-xiao.github.io/ Dong Wang – https://dong-w.github.io/ Shane McIntosh – https://shanemcintosh.org/ Hideaki Hata – https://hideakihata.github.io/ Yasutaka Kamei – https://posl.ait.kyushu-u.ac.jp/~kamei/

1 0

[openstack][octavia] How clients can debug their LB
by Nguyễn Hữu Khôi 15 Jan '25

15 Jan '25

Hello. I would like to know how clients can debug their LB such as access log or monitor LB performance? Thank you. Regards Nguyen Huu Khoi

5 7

[ironic] Ironic-Lib being deprecated
by Jay Faulkner 15 Jan '25

15 Jan '25

Hi all, As many of you know, at the recent PTG the Ironic community made a decision to move any functionality out of ironic-lib into ironic / ironic-python-agent. This library has been documented, since it's inception, as for use by Ironic projects only, so there should be little to no impact to deployers other than having one less python package to deploy in their IPA ramdisks and Ironic services. We will keep the repository's stable (and unmaintained) branches open for as long as their Ironic/IPA counterparts are also supported in order to fulfill our support obligation. Remaining patches to complete this deprecation are here: https://review.opendev.org/q/topic:%22ironic-lib-deprecation%22 -- please reach out if there are any questions or concerns. I thought it wise to inform the mailing list of this change. We'll ensure it's highlighted in Ironic cycle highlights for epoxy as well. Thanks, Jay Faulkner

1 0

[release] [upgrades] Seeking OpenStack Upgrade Success Stories
by Allison Price 15 Jan '25

15 Jan '25

Hi everyone, Ahead of the OpenStack 2025.1 ‘Epoxy’ release, the OpenInfra Foundation marketing team wants to work with the OpenStack community to highlight how upgrades have become simplified with the implementation of SLURP. This will include leveraging the governance docs to draft articles around the upstream progress as well as operator success stories for organizations who have upgraded their own deployments or customer deployments to a SLURP release. The goal is to further increase awareness on the improvements made to the upgrade process and reduce common perceptions that OpenStack upgrades remain complex. Here [1] is an etherpad where we are collecting names of volunteers who can either review materials promoting the upstream progress or share operator success stories. If you have any questions, please let me know. [1] https://etherpad.opendev.org/p/OpenStackUpgrades Cheers, Allison Allison Price VP of Marketing & Community p: +1-214-686-6821

1 0

[all] Foundation Individual Director election voting underway
by Jeremy Stanley 14 Jan '25

14 Jan '25

Voting is open now for the 2025 OpenInfra Foundation Individual Director Election. You must have joined the Open Infrastructure Foundation as an Individual Member by Sunday, July 14, 2024 to vote in this election, per the bylaws: https://openinfra.org/legal/bylaws Individual Directors serve as board representatives for the Individual Members who elect them, so are the most effective way we as participants in OpenStack can influence board decisions on the general direction of our supporting foundation. Voting in these elections, once you're eligible to do so, is also necessary in order to maintain your individual membership. To vote, check your email (or your spam folder) for your personal link to vote. It will have been sent to the primary address in your foundation member profile. The subject line is: "OpenInfra Foundation 2025 Individual Director Election" You can review the candidates here: https://openinfra.dev/election/2025-individual-director-election/candidates Voting will close this Friday, January 17th, 2025 at 1900 UTC. Please contact secretary(a)openinfra.dev with any questions you may have about the election. -- Jeremy Stanley

1 0

[kolla] [wallaby] [openvswitch]
by Albert Braden 14 Jan '25

14 Jan '25

One of our busiest clusters was rebuilt on Wallaby a few months ago (kolla-ansible) and deletions are taking a long time for VMs on some hypervisors. When I look at the ovs-vswitchd.log on the slow hypervisors I see lots of errors that apparently refer to a missing VM interface: 2024-12-27T19:06:10.482Z|01793|bridge|WARN|could not open network device qvo0430a2f7-cd (No such device) For most network devices I see "added" and "deleted" lines in the log. This one has an "added" line but no "deleted". I tried restarting the neutron_openvswitch_agent and openvswitch_vswitchd containers but that didn't make a difference. How can I get OVS to stop choking on this missing interface?

2 17

[ironic][networking] Baremetal Networking Working Group resuming 2/15
by Julia Kreger 14 Jan '25

14 Jan '25

Greetings everyone, Now that we are in the new year and a number of ironic contributors are back from PTO, we're going to resume the networking meetings. One aspect which we discussed in December was to focus the discussion. To that end, after some follow-up discussion, we're going to try and focus the meeting a bit more using the agenda. We have an agenda posted to the etherpad[0], along with a meeting link. If you have additional topics, please feel free to add to that agenda, but also be mindful between the use cases of standalone usage and openstack integrated usage and explicitly call out the use case. As a reminder, the meeting will be at 1500 UTC. Meeting link in the etherpad! Thanks! -Julia [0]: https://etherpad.opendev.org/p/ironic-networking

1 0

[publiccloud-sig] Reminder: first meeting of the new year - January 15th at 0800 UTC
by Felix Kronlage-Dammers 14 Jan '25

14 Jan '25

Hi everyone, This upcoming Wednesday (January 15th) the next meeting of the publiccloud-sig is going to happen. We’ll be meeting at 0800 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG felix -- Felix Kronlage-Dammers Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[all][tc] Election dates for the 2025.2 cycle
by Sławek Kapłoński 14 Jan '25

14 Jan '25

Hi, To align with the proposal from Goutham [1] we have updated election calendar for the upcoming elections [2] to make voting period slightly longer thus nomination will also start earlier. Please be aware of it. New calendar looks like: Release is at: 2025-04-02 Latest possible completion is at: 2025-03-19 Moving back to Wednesday: 2025-03-19 PTL Election from 2025-02-26T23:45 to 2025-03-19T23:45 TC Election from 2025-02-26T23:45 to 2025-03-19T23:45 TC Campaigning from 2025-02-19T23:45 to 2025-02-26T23:45 PTL Nominations from 2025-02-05T23:45 to 2025-02-19T23:45 TC Nominations from 2025-02-05T23:45 to 2025-02-19T23:45 Set email_deadline to 2025-02-19T00:00 Setting TC timeframe end to email_deadline Begining of 2024.2 Cycle @ 2024-03-15 00:00:00+00:00 End of 2025.1 cycle @ 2025-02-19 00:00:00+00:00 Election timeframe: 341 days, 0:00:00s --- [1] https://review.opendev.org/c/openstack/governance/+/937741 [2] https://review.opendev.org/c/openstack/election/+/938782/ -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Dalmatian Devstack: Empty Port Binding section in Horizon
by berndbausch＠gmail.com 14 Jan '25

14 Jan '25

Is there a known bug, either in Devstack or Horizon, that prevents Neutron port binding details from being displayed? My googling did not find anything. As admin, I open a port's details, and in the Binding section at the end of the page, all I get is a bold text "VNIC Type". The CLI command "openstack port show" does provide the details. Also on a Xena Devstack, Horizon displays them[1]. Horizon's setup.py seems to be identical on both releases, as far as Neutron configuration is concerned. Thanks for any input. Bernd. [1] The Binding details I expect: Binding VNIC Type Normal Host xena-202 Profile None VIF Type ovs VIF Details connectivity l2 port_filter True ovs_hybrid_plug False datapath_type system bridge_name br-int

2 1

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-11)
by Goutham Pacha Ravi 14 Jan '25

14 Jan '25

Hello Stackers, Time flies! We're 11 weeks away from the 2025.1 "Epoxy" release day [1]. OpenStack project teams must be working on their deliverables according to the schedule shared by Előd Illés (elodilles) from the release team [2] this week. In the past week, the OpenStack Technical Committee (TC) worked with election officials to adjust the dates of the elections preceding the 2025.2 ("Flamingo" [3]) release cycle. This change was made in response to a revision in the TC's charter that now allows more time for polling. Election officials will share the updated schedule on this list soon. Additionally, the TC approved the retirement of openstack-ansible roles for former projects: Murano (Application Catalog), Senlin (Clustering Service), and Sahara (Data Processing). The OpenInfra Board's Individual Member Director elections are currently underway [4]. If you are an OpenInfra Foundation member, please check your email for a ballot and participate. The election will conclude on Friday, 2025-01-17. === Weekly Meeting === The OpenStack TC resumed its regular weekly meeting schedule last week following a brief hiatus due to the year-end holidays. Last week's meeting was held on 2025-01-07 at 1800 UTC, simultaneously on Zoom and IRC. Please find the meeting minutes on eavesdrop [5] and a recording on YouTube [6]. The meeting began with a discussion about the pending proposal [7] to delete the "unmaintained/victoria" branch on OpenStack git repositories. There is interest from at least one contributing organization in keeping this branch open for specific repositories. The deadline to merge this change is 2025-01-31. We aim to identify the specific repositories and the responsible maintainers by then. I'll keep you updated in future emails. The TC then discussed initiating the election cycle for the 2025.2 release. We recognize that long holidays (such as the upcoming Chinese New Year) could impact nominations from interested candidates. We hope to encourage nominations as early as possible since even a couple of extra weeks could make a difference in coordinating for such a geographically diverse community. The TC also reviewed the status of the community goal to migrate test jobs to Python 3.12 (and "Ubuntu 24.04 / Noble Numbat" where applicable). Ghanshyam Mann (gmann), the goal champion, shared that test jobs in three projects (Heat, Skyline, and devstack-plugin-container) need attention from their respective project teams [8]. We also expressed our gratitude to the OpenDev Infrastructure team for keeping the systems running smoothly during the holidays. The next OpenStack Technical Committee meeting is today, 2025-01-14, at 1800 UTC. This meeting will be held over IRC in OFTC's #openstack-tc channel. Please find the agenda on the meeting wiki [9]. I hope you can join us. Remember, any community member can propose meeting topics—just mention your IRC nick so the meeting chair can call upon you. === Governance Proposals === ==== Merged ==== - Allow more than 2 weeks for elections | https://review.opendev.org/c/openstack/governance/+/937741 - Put whitebox-tempest-plugin under release management | https://review.opendev.org/c/openstack/governance/+/938401 - Retire Murano/Senlin/Sahara OpenStack-Ansible roles | https://review.opendev.org/c/openstack/governance/+/935677 ==== Open for Review ==== - Rework the eventlet-removal goal proposal | https://review.opendev.org/c/openstack/governance/+/931254 - Add ansible-role-httpd repo to OSA-owned projects | https://review.opendev.org/c/openstack/governance/+/935694 - Retire Freezer DR | https://review.opendev.org/c/openstack/governance/+/938183 - Retire qdrouterd role | https://review.opendev.org/c/openstack/governance/+/938193 - Remove Freezer from inactive state | https://review.opendev.org/c/openstack/governance/+/938938 - Propose to select the eventlet-removal community goal | https://review.opendev.org/c/openstack/governance/+/934936 - Resolve to adhere to non-biased language | https://review.opendev.org/c/openstack/governance/+/934907 === How to Contact the TC === You can reach the TC in several ways: - Email: Send an email with the tag [tc] on this mailing list. - Ping us using the 'tc-members' keyword on the #openstack-tc IRC channel on OFTC. - Join us at our weekly meeting: The Technical Committee meets every week on Tuesdays at 1800 UTC [9]. === Upcoming Events === - 2025-01-17: 2025 OpenInfra Board Individual Member Director Elections conclude - 2025-02-01: FOSDEM 2025 (https://fosdem.org/2025/) OpenStack's 15th Birthday Celebration - 2025-02-28: 2025.1 ("Epoxy") Feature Freeze and release milestone 3 [1] - 2025-03-06: SCALE 2025 + OpenInfra Days NA (https://www.socallinuxexpo.org/scale/22x) Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] Release countdown for week R-11: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] OpenStack 2025.2 'F' Release Naming Poll: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [4] OpenInfra Foundation Board Elections: https://openinfra.dev/election/2025-individual-director-election [5] TC Meeting IRC Log 2025-01-07: https://meetings.opendev.org/meetings/tc/2025/tc.2025-01-07-18.00.log.html [6] TC Meeting Video Recording, 2025-01-07: https://youtu.be/-Nxul8_ykto [7] Transition unmaintained/victoria to EOL: https://review.opendev.org/c/openstack/releases/+/937515 [8] Projects failing the "migrate-to-noble" goal: https://etherpad.opendev.org/p/migrate-to-noble#L172 [9] TC Meeting Agenda, 2025-01-14: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[openstack][octavia] Override default 50000 OctaviaTimeoutMemberData
by Andy Speagle 13 Jan '25

13 Jan '25

Hi Folks, We're using charmed openstack and charmed k8s on top of that. In our k8s environment, we have an nginx ingress controller that uses the openstack-integrator charm to create and manage octavia LBs. I'm trying to figure out how to manage the haproxy frontend and backend timeout parameters that seem to default to 50000ms via: OctaviaTimeoutClientData OctaviaTimeoutMemberData Ideally, being able to specify this in the k8s service manifest that gets passed down into the octavia config would be "the way." Any ideas? Thanks! -- Andy Speagle

1 1

Assigning specific IP address to a host
by William Muriithi 13 Jan '25

13 Jan '25

Hello I have a system that I would like to assign a specific IP address. I am aware of a way I can set it up on creation but want to change it post creation so that I can prepare the host before swapping the IP address to bring it to production. This is what I have tried:- Set up a second interface: openstack server add fixed ip --fixed-ip-address 192.168.20.61 f3a20bef-bb03-4e9a-8996-77567ef96ff0 d1051b62-46a7-487e-b188-f742aec6d711 Remove the default security group: openstack port unset --security-group c0108624-dbd8-42bb-acaa-bbbfd32fbbae 6978e153-47dc-42d4-bdb8-63b325c77cf0 Assign the relevant security group: openstack port set --security-group 415a6ef0-e461-41fc-85f0-c92c043e5b50 6978e153-47dc-42d4-bdb8-63b325c77cf0 After that, when I ssh to the virtual machine, I can see the new interface and indeed also ping/ssh to the system using that IP address. However, its not reachable from outside the virtual machine What could I be missing here? Regards, William

2 1

[Neutron] Bug report Jan 6 to Jan 12
by Rodolfo Alonso Hernandez 13 Jan '25

13 Jan '25

Hello Neutrinos: This is the bug report from the past week. *HIGH* * DVR flow is not installed for allowed address pairs mac in DVR_TO_SRC_MAC table=1 (https://bugs.launchpad.net/neutron/+bug/2093248) Patch proposed: https://review.opendev.org/c/openstack/neutron/+/938657 * [OVN] Port unbinding operation receives rogue port UP event too ( https://bugs.launchpad.net/neutron/+bug/2093327) *Not assigned*. * [ovn-octavia-provider] first request is stuck on OVN txn ( https://bugs.launchpad.net/neutron/+bug/2093347) Patch proposed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/938797 *MEDIUM* * OVS Manager creation is failing at ovs-agent startup causing that values like inactivity_timeout are not set ( https://bugs.launchpad.net/neutron/+bug/2093433) Assigned. *LOW* * Missing Documentation for vpnaas with ovn ( https://bugs.launchpad.net/neutron/+bug/2093317) *Not assigned*. Regards.

1 0

[all][gate] Bandit version 1.8.1 issues
by Brian Haley 13 Jan '25

13 Jan '25

Hi Stackers, I was re-checking some simple Neutron changes and noticed our pep8 job started failing running bandit. Seems there was a new version (1.8.1) just released and it's generating false-positives [0]. I filed a bug against it at [1], and for now I'll just work around it by ignoring B106 warnings in Neutron with [2]. Didn't know if we wanted to pin/denylist it while the issue is addressed as I'm assuming others projects will start seeing the same failure. -Brian [0] https://bugs.launchpad.net/neutron/+bug/2093849 [1] https://github.com/PyCQA/bandit/issues/1216 [2] https://review.opendev.org/c/openstack/neutron/+/939004

1 1

keystone concepts
by Budai Laszlo 12 Jan '25

12 Jan '25

Dear all, reading the keystone documentation (https://docs.openstack.org/keystone/latest/getting-started/architecture.html) we can see the following sentence: "|Projects| represent the base unit of |ownership| in OpenStack, in that all resources in OpenStack should be owned by a specific project." What are "resources" in the previous statement? Thank you in advance, Laszlo

2 3

keystone concepts
by Budai Laszlo 12 Jan '25

12 Jan '25

Dear all, reading the keystone documentation (https://docs.openstack.org/keystone/latest/getting-started/architecture.html) we can see the following sentence: "|Projects| represent the base unit of |ownership| in OpenStack, in that all resources in OpenStack should be owned by a specific project." What are "resources" in the previous statement? Thank you in advance, Laszlo

1 0

[release] OpenStack 2025.2 'F' Release Naming Poll
by Allison Price 10 Jan '25

10 Jan '25

Hi everyone, Please join me in voting to name OpenStack 2025.2, starting with the letter ‘F’! https://civs1.civs.us/cgi-bin/vote.pl?id=E_7207f29f7e2e222d&akey=f80b0a4bbd… The deadline to vote is Friday, January 10 at 1700 UTC. Cheers, Allison

1 1

Step 2 of HA Wallaby overcloud fails with "ERROR: Can't run container mysql_wait_bundle"
by davorin.mikulic＠gmail.com 10 Jan '25

10 Jan '25

Dear Openstack community, I am trying to deploy Wallaby overcloud on HP Gen9 machines with CentOS 9 installed on them. The very similar setup with CentOS 8 deployed and worked successfully. Deploy fails on step 2 with: 2025-01-07 12:07:31.485297 | | WARNING | ERROR: Can't run container mysql_wait_bundle stderr: + STEP=2 + TAGS=file,file_line,concat,augeas,galera_ready,mysql_database,mysql_grant,mysql_user + CONFIG='include tripleo::profile::pacemaker::database::mysql_bundle' + EXTRA_ARGS= + '[' -d /tmp/puppet-etc ']' + cp -a /tmp/puppet-etc/devices /tmp/puppet-etc/hieradata /tmp/puppet-etc/hiera.yaml /tmp/puppet-etc/modules /tmp/puppet-etc/puppet.conf /tmp/puppet-etc/ssl /etc/puppet + echo '{"step": 2}' + export FACTER_deployment_type=containers + FACTER_deployment_type=containers + set +e + puppet apply --verbose --detailed-exitcodes --summarize --color=false --modulepath /etc/puppet/modules:/opt/stack/puppet-modules:/usr/share/openstack-puppet/modules --tags file,file_line,concat,augeas,galera_ready,mysql_database,mysql_grant,mysql_user -e 'noop_resource('\''package'\''); include tripleo::profile::pacemaker::database::mysql_bundle' Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5 (file: /etc/puppet/hiera.yaml) Warning: Undefined variable '::deploy_config_name'; (file & line not available) Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/7.6/deprecated_language.html (file & line not available) Warning: Unknown variable: '::pacemaker::pcs_010'. (file: /etc/puppet/modules/pacemaker/manifests/resource/bundle.pp, line: 159, column: 6) Error: '/usr/bin/clustercheck >/dev/null' returned 1 instead of one of [0] Error: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/Exec[galera-ready]/returns: change from 'notrun' to ['0'] failed: '/usr/bin/clustercheck >/dev/null' returned 1 instead of one of [0] Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111) Warning: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/Mysql_user[root@localhost]: Skipping because of failed dependencies Warning: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/File[/root/.my.cnf]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[root@127.0.0.1]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[root@::1]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[@localhost]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[@%]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[root@localhost.localdomain]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[@localhost.localdomain]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[root@overcloud-controller-0.tb27.cld]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[@overcloud-controller-0.tb27.cld]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[root@overcloud-controller-0]: Skipping because of failed dependencies Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_user[@overcloud-controller-0]: Skipping because of failed dependencies Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111) Warning: /Stage[main]/Mysql::Server::Account_security/Mysql_database[test]: Skipping because of failed dependencies Warning: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/Mysql_user[root@%]: Skipping because of failed dependencies Warning: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/Mysql_user[clustercheck@localhost]: Skipping because of failed dependencies Warning: /Stage[main]/Cinder::Db::Mysql/Openstacklib::Db::Mysql[cinder]/Mysql_database[cinder]: Skipping because of failed dependencies Warning: /Stage[main]/Glance::Db::Mysql/Openstacklib::Db::Mysql[glance]/Mysql_database[glance]: Skipping because of failed dependencies Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Mysql_database[heat]: Skipping because of failed dependencies Warning: /Stage[main]/Keystone::Db::Mysql/Openstacklib::Db::Mysql[keystone]/Mysql_database[keystone]: Skipping because of failed dependencies Warning: /Stage[main]/Neutron::Db::Mysql/Openstacklib::Db::Mysql[neutron]/Mysql_database[ovs_neutron]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Mysql_database[nova]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova_cell0]/Mysql_database[nova_cell0]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Mysql_database[nova_api]: Skipping because of failed dependencies Warning: /Stage[main]/Placement::Db::Mysql/Openstacklib::Db::Mysql[placement]/Mysql_database[placement]: Skipping because of failed dependencies Warning: /Stage[main]/Cinder::Db::Mysql/Openstacklib::Db::Mysql[cinder]/Openstacklib::Db::Mysql::Host_access[cinder_%]/Mysql_user[cinder@%]: Skipping because of failed dependencies Error: Could not prefetch mysql_grant provider 'mysql': Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111) Warning: /Stage[main]/Cinder::Db::Mysql/Openstacklib::Db::Mysql[cinder]/Openstacklib::Db::Mysql::Host_access[cinder_%]/Mysql_grant[cinder@%/cinder.*]: Skipping because of failed dependencies Warning: /Stage[main]/Glance::Db::Mysql/Openstacklib::Db::Mysql[glance]/Openstacklib::Db::Mysql::Host_access[glance_%]/Mysql_user[glance@%]: Skipping because of failed dependencies Warning: /Stage[main]/Glance::Db::Mysql/Openstacklib::Db::Mysql[glance]/Openstacklib::Db::Mysql::Host_access[glance_%]/Mysql_grant[glance@%/glance.*]: Skipping because of failed dependencies Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_%]/Mysql_user[heat@%]: Skipping because of failed dependencies Warning: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Openstacklib::Db::Mysql::Host_access[heat_%]/Mysql_grant[heat@%/heat.*]: Skipping because of failed dependencies Warning: /Stage[main]/Keystone::Db::Mysql/Openstacklib::Db::Mysql[keystone]/Openstacklib::Db::Mysql::Host_access[keystone_%]/Mysql_user[keystone@%]: Skipping because of failed dependencies Warning: /Stage[main]/Keystone::Db::Mysql/Openstacklib::Db::Mysql[keystone]/Openstacklib::Db::Mysql::Host_access[keystone_%]/Mysql_grant[keystone@%/keystone.*]: Skipping because of failed dependencies Warning: /Stage[main]/Neutron::Db::Mysql/Openstacklib::Db::Mysql[neutron]/Openstacklib::Db::Mysql::Host_access[ovs_neutron_%]/Mysql_user[neutron@%]: Skipping because of failed dependencies Warning: /Stage[main]/Neutron::Db::Mysql/Openstacklib::Db::Mysql[neutron]/Openstacklib::Db::Mysql::Host_access[ovs_neutron_%]/Mysql_grant[neutron@%/ovs_neutron.*]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Openstacklib::Db::Mysql::Host_access[nova_%]/Mysql_user[nova@%]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova]/Openstacklib::Db::Mysql::Host_access[nova_%]/Mysql_grant[nova@%/nova.*]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql/Openstacklib::Db::Mysql[nova_cell0]/Openstacklib::Db::Mysql::Host_access[nova_cell0_%]/Mysql_grant[nova@%/nova_cell0.*]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Openstacklib::Db::Mysql::Host_access[nova_api_%]/Mysql_user[nova_api@%]: Skipping because of failed dependencies Warning: /Stage[main]/Nova::Db::Mysql_api/Openstacklib::Db::Mysql[nova_api]/Openstacklib::Db::Mysql::Host_access[nova_api_%]/Mysql_grant[nova_api@%/nova_api.*]: Skipping because of failed dependencies Warning: /Stage[main]/Placement::Db::Mysql/Openstacklib::Db::Mysql[placement]/Openstacklib::Db::Mysql::Host_access[placement_%]/Mysql_user[placement@%]: Skipping because of failed dependencies Warning: /Stage[main]/Placement::Db::Mysql/Openstacklib::Db::Mysql[placement]/Openstacklib::Db::Mysql::Host_access[placement_%]/Mysql_grant[placement@%/placement.*]: Skipping because of failed dependencies + rc=6 + set -e + set +ux 2025-01-07 12:07:31.488412 | | WARNING | ERROR: Can't run container rabbitmq_wait_bundle stderr: + STEP=2 + TAGS=file,file_line,concat,augeas,rabbitmq_policy,rabbitmq_user,rabbitmq_ready + CONFIG='include tripleo::profile::pacemaker::rabbitmq_bundle' + EXTRA_ARGS= + '[' -d /tmp/puppet-etc ']' + cp -a /tmp/puppet-etc/devices /tmp/puppet-etc/hieradata /tmp/puppet-etc/hiera.yaml /tmp/puppet-etc/modules /tmp/puppet-etc/puppet.conf /tmp/puppet-etc/ssl /etc/puppet + echo '{"step": 2}' + export FACTER_deployment_type=containers + FACTER_deployment_type=containers + set +e + puppet apply --verbose --detailed-exitcodes --summarize --color=false --modulepath /etc/puppet/modules:/opt/stack/puppet-modules:/usr/share/openstack-puppet/modules --tags file,file_line,concat,augeas,rabbitmq_policy,rabbitmq_user,rabbitmq_ready -e 'noop_resource('\''package'\''); include tripleo::profile::pacemaker::rabbitmq_bundle' Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5 (file: /etc/puppet/hiera.yaml) Warning: Undefined variable '::deploy_config_name'; (file & line not available) Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/7.6/deprecated_language.html (file & line not available) Warning: Unknown variable: '::pacemaker::pcs_010'. (file: /etc/puppet/modules/pacemaker/manifests/resource/bundle.pp, line: 159, column: 6) Error: 'rabbitmqctl eval "rabbit_nodes:is_running(node(), rabbit)." | grep -q true && rabbitmqctl eval "rabbit_mnesia:is_clustered()." | grep -q true' returned 1 instead of one of [0] Error: /Stage[main]/Tripleo::Profile::Pacemaker::Rabbitmq_bundle/Exec[rabbitmq-ready]/returns: change from 'notrun' to ['0'] failed: 'rabbitmqctl eval "rabbit_nodes:is_running(node(), rabbit)." | grep -q true && rabbitmqctl eval "rabbit_mnesia:is_clustered()." | grep -q true' returned 1 instead of one of [0] Warning: /Stage[main]/Tripleo::Profile::Base::Rabbitmq/Rabbitmq_user[guest]: Skipping because of failed dependencies Warning: /Stage[main]/Tripleo::Profile::Pacemaker::Rabbitmq_bundle/Rabbitmq_policy[ha-all@/]: Skipping because of failed dependencies + rc=6 + set -e + set +ux 2025-01-07 12:07:31.489548 | 000c29c7-8523-9a37-c0c4-00000000f883 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_2 | overcloud-controller-0 | error={"changed": false, "msg": "Failed containers: mysql_wait_bundle, rabbitmq_wait_bundle"} 2025-01-07 12:07:31.490512 | 000c29c7-8523-9a37-c0c4-00000000f883 | TIMING | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_2 | overcloud-controller-0 | 0:48:23.163306 | 1989.39s PLAY RECAP ********************************************************************* localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 overcloud-cephstorage-0 : ok=273 changed=70 unreachable=0 failed=0 skipped=151 rescued=0 ignored=1 overcloud-cephstorage-1 : ok=275 changed=70 unreachable=0 failed=0 skipped=146 rescued=0 ignored=1 overcloud-cephstorage-2 : ok=275 changed=70 unreachable=0 failed=0 skipped=146 rescued=0 ignored=1 overcloud-computesriov-0 : ok=342 changed=92 unreachable=0 failed=0 skipped=163 rescued=0 ignored=1 overcloud-computesriov-1 : ok=349 changed=92 unreachable=0 failed=0 skipped=156 rescued=0 ignored=1 overcloud-controller-0 : ok=372 changed=99 unreachable=0 failed=1 skipped=217 rescued=0 ignored=1 overcloud-controller-1 : ok=392 changed=98 unreachable=0 failed=0 skipped=212 rescued=0 ignored=1 overcloud-controller-2 : ok=392 changed=98 unreachable=0 failed=0 skipped=212 rescued=0 ignored=1 undercloud : ok=76 changed=19 unreachable=0 failed=0 skipped=31 rescued=0 ignored=1 2025-01-07 12:07:31.561753 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2025-01-07 12:07:31.562447 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 1905 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2025-01-07 12:07:31.563026 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:48:23.235822 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2025-01-07 12:07:31.563565 | UUID | Info | Host | Task Name | Run Time 2025-01-07 12:07:31.564127 | 000c29c7-8523-9a37-c0c4-00000000f883 | SUMMARY | overcloud-controller-0 | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_2 | 1989.39s 2025-01-07 12:07:31.564700 | 000c29c7-8523-9a37-c0c4-00000000bb0c | SUMMARY | undercloud | tripleo_container_image_prepare : Container image prepare | 137.67s 2025-01-07 12:07:31.565400 | 000c29c7-8523-9a37-c0c4-00000000e34e | SUMMARY | undercloud | tripleo_run_cephadm : run cephadm playbook | 59.69s 2025-01-07 12:07:31.565937 | 000c29c7-8523-9a37-c0c4-00000000e715 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for haproxy | 56.48s 2025-01-07 12:07:31.566525 | 000c29c7-8523-9a37-c0c4-00000000e747 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run pacemaker restart if the config file for the service changed | 46.63s 2025-01-07 12:07:31.567080 | 000c29c7-8523-9a37-c0c4-000000004353 | SUMMARY | overcloud-controller-1 | tripleo_firewall : Manage firewall rules | 36.87s 2025-01-07 12:07:31.567628 | 000c29c7-8523-9a37-c0c4-000000004353 | SUMMARY | overcloud-controller-0 | tripleo_firewall : Manage firewall rules | 36.75s 2025-01-07 12:07:31.568206 | 000c29c7-8523-9a37-c0c4-000000004353 | SUMMARY | overcloud-controller-2 | tripleo_firewall : Manage firewall rules | 36.65s 2025-01-07 12:07:31.568767 | 000c29c7-8523-9a37-c0c4-00000000e776 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run pacemaker restart if the config file for the service changed | 36.45s 2025-01-07 12:07:31.569335 | 000c29c7-8523-9a37-c0c4-00000000e745 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for mysql | 24.87s 2025-01-07 12:07:31.569886 | 000c29c7-8523-9a37-c0c4-00000000e774 | SUMMARY | overcloud-controller-0 | tripleo_ha_wrapper : Run init bundle puppet on the host for oslo_messaging_rpc | 23.94s 2025-01-07 12:07:31.570454 | 000c29c7-8523-9a37-c0c4-00000000caa9 | SUMMARY | overcloud-controller-0 | Wait for puppet host configuration to finish | 20.83s 2025-01-07 12:07:31.571025 | 000c29c7-8523-9a37-c0c4-00000000f709 | SUMMARY | overcloud-controller-0 | Wait for puppet host configuration to finish | 20.82s 2025-01-07 12:07:31.571574 | 000c29c7-8523-9a37-c0c4-00000000be26 | SUMMARY | overcloud-controller-2 | Pre-fetch all the containers | 15.90s 2025-01-07 12:07:31.572173 | 000c29c7-8523-9a37-c0c4-00000000bdad | SUMMARY | overcloud-controller-1 | Pre-fetch all the containers | 15.88s 2025-01-07 12:07:31.572740 | 000c29c7-8523-9a37-c0c4-00000000bdad | SUMMARY | overcloud-controller-0 | Pre-fetch all the containers | 15.68s 2025-01-07 12:07:31.573331 | 000c29c7-8523-9a37-c0c4-0000000024d7 | SUMMARY | overcloud-computesriov-1 | tripleo_hieradata : Render hieradata from template | 11.32s 2025-01-07 12:07:31.573885 | 000c29c7-8523-9a37-c0c4-0000000024d7 | SUMMARY | overcloud-computesriov-0 | tripleo_hieradata : Render hieradata from template | 11.14s 2025-01-07 12:07:31.574467 | 000c29c7-8523-9a37-c0c4-00000000c4a0 | SUMMARY | overcloud-computesriov-1 | Wait for puppet host configuration to finish | 11.07s 2025-01-07 12:07:31.575030 | 000c29c7-8523-9a37-c0c4-00000000c452 | SUMMARY | overcloud-computesriov-0 | Wait for puppet host configuration to finish | 10.98s 2025-01-07 12:07:31.575572 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2025-01-07 12:07:31.576197 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2025-01-07 12:07:31.576756 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~ 2025-01-07 12:07:31.577328 | The following node(s) had failures: overcloud-controller-0 2025-01-07 12:07:31.577879 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2025-01-07 12:07:32.979 180292 INFO tripleoclient.utils.utils [-] Temporary directory [ /tmp/tripleogr2nbip5 ] cleaned up 2025-01-07 12:07:32.979 180292 ERROR tripleoclient.utils.utils [-] Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2025-01-07 12:07:32.981 180292 WARNING tripleoclient.utils.safe_write [-] The output file /home/stack/overcloud-deploy/overcloud/overcloud-deployment_status.yaml will be overriden: RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh Host 10.100.27.127 not found in /home/stack/.ssh/known_hosts Overcloud Endpoint: http://10.100.27.127:5000 Overcloud Horizon Dashboard URL: http://10.100.27.127:80/dashboard Overcloud rc file: /home/stack/overcloud-deploy/overcloud/overcloudrc and /home/stack/overcloudrc Overcloud Deployed with error 2025-01-07 12:07:33.690 180292 INFO tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Stopping ephemeral heat. 2025-01-07 12:07:33.774 180292 INFO tripleoclient.heat_launcher [-] Pod does not exist: ephemeral-heat 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last): 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/command.py", line 32, in run 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, self).run(parsed_args) 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/osc_lib/command/command.py", line 39, in run 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, self).run(parsed_args) 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/cliff/command.py", line 185, in run 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = self.take_action(parsed_args) or 0 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1331, in take_action 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud deployment.set_deployment_status( 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud self.force_reraise() 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise self.value 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1303, in take_action 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud deployment.config_download( 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/workflows/deployment.py", line 386, in config_download 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud utils.run_ansible_playbook( 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/utils.py", line 775, in run_ansible_playbook 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise RuntimeError(err_msg) 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2025-01-07 12:07:36.431 180292 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud 2025-01-07 12:07:36.434 180292 ERROR openstack [-] Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh: RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2025-01-07 12:07:36.437 180292 INFO osc_lib.shell [-] END return value: 1 I can see the following logs on the failed controller-0 node: [root@overcloud-controller-0 log]# less /var/log/containers/rabbitmq/log/crash.log ** Connection attempt from disallowed node 'rabbitmqcli-170662-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:55 =ERROR REPORT==== ** Connection attempt from disallowed node 'rabbitmqcli-170889-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:55 =ERROR REPORT==== ** Connection attempt from disallowed node 'rabbitmqcli-170889-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:56 =ERROR REPORT==== ** Connection attempt from disallowed node 'rabbitmqcli-171103-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:56 =ERROR REPORT==== ** Connection attempt from disallowed node 'rabbitmqcli-171103-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:29:03 =ERROR REPORT==== ** Generic server rabbit_disk_monitor terminating ** Last message in was update ** When Server state == {state,"/var/lib/rabbitmq/mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld",50000000,1173484339200,100,10000,#Ref<0.112052652.1425801217.106754>,false,true,10,120000} ** Reason for termination == ** {{unparseable,"df: /var/lib/rabbitmq/mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"},[{rabbit_disk_monitor,parse_free_unix,1,[{file,"src/rabbit_disk_monitor.erl"},{line,272}]},{rabbit_disk_monitor,internal_update,1,[{file,"src/rabbit_disk_monitor.erl"},{line,218}]},{rabbit_disk_monitor,handle_info,2,[{file,"src/rabbit_disk_monitor.erl"},{line,190}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]} 2025-01-07 10:29:03 =CRASH REPORT==== crasher: initial call: rabbit_disk_monitor:init/1 pid: <0.309.0> registered_name: rabbit_disk_monitor exception exit: {{unparseable,"df: /var/lib/rabbitmq/mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"},[{rabbit_disk_monitor,parse_free_unix,1,[{file,"src/rabbit_disk_monitor.erl"},{line,272}]},{rabbit_disk_monitor,internal_update,1,[{file,"src/rabbit_disk_monitor.erl"},{line,218}]},{rabbit_disk_monitor,handle_info,2,[{file,"src/rabbit_disk_monitor.erl"},{line,190}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]} ancestors: [rabbit_disk_monitor_sup,rabbit_sup,<0.301.0>] message_queue_len: 0 messages: [] links: [<0.308.0>] dictionary: [] trap_exit: false status: running heap_size: 2586 stack_size: 27 reductions: 314108420 neighbours: 2025-01-07 10:29:03 =SUPERVISOR REPORT==== Supervisor: {local,rabbit_disk_monitor_sup} Context: child_terminated Reason: {unparseable,"df: /var/lib/rabbitmq/mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"} Offender: [{pid,<0.309.0>},{name,rabbit_disk_monitor},{mfargs,{rabbit_disk_monitor,start_link,[50000000]}},{restart_type,{transient,1}},{shutdown,30000},{child_type,worker}] less /var/log/containers/rabbitmq/rabbit(a)overcloud-controller-0.internalapi.tb27.cld.log 2024-12-19 19:18:44.490 [info] <0.9.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-01-07 10:28:44.393 [error] <0.11277.105> ** Connection attempt from disallowed node 'rabbitmqcli-169019-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:44.404 [error] <0.11280.105> ** Connection attempt from disallowed node 'rabbitmqcli-169019-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:45.167 [error] <0.11284.105> ** Connection attempt from disallowed node 'rabbitmqcli-169251-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:45.179 [error] <0.11286.105> ** Connection attempt from disallowed node 'rabbitmqcli-169251-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:46.282 [error] <0.11282.105> ** Connection attempt from disallowed node 'rabbitmqcli-169491-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:46.293 [error] <0.11293.105> ** Connection attempt from disallowed node 'rabbitmqcli-169491-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:46.779 [error] <0.11298.105> ** Connection attempt from disallowed node 'rabbitmqcli-169710-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:46.788 [error] <0.11300.105> ** Connection attempt from disallowed node 'rabbitmqcli-169710-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:52.948 [error] <0.11303.105> ** Connection attempt from disallowed node 'rabbitmqcli-169935-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:52.958 [error] <0.11312.105> ** Connection attempt from disallowed node 'rabbitmqcli-169935-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:54.824 [error] <0.11317.105> ** Connection attempt from disallowed node 'rabbitmqcli-170662-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:54.837 [error] <0.11318.105> ** Connection attempt from disallowed node 'rabbitmqcli-170662-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:55.645 [error] <0.11325.105> ** Connection attempt from disallowed node 'rabbitmqcli-170889-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:55.655 [error] <0.11327.105> ** Connection attempt from disallowed node 'rabbitmqcli-170889-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:56.160 [error] <0.11328.105> ** Connection attempt from disallowed node 'rabbitmqcli-171103-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:28:56.171 [error] <0.11334.105> ** Connection attempt from disallowed node 'rabbitmqcli-171103-rabbit(a)overcloud-controller-0.internalapi.tb27.cld' ** 2025-01-07 10:29:03.051 [error] <0.309.0> ** Generic server rabbit_disk_monitor terminating ** Last message in was update ** When Server state == {state,"/var/lib/rabbitmq/mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld",50000000,1173484339200,100,10000,#Ref<0.112052652.1425801217.106754>,f alse,true,10,120000} ** Reason for termination == ** {{unparseable,"df: /var/lib/rabbitmq/mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"},[{rabbit_disk_monitor,parse_free_unix,1,[{file,"sr c/rabbit_disk_monitor.erl"},{line,272}]},{rabbit_disk_monitor,internal_update,1,[{file,"src/rabbit_disk_monitor.erl"},{line,218}]},{rabbit_disk_monitor,handle_info,2,[{file,"src/r abbit_disk_monitor.erl"},{line,190}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,ini t_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]} 2025-01-07 10:29:03.051 [error] <0.309.0> CRASH REPORT Process rabbit_disk_monitor with 0 neighbours exited with reason: {unparseable,"df: /var/lib/rabbitmq/mnesia/rabbit@overclou d-controller-0.internalapi.tb27.cld: No such file or directory\n"} in rabbit_disk_monitor:parse_free_unix/1 line 272 2025-01-07 10:29:03.051 [error] <0.308.0> Supervisor rabbit_disk_monitor_sup had child rabbit_disk_monitor started with rabbit_disk_monitor:start_link(50000000) at <0.309.0> exit with reason {unparseable,"df: /var/lib/rabbitmq/mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"} in context child_terminated 2025-01-07 10:29:03.059 [info] <0.11345.105> Free disk space monitor encountered an error (e.g. failed to parse output from OS tools): {{'EXIT',{unparseable,"df: /var/lib/rabbitmq /mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"}},202138791936}, retries left: 10 2025-01-07 10:31:03.068 [info] <0.11345.105> Free disk space monitor encountered an error (e.g. failed to parse output from OS tools): {{'EXIT',{unparseable,"df: /var/lib/rabbitmq /mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"}},202138791936}, retries left: 9 2025-01-07 10:33:03.076 [info] <0.11345.105> Free disk space monitor encountered an error (e.g. failed to parse output from OS tools): {{'EXIT',{unparseable,"df: /var/lib/rabbitmq /mnesia/rabbit(a)overcloud-controller-0.internalapi.tb27.cld: No such file or directory\n"}},202138791936}, retries left: 8 [root@overcloud-controller-0 ~]# pcs status --full Cluster name: tripleo_cluster Cluster Summary: * Stack: corosync (Pacemaker is running) * Current DC: overcloud-controller-1 (2) (version 2.1.9-1.el9-49aab9983) - partition with quorum * Last updated: Tue Jan 7 11:49:48 2025 on overcloud-controller-0 * Last change: Tue Jan 7 10:33:11 2025 by root via root on overcloud-controller-0 * 9 nodes configured * 26 resource instances configured (1 BLOCKED from further action due to failure) Node List: * GuestNode galera-bundle-0@overcloud-controller-2: online * GuestNode galera-bundle-1@overcloud-controller-0: online * GuestNode galera-bundle-2@overcloud-controller-1: online * Node overcloud-controller-0 (1): online, feature set 3.19.6 * Node overcloud-controller-1 (2): online, feature set 3.19.6 * Node overcloud-controller-2 (3): online, feature set 3.19.6 * GuestNode rabbitmq-bundle-0@overcloud-controller-2: online * GuestNode rabbitmq-bundle-1@overcloud-controller-0: online * GuestNode rabbitmq-bundle-2@overcloud-controller-1: online Full List of Resources: * ip-10.100.21.127 (ocf:heartbeat:IPaddr2): Started overcloud-controller-0 * ip-10.100.27.127 (ocf:heartbeat:IPaddr2): Started overcloud-controller-1 * ip-10.100.22.127 (ocf:heartbeat:IPaddr2): Started overcloud-controller-2 * ip-10.100.24.127 (ocf:heartbeat:IPaddr2): Started overcloud-controller-0 * ip-10.100.25.127 (ocf:heartbeat:IPaddr2): Started overcloud-controller-1 * Container bundle set: haproxy-bundle [cluster.common.tag/haproxy:pcmklatest]: * Replica[0] * haproxy-bundle-podman-0 (ocf:heartbeat:podman): Started overcloud-controller-0 * Replica[1] * haproxy-bundle-podman-1 (ocf:heartbeat:podman): Started overcloud-controller-1 * Replica[2] * haproxy-bundle-podman-2 (ocf:heartbeat:podman): Started overcloud-controller-2 * Container bundle set: galera-bundle [cluster.common.tag/mariadb:pcmklatest]: * Replica[0] * galera (ocf:heartbeat:galera): FAILED Promoted galera-bundle-0 (blocked) * galera-bundle-podman-0 (ocf:heartbeat:podman): Started overcloud-controller-2 * galera-bundle-0 (ocf:pacemaker:remote): Started overcloud-controller-2 * Replica[1] * galera (ocf:heartbeat:galera): Unpromoted galera-bundle-1 * galera-bundle-podman-1 (ocf:heartbeat:podman): Started overcloud-controller-0 * galera-bundle-1 (ocf:pacemaker:remote): Started overcloud-controller-0 * Replica[2] * galera (ocf:heartbeat:galera): Unpromoted galera-bundle-2 * galera-bundle-podman-2 (ocf:heartbeat:podman): Started overcloud-controller-1 * galera-bundle-2 (ocf:pacemaker:remote): Started overcloud-controller-1 * Container bundle set: rabbitmq-bundle [cluster.common.tag/rabbitmq:pcmklatest]: * Replica[0] * rabbitmq (ocf:heartbeat:rabbitmq-cluster): Started rabbitmq-bundle-0 * rabbitmq-bundle-podman-0 (ocf:heartbeat:podman): Started overcloud-controller-2 * rabbitmq-bundle-0 (ocf:pacemaker:remote): Started overcloud-controller-2 * Replica[1] * rabbitmq (ocf:heartbeat:rabbitmq-cluster): Started rabbitmq-bundle-1 * rabbitmq-bundle-podman-1 (ocf:heartbeat:podman): Started overcloud-controller-0 * rabbitmq-bundle-1 (ocf:pacemaker:remote): Started overcloud-controller-0 * Replica[2] * rabbitmq (ocf:heartbeat:rabbitmq-cluster): Started rabbitmq-bundle-2 * rabbitmq-bundle-podman-2 (ocf:heartbeat:podman): Started overcloud-controller-1 * rabbitmq-bundle-2 (ocf:pacemaker:remote): Started overcloud-controller-1 Node Attributes: * Node: overcloud-controller-0 (1): * galera-last-committed : -1 * galera-no-grastate : true * galera-role : true * haproxy-role : true * rabbitmq-role : true * rmq-node-attr-last-known-rabbitmq : rabbit(a)overcloud-controller-0.internalapi.tb27.cld * rmq-node-attr-rabbitmq : rabbit(a)overcloud-controller-0.internalapi.tb27.cld * Node: overcloud-controller-1 (2): * galera-last-committed : -1 * galera-no-grastate : true * galera-role : true * haproxy-role : true * rabbitmq-role : true * rmq-node-attr-last-known-rabbitmq : rabbit(a)overcloud-controller-1.internalapi.tb27.cld * rmq-node-attr-rabbitmq : rabbit(a)overcloud-controller-1.internalapi.tb27.cld * Node: overcloud-controller-2 (3): * galera-bootstrap : true * galera-role : true * haproxy-role : true * master-galera : 100 * rabbitmq-role : true * rmq-node-attr-last-known-rabbitmq : rabbit(a)overcloud-controller-2.internalapi.tb27.cld * rmq-node-attr-rabbitmq : rabbit(a)overcloud-controller-2.internalapi.tb27.cld Migration Summary: * Node: galera-bundle-0@overcloud-controller-2: * galera: migration-threshold=1000000 fail-count=1 last-failure='Tue Jan 7 10:32:38 2025' Failed Resource Actions: * galera_promote_0 on galera-bundle-0 'error' (1): call=116, status='complete', exitreason='Failed initial monitor action', last-rc-change='Tue Jan 7 10:32:28 2025', queued=0ms, exec=10603ms Tickets: PCSD Status: overcloud-controller-0: Online overcloud-controller-1: Online overcloud-controller-2: Online Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled Does someone perhaps knows where the issue lies? Thank you very much!

3 4

[release] Release countdown for week R-11, Jan 13 - 17
by Elõd Illés 10 Jan '25

10 Jan '25

Development Focus ----------------- We are now past the Epoxy-2 milestone, and entering the last development phase of the cycle. Teams should be focused on implementing planned work for the cycle. Now is a good time to review those plans and reprioritize anything if needed based on the what progress has been made and what looks realistic to complete in the next few weeks. General Information ------------------- Looking ahead to the end of the release cycle, please be aware of the feature freeze dates. Those vary depending on deliverable type: * General libraries (except client libraries) need to have their last feature release before Non-client library freeze (February 20th, 2025). Their stable branches are cut early. * Client libraries (think python-*client libraries) need to have their last feature release before Client library freeze (February 27th, 2025) * Deliverables following a cycle-with-rc model (that would be most services) observe a Feature freeze on that same date, February 27th, 2025. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. After feature freeze, cycle-with-rc deliverables need to produce a first release candidate (and a stable branch) before RC1 deadline (March 13th, 2025) * Deliverables following cycle-with-intermediary model can release as necessary, but in all cases before Final RC deadline (March 27th, 2025) Upcoming Deadlines & Dates -------------------------- Non-client library freeze: February 20th, 2025 (R-6 week) Client library freeze: February 27th, 2025 (R-5 week) Epoxy-3 milestone: February 27th, 2025 (R-5 week) Előd Illés irc: elodilles @ #openstack-release

1 0

Openstack kolla networking setup
by matek7323＠interia.pl 10 Jan '25

10 Jan '25

I am setting up an Openstack Kolla environment with the goal of moving all virtualizations into one place At the moment I have 3 management controllers and 5 compute node for virtualization + CEPH I have there such settings as eth0 management network 172.16.50.0 and eth1 as external neutron 91.1.1.0/24 as public address And now I want to achieve two things: 1) How to make the vmka get the public address directly without natting it on the principle. Private address --> 10.224.1.15 --> public 91.1.1.15 2) Leave the option of nating after through the vrouter so that the machine assigns a private address and then a public IP address To all these servers for now the situation looks like this Port eth0 = vlan 750 access on the switch - MGMT port eth1 = vlan 888 access on switch - VLAN to operator so that I can broadcast public IP address network: version: 2 ethernets: eth0: match: macaddress: “***********” addresses: - “172.16.50.11/24” nameservers: addresses: - 8.8.8.8 set-name: eth0 routes: - to: default via: 172.16.50.254 eth1: match: macaddress: “**********” dhcp4: false dhcp6: false set-name: eth1 Each of the servers has the same config and now how to configure this to adjust the config of the coll, the servers as well as the openstack to be able to work natively and on the principle of assigning IP addresses directly to the machine by vrouter Translated with DeepL.com (free version)

1 0

[cinder][kubernetes] pvc stuck in reserved state
by Eugen Block 10 Jan '25

10 Jan '25

Hi, we have a couple of k8s clusters in our openstack environment (Victoria) and are facing some problems when pods get moved to different worker nodes. The PVCs get stuck in "reserved" state, and they only get attached after resetting the volumes to "available" state. First I thought it was due to the missing multiattach volume type which seems to work fine in our customer's cluster. I also found the "duplicate connector" error messages in the nova-compute.log: > Unable to update attachment.(Invalid volume: duplicate connectors > detected on volume But after adding the multiattach volume type and retyping the affected volumes, the PVCs still get stuck in "reserved" state while I don't see any error messages on the hypervisors anymore. So it seems that something has changed/improved, but now I don't have a trace where to look next. Could this be a cinder-csi driver issue or something? Our cinder backend is ceph (Pacific). Not sure which information could help so please ask for anything I can provide. Any pointers are highly appreciated! Thanks! Eugen

1 2

Third Party CI using Zuul v3 and nodepool - request for set up
by Roquesalane, Jean-Pierre 09 Jan '25

09 Jan '25

Hello OpenStack lovers! First, let me wish you all the best for this upcoming year. All our Cis are currently broken and we recently lost the team responsible for maintaining it. Unfortunately, there is a very poor documentation, and I guess we must rebuild something. I'm currently looking at Software Factory but can't get my hands on a starting point. I have a system deployed but would appreciate if you can lead me to people who have configured Zuul and nodepool. Our previous system was based on Jenkins and devstack-gate as a side note. I'll document the whole process as a help for future users. Thank you. Jean-Pierre Roquesalane SW System Sr Principal Engineer - OpenStack Champion Dell Technologies | Security and Ecosystem Software Mobile +33 6 21 86 79 04<tel:1-613-314-8106> Email : jeanpierre.roquesalane(a)dell.com<mailto:jeanpierre.roquesalane@dell.com> [cid:image001.png@01DB62C5.98BB1B40] Dell SAS, Si?ge Social 1 rond point Benjamin Franklin 34000 Montpellier, France. Capital 1,946,545.26 Euros. RCS Montpellier 351 528 229 - APE 4651Z. TVA Intracommunautaire FR 20 351 528 229. SIRET 351 528 229 00088. Vat Numbers: FR 20351528229 (France) / IT00001709997 (Italy) / ESN0012622G (Spain) Internal Use - Confidential

1 0

LXC containers
by engineer2024 09 Jan '25

09 Jan '25

I am trying to setup OSA branch 29.2.0 caracal. Whenever I try creating lxc containers using setup-hosts playbook, the new containers of version ubuntu 22 , as is the host, are missing openssh-server packages. The file /etc/ansible/roles/openstack_hosts/vars/ubuntu-22.04.yml, doesn't mention the ssh package in the distro packages. How to get ssh pkgs installed in the lxc containers ? Please help with this issue

2 1

LXC networking
by engineer2024 09 Jan '25

09 Jan '25

I have observed that in the OSA setups, on the controller nodes, each lxc container has two networks inside , eth0, eth1 interfaces. Ideally eth1 should be attached to br-mgmt bridge via veth pair. When I do 'ip link show type veth' on the host node, it is listing both the veth interfaces for each lxc container, but it is not listing the other member of that veth pair. --- # ip link show type veth 9f9add59_eth0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxcbr0 state UP mode DEFAULT group default qlen 1000 link/ether fe:b6:db:27:e4:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 9f9add59_eth1@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-mgmt state UP mode DEFAULT group default qlen 1000 link/ether fe:a8:30:7a:ac:3f brd ff:ff:ff:ff:ff:ff link-netnsid 0 --- --- # ip link show type veth | grep 9f9add59 72: 9f9add59_eth0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxcbr0 state UP mode DEFAULT group default qlen 1000 73: 9f9add59_eth1@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-mgmt state UP mode DEFAULT group default qlen 1000 # --- But I can't find the other member of the veth pair, for example, for the interface 9f9add59_eth1, in the ip list. But the containers are running and reachable on the br-mgmt network. How to find out the other end of a veth pair, which is connected to the bridge ?

2 3

[blazar] reservation quota
by Winicius Allan 09 Jan '25

09 Jan '25

Is there a way to determine the quota of a project that the users can reserve? Regards.

2 1

[neutron] Proposal to deprecate and remove vlan_transparent config option
by Sławek Kapłoński 09 Jan '25

09 Jan '25

Hi, We have config option to enable vlan_transparent api extension currently. We will also have similar config option for vlan_qinq (it is now proposed in [1]). In both cases we have validation by mechanism drivers if option is or is not supported while networks are created. We can also filter out enabled API extensions by mechanism drivers, see [2] for example. So I proposed [3] to remove that config option and keep API extension to be loaded always if enabled ML2 drivers supports it. But before I will propose patch for that I would like to ask if there is maybe some use case for this config option really and maybe some of you somehow depends on it. In such case maybe we should keep it like it is now and don't remove it. Please let me know if you are using somehow this option. [1] https://review.opendev.org/c/openstack/neutron/+/937372 [2] https://github.com/openstack/neutron/blob/559672f777312d00c4358626650cc577e… [3] https://bugs.launchpad.net/neutron/+bug/2092174 -- Slawek Kaplonski Principal Software Engineer Red Hat

3 3

[Tempest] Devstack to configure tempest on a remote node
by Jakub Libosvar 09 Jan '25

09 Jan '25

Hello stackers, I have been attempting to deploy a “spine & leaf” like topology for upstream testing job of the ovn-bgp-agent [1]. The idea is to have an all-in-one devstack VM where all openstack services are running. This VM is plugged to two leaf switches respectively with its two interfaces, and each switch is connected to the “spine” switch. With the leaf switches behaving as route reflected we can validate routes are advertised to the spine. Now in order to validate that, I need to execute the tempest tests on the spine. That means I want to use devstack on the spine VM that calls OpenStack services API on the devstack all-in-one VM. The spine VM should not have any openstack service running. The problem I’m struggling with that the tempest is configured based on the ENABLED_SERVICES for the current box. I tried to hack the spine VM portion by calling configure_tempest with custom ENABLED_SERVICES from a local.sh [2] which seems to configure the tempest.conf correctly but the stack.sh script calls test-config [3] phase after local.sh is invoked, which re-configures tempest.conf back to the state as if no OpenStack services were enabled. I kinda hit the wall and ran out of ideas for hacks/workarounds with the current devstack codebase. I thought one way of solving this would be to decouple ENABLED_SERVICES into two separate variables, one being used to configure services and one for tempest purposes but before I dive into that, I wanted to ask if anybody has any experience doing something similar (configuring only tempest with devstack) or maybe there is already some variable I am not aware of. Thank you for any inputs, ideas or suggestions :) Kuba [1] https://review.opendev.org/c/openstack/ovn-bgp-agent/+/936968 [2] https://review.opendev.org/c/openstack/ovn-bgp-agent/+/936968/54/playbooks/… [3] https://opendev.org/openstack/devstack/src/branch/master/stack.sh#L1482-L15…

2 1

RDO User Survey
by Amy Marrich 08 Jan '25

08 Jan '25

Hi everyone, During today's RDO meeting we had a few topics that we need further information on in order to make decisions. As a result, we have created a 2 question survey[0] to collect the needed information. This survey will remain open for 2 weeks so we can make plans for the Epoxy release with the gathered information. The first question is in regards to CentOS Stream version(s) for the Epoxy release, CentOS Stream 9 only, 10 only, or both versions. The second question is to determine whether future RDO releases will continue to be all releases or only SLURP releases, if SLURP are the only releases folks are using. Thanks, Amy 0 - https://forms.gle/44atURvazYPXVvwZ8

1 0

[Neutron] Bug report last 2 weeks
by Rodolfo Alonso Hernandez 08 Jan '25

08 Jan '25

Hello Neutrinos: This is the bug report of the last 2 weeks. HIGH * Packet Logging Framework (OVN/ML2) ( https://bugs.launchpad.net/neutron/+bug/2092474) *Not assigned*. * with pyroute2 master dvr functional tests failing with route mismatch as returned routes are not from specific table ( https://bugs.launchpad.net/neutron/+bug/2092567) There is an open bug for pyroute2 (https://github.com/svinota/pyroute2/issues/1225) that should fix this problem. * with pyroute2 master dvr functional tests failing as pyroute.NetNS not honoring flags (https://bugs.launchpad.net/neutron/+bug/2092722) There is an open bug for pyroute2 (https://github.com/svinota/pyroute2/issues/1226) that should fix this problem. * Potential GPL violation in master branch due to GPL script embedded into Apache2 python code (https://bugs.launchpad.net/neutron/+bug/2092907) *Not assigned*. * Metadata service does not work when port security is disabled ( https://bugs.launchpad.net/neutron/+bug/2092971) *Not assigned*. MEDIUM * DeprecationWarning: create_subnetpool ( https://bugs.launchpad.net/neutron/+bug/2092657) *Not assigned*. * NoneType' object has no attribute 'registered_rules' ( https://bugs.launchpad.net/neutron/+bug/2092659) *Not assigned*. * OVN: Ports not activated on time when DPDK is used ( https://bugs.launchpad.net/neutron/+bug/2092407) Assigned to Ihar, patch proposed: https://review.opendev.org/c/openstack/neutron/+/938197. INCOMPLETE * deadlock： delete the network ( https://bugs.launchpad.net/neutron/+bug/2092983) Regards.

3 5

check Nova keyring file
by Michel Niyoyita 08 Jan '25

08 Jan '25

Hello Team, I am integrating Openstack 2024.1 with ceph cluster reef running on ubuntu 22.05, I have configured openstack global as follow: ceph_nova_keyring: "ceph.client.nova.keyring" ceph_nova_user: "nova" ceph_nova_pool_name: "vms" I have copied nova keyring too from /etc/ceph to /etc/kolla/config/nova ; /etc/kolla/config/nova$ ls -l total 16 -rw-r--r-- 1 stack stack 64 Jan 7 13:12 ceph.client.cinder.keyring -rw-r--r-- 1 stack stack 62 Jan 7 15:11 ceph.client.nova.keyring -rw-r--r-- 1 stack stack 283 Jan 7 14:51 ceph.conf -rw-rw-r-- 1 stack stack 182 Jan 7 15:14 nova.conf (kolla-ansible) stack@open-deployment:/etc/kolla/config/nova$ I have created nova user in ceph : sudo ceph auth get-or-create client.nova mon 'profile rbd' osd 'profile rbd pool=vms, profile rbd pool=images' mgr 'profile rbd pool=vms' -o /etc/ceph/ceph.client.nova.keyring But while deploying I am facing the following issue : TASK [nova-cell : Check nova keyring file] ******************************************************************************************** fatal: [compute1 -> localhost]: FAILED! => {"msg": "No file was found when using first_found."} fatal: [compute2 -> localhost]: FAILED! => {"msg": "No file was found when using first_found."} fatal: [compute3 -> localhost]: FAILED! => {"msg": "No file was found when using first_found."} the same error is occuring also when deploying allinone . I tried two different release : 2023.2 and 2024.1 , this is the ansible version installed : pip install 'ansible>=8,<9' tree /etc/kolla/config/ /etc/kolla/config/ ├── cinder │ ├── ceph.conf │ ├── cinder-backup │ │ ├── ceph.client.cinder-backup.keyring │ │ └── ceph.client.cinder.keyring │ └── cinder-volume │ └── ceph.client.cinder.keyring ├── glance │ ├── ceph.client.cinder.keyring │ ├── ceph.client.glance.keyring │ └── ceph.conf └── nova ├── ceph.client.cinder.keyring ├── ceph.client.nova.keyring └── ceph.conf Kindly help to solve the issue , it takes long time without solution. previously was not occured on 2023.1 Best Regards Michel

2 2

Nova create vm with special ipv4 and ipv6
by Alex Song (宋文平) 07 Jan '25

07 Jan '25

Hi Nova team, does nova support create vm with special ipv4 and ipv6? As port create with ipv4 and ipv6 works by: openstack port create 111 --network 10851e96-8995-4988-b819-0514b38d7353 --fixed-ip subnet=a564ac90-3ff3-47d7-8ae5-20e2666eaa3c,ip-address=172.24.4.54 --fixed-ip subnet=afa3c858-c3d3-4784-8964-2c46840b714d,ip-address=2001:db8::183 but vm create with ipv4 and ipv6 with the cmd openstack server create --flavor 1 --image 62254c92-a579-4c8a-a2f4-f2624cef8ec1 --nic net-id=10851e96-8995-4988-b819-0514b38d7353,v4-fixed-ip=172.24.4.53,v6-fixed-ip=2001:db8::188 test raise exception: openstack server create: error: argument --nic: Invalid argument net-id=10851e96-8995-4988-b819-0514b38d7353,v4-fixed-ip=172.24.4.53,v6-fixed-ip=2001:db8::188; either 'v4-fixed-ip' or 'v6-fixed-ip' should be specified but not both. So why nova doesnot support?

2 3

[ops][nova] How to make sure that on a hypervisor there is room for a VM with a specific flavor
by Massimo Sgaravatto 07 Jan '25

07 Jan '25

Dear all I have the following use case: I defined a flavor F for VMS with 10 VCPUs, 20 GB of RAM, 30 GB of root disk, 1 GPU XYZ. I use PCI passthrough to manage GPUs. I have only a XYZ GPU on my cluster (on a H hypervisor), so a VM using this flavor F must be "placed" to the compute node H (and this is not a problem: I know how to do that). H is very large: it has a lot of RAM and cores, so it can host other VMs that do not use GPUs. How can I make it so that VMs without GPU can go on H, but there must be room for a VM with flavor F (if it doesn't already exist)? Thanks, and happy new year ! Cheers, Massimo

3 3

ERROR dpif_netlink|INFO|dpif_netlink_meter_transact OVS_METER_CMD_SET failed
by alibasafa18＠gmail.com 07 Jan '25

07 Jan '25

I have set up OpenStack using Kolla Ansible with OVS and OVN. On one compute node, meters are functioning correctly, but on another, I am encountering an issue. Both nodes use the system datapath, have the same kernel version, and run the same version of OVS. Despite this, the problem persists. Has anyone experienced a similar issue? What could be the root cause? For example, I have two compute nodes. One of them does not have any issues, but the other meter did not work . following compute meter does not working: > sudo docker exec -it openvswitch_vswitchd bash (openvswitch-vswitchd)[root@compute1 /]# lsmod | grep open openvswitch 172032 13 nsh 16384 1 openvswitch nf_conncount 24576 1 openvswitch nf_nat 49152 1 openvswitch nf_conntrack 172032 6 nf_nat,nfnetlink_cttimeout,openvswitch,nf_conntrack_netlink,nf_conncount,ip_vs nf_defrag_ipv6 24576 3 nf_conntrack,openvswitch,ip_vs libcrc32c 16384 6 nf_conntrack,nf_nat,openvswitch,btrfs,raid456,ip_vs (openvswitch-vswitchd)[root@compute1 /]# ovs-appctl dpif/show-dp-features br-int Masked set action: Yes Tunnel push pop: No Ufid: Yes Truncate action: Yes Clone action: Yes Sample nesting: 10 Conntrack eventmask: Yes Conntrack clear: Yes Max dp_hash algorithm: 0 Check pkt length action: Yes Conntrack timeout policy: Yes Explicit Drop action: No Optimized Balance TCP mode: No Conntrack all-zero IP SNAT: Yes MPLS Label add: Yes Max VLAN headers: 2 Max MPLS depth: 3 Recirc: Yes CT state: Yes CT zone: Yes CT mark: Yes CT label: Yes CT state NAT: Yes CT orig tuple: Yes CT orig tuple for IPv6: Yes IPv6 ND Extension: No (openvswitch-vswitchd)[root@compute1 /]# ovs-ofctl -OOpenFlow15 meter-features br-int OFPST_METER_FEATURES reply (OF1.5) (xid=0x2): max_meter:0 max_bands:0 max_color:0 band_types: 0 capabilities: 0 (openvswitch-vswitchd)[root@compute1 /]# ovs-vsctl --version ovs-vsctl (Open vSwitch) 3.3.2 DB Schema 8.5.0 (openvswitch-vswitchd)[root@compute1 /]# uname -r 5.15.0-86-generic (openvswitch-vswitchd)[root@compute1 /]# uptime 08:27:39 up 87 days, 19:51, 0 users, load average: 3.36, 4.58, 4.19 ====================== following compute working without issue: > sudo docker exec -it openvswitch_vswitchd bash (openvswitch-vswitchd)[root@compute2 /]# lsmod | grep open openvswitch 172032 23 nsh 16384 1 openvswitch nf_conncount 24576 1 openvswitch nf_nat 49152 1 openvswitch nf_conntrack 172032 6 nf_nat,nfnetlink_cttimeout,openvswitch,nf_conntrack_netlink,nf_conncount,ip_vs nf_defrag_ipv6 24576 3 nf_conntrack,openvswitch,ip_vs libcrc32c 16384 6 nf_conntrack,nf_nat,openvswitch,btrfs,raid456,ip_vs (openvswitch-vswitchd)[root@compute2 /]# ovs-appctl dpif/show-dp-features br-int Masked set action: Yes Tunnel push pop: No Ufid: Yes Truncate action: Yes Clone action: Yes Sample nesting: 10 Conntrack eventmask: Yes Conntrack clear: Yes Max dp_hash algorithm: 0 Check pkt length action: Yes Conntrack timeout policy: Yes Explicit Drop action: No Optimized Balance TCP mode: No Conntrack all-zero IP SNAT: Yes MPLS Label add: Yes Max VLAN headers: 2 Max MPLS depth: 3 Recirc: Yes CT state: Yes CT zone: Yes CT mark: Yes CT label: Yes CT state NAT: Yes CT orig tuple: Yes CT orig tuple for IPv6: Yes IPv6 ND Extension: No (openvswitch-vswitchd)[root@compute27 /]# ovs-ofctl -OOpenFlow15 meter-features br-int OFPST_METER_FEATURES reply (OF1.5) (xid=0x2): max_meter:200000 max_bands:1 max_color:0 band_types: drop capabilities: kbps pktps burst stats (openvswitch-vswitchd)[root@compute2 /]# ovs-vsctl --version ovs-vsctl (Open vSwitch) 3.3.2 DB Schema 8.5.0 (openvswitch-vswitchd)[root@compute2 /]# uname -r 5.15.0-86-generic (openvswitch-vswitchd)[root@compute2 /]# uptime 08:29:47 up 87 days, 19:53, 0 users, load average: 11.23, 13.13, 13.29

1 0

ERROR dpif_netlink|INFO|dpif_netlink_meter_transact OVS_METER_CMD_SET failed
by alibasafa18＠gmail.com 07 Jan '25

07 Jan '25

I have set up OpenStack using Kolla Ansible with OVS and OVN. On one compute node, meters are functioning correctly, but on another, I am encountering an issue. Both nodes use the system datapath, have the same kernel version, and run the same version of OVS. Despite this, the problem persists. Has anyone experienced a similar issue? What could be the root cause? i have both compute for example in the following. following compute node i have issue: > sudo docker exec -it openvswitch_vswitchd bash (openvswitch-vswitchd)[root@si1-compute1 /]# lsmod | grep open openvswitch 172032 13 nsh 16384 1 openvswitch nf_conncount 24576 1 openvswitch nf_nat 49152 1 openvswitch nf_conntrack 172032 6 nf_nat,nfnetlink_cttimeout,openvswitch,nf_conntrack_netlink,nf_conncount,ip_vs nf_defrag_ipv6 24576 3 nf_conntrack,openvswitch,ip_vs libcrc32c 16384 6 nf_conntrack,nf_nat,openvswitch,btrfs,raid456,ip_vs (openvswitch-vswitchd)[root@si1-compute4174 /]# ovs-appctl dpif/show-dp-features br-int Masked set action: Yes Tunnel push pop: No Ufid: Yes Truncate action: Yes Clone action: Yes Sample nesting: 10 Conntrack eventmask: Yes Conntrack clear: Yes Max dp_hash algorithm: 0 Check pkt length action: Yes Conntrack timeout policy: Yes Explicit Drop action: No Optimized Balance TCP mode: No Conntrack all-zero IP SNAT: Yes MPLS Label add: Yes Max VLAN headers: 2 Max MPLS depth: 3 Recirc: Yes CT state: Yes CT zone: Yes CT mark: Yes CT label: Yes CT state NAT: Yes CT orig tuple: Yes CT orig tuple for IPv6: Yes IPv6 ND Extension: No (openvswitch-vswitchd)[root@si1-compute1 /]# ovs-ofctl -OOpenFlow15 meter-features br-int OFPST_METER_FEATURES reply (OF1.5) (xid=0x2): max_meter:0 max_bands:0 max_color:0 band_types: 0 capabilities: 0 (openvswitch-vswitchd)[root@si1-compute1 /]# ovs-vsctl --version ovs-vsctl (Open vSwitch) 3.3.2 DB Schema 8.5.0 (openvswitch-vswitchd)[root@si1-compute1 /]# uname -r 5.15.0-86-generic (openvswitch-vswitchd)[root@si1-compute1 /]# uptime 08:27:39 up 87 days, 19:51, 0 users, load average: 3.36, 4.58, 4.19 ================================= following compute node working correctly: (openvswitch-vswitchd)[root@si1-compute2 /]# lsmod | grep open openvswitch 172032 23 nsh 16384 1 openvswitch nf_conncount 24576 1 openvswitch nf_nat 49152 1 openvswitch nf_conntrack 172032 6 nf_nat,nfnetlink_cttimeout,openvswitch,nf_conntrack_netlink,nf_conncount,ip_vs nf_defrag_ipv6 24576 3 nf_conntrack,openvswitch,ip_vs libcrc32c 16384 6 nf_conntrack,nf_nat,openvswitch,btrfs,raid456,ip_vs (openvswitch-vswitchd)[root@si1-compute2 /]# ovs-appctl dpif/show-dp-features br-int Masked set action: Yes Tunnel push pop: No Ufid: Yes Truncate action: Yes Clone action: Yes Sample nesting: 10 Conntrack eventmask: Yes Conntrack clear: Yes Max dp_hash algorithm: 0 Check pkt length action: Yes Conntrack timeout policy: Yes Explicit Drop action: No Optimized Balance TCP mode: No Conntrack all-zero IP SNAT: Yes MPLS Label add: Yes Max VLAN headers: 2 Max MPLS depth: 3 Recirc: Yes CT state: Yes CT zone: Yes CT mark: Yes CT label: Yes CT state NAT: Yes CT orig tuple: Yes CT orig tuple for IPv6: Yes IPv6 ND Extension: No (openvswitch-vswitchd)[root@si1-compute2 /]# ovs-ofctl -OOpenFlow15 meter-features br-int OFPST_METER_FEATURES reply (OF1.5) (xid=0x2): max_meter:200000 max_bands:1 max_color:0 band_types: drop capabilities: kbps pktps burst stats (openvswitch-vswitchd)[root@si1-compute2 /]# ovs-vsctl --version ovs-vsctl (Open vSwitch) 3.3.2 DB Schema 8.5.0 (openvswitch-vswitchd)[root@si1-compute2 /]# uname -r 5.15.0-86-generic (openvswitch-vswitchd)[root@si1-compute2 /]# uptime 08:29:47 up 87 days, 19:53, 0 users, load average: 11.23, 13.13, 13.29

1 0

[tc][all] OpenStack Technical Committee Weekly Meeting Agenda (2025.1/R-12)
by Goutham Pacha Ravi 07 Jan '25

07 Jan '25

Hello Stackers, Happy New Year! The OpenStack Technical Committee will resume its regular meeting schedule this week. The next weekly meeting is scheduled to take place simultaneously on Video (Zoom) and IRC (OFTC) on January 7, 2024, at 1800 UTC. Please find the meeting agenda and details for joining the meeting on the meeting wiki [1]. On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] TC Meeting Agenda, 2025-01-07: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

custom playbook for nova
by engineer2024 06 Jan '25

06 Jan '25

can anyone guide me what are the tasks required for configuring nova service in OSA, so that I want to execute only those tasks and db steps , skipping the package installation steps. So I need to run nova playbook just to configure the nova like nova.conf, populating it with relevant options, do nova db migrate and then just restart the services. Is there a way to do it ? Thanks elinux

2 2

yoga to antelope upgrade path
by Satish Patel 05 Jan '25

05 Jan '25

Folks, I am running yoga in production with ovn driver and planning to upgrade to the 2023.1 release. Are there any issues to directly go to 2023.1 before intermediate release? Or anything critical I should be aware of?

3 3

[neutron][ovn] ovn database backup and restore
by Satish Patel 05 Jan '25

05 Jan '25

Folks, I have OVN running on 3 node controllers with a RAFT clustering. What is the best way to take backup of all the databases of nb_db and sb_db and how to restore them during a full disaster situation? Is the following the correct way to take backup? ovsdb-client backup unix:///run/ovn/ovnnb_db.sock OVN_Northbound > nb-snap-$(date +%s).txt ovsdb-client backup unix:///run/ovn/ovnsb_db.sock OVN_Southbound > sb-snap-$(date +%s).txt During restore on what node we should restore and does restore work on fresh cluster or existing cluster? How do other folks handle disaster scenarios where all 3 nodes are down or required migration?

3 6

Failing at TASK [glance : Copy over multiple ceph configs for Glance]
by Karera Tony 04 Jan '25

04 Jan '25

Hello Team, I am deploying Openstack release 2024.02 with External Ceph. However I am encountering the error below. But it is failing here TASK [glance : Copy over multiple ceph configs for Glance] Please assist Regards Tony Karera

2 1

[release] Release countdown for week R-12
by Thierry Carrez 03 Jan '25

03 Jan '25

Development Focus ----------------- The epoxy-2 milestone is next week, on January 9th, 2025! Epoxy-related specs should now be finalized so that teams can move to implementation ASAP. Some teams observe specific deadlines on the second milestone (mostly spec freezes): please refer to https://releases.openstack.org/epoxy/schedule.html for details. General Information ------------------- Libraries need to be released at least once per milestone period. Next week, the release team will propose releases for any library that has not been otherwise released since milestone 1. PTL's and release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well. A +1 would be appreciated, but if we do not hear anything at all by the end of the week, we will assume things are OK to proceed. Remember that non-library deliverables that follow the cycle-with-intermediary release model should have an intermediary release before milestone-2. Those who haven't will be proposed to switch to the cycle-with-rc model, which is more suited to deliverables that are released only once per cycle. Next week is also the deadline to freeze the contents of the final release. All new 2025.1 Epoxy deliverables need to have a deliverable file in https://opendev.org/openstack/releases/src/branch/master/deliverables and need to have done a release by milestone-2. Changes proposing those deliverables for inclusion in 2025.1 Epoxy have been posted, please update them with an actual release request before the milestone-2 deadline if you plan on including that deliverable in 2025.1 Epoxy, or -1 if you need one more cycle to be ready. Upcoming Deadlines & Dates -------------------------- Epoxy-2 Milestone: January 9th, 2025 2025.1 Epoxy final release: April 2nd, 2025 -- Thierry Carrez (ttx)

1 0