March 2025 - openstack-discuss - lists.openstack.org

[ops][neutron] Neutron-db-manage upgrade from Antelope to Bobcat fails with "foreign key constraint 'porthints_ibfk_1' are incompatible" error
by federica fanzago 20 May '25

20 May '25

Dear all, we are running an OpenStack deployment Yoga on AlmaLinux 9.5 and we are attempting the upgrade Openstack up to Caracal release, one release at time (Zed--Antelope--Bobcat--Caracal) The procedure we are following is to shut down all openstack services, to remove the "old" openstack release and to install the new one, upgrading all service databases. This procedure works well for Zed and Antelope, but during the installation of the Bobcat release, the Neutron database upgrade fails when creating the new "porthints" table. The command we are using is su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron and the error message we receive is ... File "/usr/lib/python3.9/site-packages/pymysql/err.py", line 107, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (3780, "Referencing column 'port_id' and referenced column 'id' in foreign key constraint 'porthints_ibfk_1' are incompatible.") [SQL: CREATE TABLE porthints ( port_id VARCHAR(36) NOT NULL, hints VARCHAR(4095) NOT NULL, PRIMARY KEY (port_id), FOREIGN KEY(port_id) REFERENCES ports (id) ON DELETE CASCADE )ENGINE=InnoDB ... Our "ports" table uses the collation utf8mb3_general_ci mysql> SHOW FULL COLUMNS FROM ports; +------------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+ | Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment | +------------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+ | project_id | varchar(255) | utf8mb3_general_ci | YES | MUL | NULL | | select,insert,update,references | | | id | varchar(36) | utf8mb3_general_ci | NO | PRI | NULL | | select,insert,update,references | | | name | varchar(255) | utf8mb3_general_ci | YES | | NULL | | select,insert,update,references | | | network_id | varchar(36) | utf8mb3_general_ci | NO | MUL | NULL | | select,insert,update,references | | | mac_address | varchar(32) | utf8mb3_general_ci | NO | | NULL | | select,insert,update,references | | | admin_state_up | tinyint(1) | NULL | NO | | NULL | | select,insert,update,references | | | status | varchar(16) | utf8mb3_general_ci | NO | | NULL | | select,insert,update,references | | | device_id | varchar(255) | utf8mb3_general_ci | NO | MUL | NULL | | select,insert,update,references | | | device_owner | varchar(255) | utf8mb3_general_ci | NO | | NULL | | select,insert,update,references | | | standard_attr_id | bigint | NULL | NO | UNI | NULL | | select,insert,update,references | | | ip_allocation | varchar(16) | utf8mb3_general_ci | YES | | NULL | | select,insert,update,references | | +------------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+ in Bobcat the collation required for the new "porthints" table is latin1_swedish_ci MariaDB [neutron]> show create table porthints; +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | porthints | CREATE TABLE `porthints` ( `port_id` varchar(36) NOT NULL, `hints` varchar(4095) NOT NULL, PRIMARY KEY (`port_id`), CONSTRAINT `porthints_ibfk_1` FOREIGN KEY (`port_id`) REFERENCES `ports` (`id`) ON DELETE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci | So, we have tried to create the porthints table in mysql, forcing the collation to utf8mb3_general_ci mysql> CREATE TABLE porthints ( port_id VARCHAR(36) NOT NULL, hints VARCHAR(4095) NOT NULL, PRIMARY KEY (port_id), FOREIGN KEY(port_id) REFERENCES ports (id) ON DELETE CASCADE )ENGINE=InnoDB DEFAULT CHARSET=utf8 DEFAULT COLLATE utf8mb3_general_ci; and commenting out the creation of the new table in the neutron code /usr/lib/python3.9/site-packages/neutron/db/migration/alembic_migrations/versions/2023.2/expand/6f1145bff34c_port_hints.py. In this way, the upgrade works, but we encounter the same kind of problem during the upgrade to Caracal, specifically with the creation of the new table porthardwareoffloadtype. Have you already encountered this issue during the Neutron upgrade? Do you have any suggestions on how to upgrade the neutron DB, without manually creating new tables? Thanks a lot, cheers, Federica -- Federica Fanzago INFN Sezione di Padova Via Marzolo, 8 35131 Padova - Italy Tel: +39 049.967.7367 --

4 8

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 19 May '25

19 May '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 9

[openstack-helm] Are there any real users of the umbrella chart?
by Vladimir Kozhukalov 17 May '25

17 May '25

Hi team, This is to start the discussion about the future of the Openstack super (umbrella) chart [1]. My personal impression (maybe I am wrong) is that this chart is of extremely little interest. We continue to maintain this code (and spend testing resources) but maybe there are no real users of it. The chart deploys a compute-kit scheme but the same result can be achieved if one deploys multiple separate charts one-by-one. It seems convenient to have this single super chart that deploys Openstack with a single command, but IMO even small operators prefer using separate charts. Having said that, I would suggest removing this chart if it is not needed. [1] https://opendev.org/openstack/openstack-helm/src/branch/master/openstack -- Best regards, Kozhukalov Vladimir

3 4

[manila] Removal of the Standalone CephFS NFS protocol helper in Epoxy
by Ashley Rodriguez 02 May '25

02 May '25

Hi zorillas! As per subject line, we will be deprecating the standalone CephFS NFS protocol helper in this cycle. This is following our deprecation warning we put out in 2024.1 / Caracal [1] [2]. We've supported using a clustered Ceph NFS service since the Zed release. We've also added upgrade helpers to migrate from a standalone NFS-Ganesha service to the ceph-orch-deployed clustered NFS service; this code was backported to stable/2023.1 (Antelope). Although the upgrade path is disruptive, there is a UX improvement in Manila to ease this migration through surfacing “preferred” export paths. Moving forward, deployments must have “cephfs_nfs_cluster_id” in your configuration, otherwise the driver will fail to start. Manila will not make any export changes to NFS-Ganesha, so upgrading manila without an NFS cluster will mean that only the control plane is not going to work; but, data plane will remain unaffected. To assist migration, admins can also configure "cephfs_ganesha_export_ips" (or alternatively, "cephfs_ganesha_server_ip") alongside "cephfs_nfs_cluster_id". Setting these options will allow the CephFS driver to report additional export paths. These additional export paths will have the "preferred" metadata key set to False. Best, Manila team [1] https://review.opendev.org/c/openstack/manila/+/912648 [2] https://docs.openstack.org/manila/2024.2/configuration/shared-file-systems/…

5 7

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

[oslo] python-consul2 is unmaintained
by Takashi Kajinami 21 Apr '25

21 Apr '25

Hello, I recently noticed that python-consul2 library[1], which is used by the consul coordination driver in tooz, hasn't been updated for 3+ years. Looking at issues in the project, there is an indication that the library is not compatible with ACL API in consul >= 1.11[2]. There is another fork created[3] and this includes fixes to support new API endpoints. Although the driver in tooz does not directly use the feature, I think it's better to discuss our plan to mitigate risks caused by the unmaintained library. As far as I can think of, there are three options now 1. Deprecate consul driver and remove it 2. Keep using python-consul2 now. At least basic functionality is proven to work by our functional tests (Note that the tests do not cover all configuration patterns) 3. Replace python-consul2 by py-consul Before we start the overall discussions I'd like to ask a few questions to the wider audience, as our inputs. - Is anyone using the consul coordination driver now ? - Does anyone have experience with python-consul2 and/or py-consul ? Note that this discussion affects the http messaging driver[4] currently proposed, because it also requires client library to communicate with Consul. Actually I noticed the situation when I noticed that the requirement check job is failing because the library is not part of global requirements list... Thank you, Takashi Kajinami [1] https://github.com/poppyred/python-consul2 [2] https://github.com/poppyred/python-consul2/issues/42 [3] https://github.com/criteo-forks/py-consul [4] https://review.opendev.org/c/openstack/oslo.messaging/+/912499 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 3

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

GPU PCI passthrough woes.
by Andy Speagle 09 Apr '25

09 Apr '25

Hey Folks, I could use a little assistance getting GPU passthrough working. I had this working already for one flavor of nvidia gpu... and I've added some hosts with a much newer gpu... I've updated the pci_alias and pci_passthrough variables and those seem to be getting set properly in nova.conf passthrough_whitelist = [{"vendor_id":"10de", "product_id":"1b06"},{"vendor_id":"10de", "product_id":"26b9"}] alias = {"name": "gpu", "product_id": "1b06", "vendor_id": "10de"} alias = {"name": "gpu-l40s", "product_id": "26b9", "vendor_id": "10de"} I believe I have all of the iommu stuff configured and have the pci- stub module entries... dmesg output shows that the GPUs are being claimed by the stub module. $ openstack flavor show t1.small_gpu_l40s +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 0 | | id | af70c94e-0026-4a39-bc1e-dfb93b286a54 | | name | t1.small_gpu_l40s | | os-flavor-access:is_public | True | | properties | pci_passthrough:alias='gpu-l40s:1' | | ram | 2048 | | rxtx_factor | 1.0 | | swap | 0 | | vcpus | 1 | +----------------------------+--------------------------------------+ Yet... I can't seem to get an instance to run using that new flavor... keeps complaining about there not being enough hosts available. Fault: code: 500 created: 2024-10-30T02:04:33Z message: "No valid host was found. There are not enough hosts available." details: | Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 1580, in schedule_and_build_instances host_lists = self._schedule_instances(context, request_specs[0], File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 940, in _schedule_instances host_lists = self.query_client.select_destinations( File "/usr/lib/python3/dist- packages/nova/scheduler/client/query.py", line 41, in select_destinations return self.scheduler_rpcapi.select_destinations(context, spec_obj, File "/usr/lib/python3/dist-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/usr/lib/python3/dist- packages/oslo_messaging/rpc/client.py", line 189, in call result = self.transport._send( File "/usr/lib/python3/dist- packages/oslo_messaging/transport.py", line 123, in _send return self._driver.send(target, ctxt, message, File "/usr/lib/python3/dist- packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send return self._send(target, ctxt, message, wait_for_reply, timeout, File "/usr/lib/python3/dist- packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send raise result nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. Traceback (most recent call last): File "/usr/lib/python3/dist- packages/oslo_messaging/rpc/server.py", line 241, in inner return func(*args, **kwargs) File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 223, in select_destinations selections = self._select_destinations( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 250, in _select_destinations selections = self._schedule( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 416, in _schedule self._ensure_sufficient_hosts( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 455, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason) nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. Any clues on how/where to dig into this more to see what might be missing? Thanks. -- Andy Speagle

3 2

[all] OpenInfra Summit Europe Scheduled for October 2025
by Jeremy Stanley 08 Apr '25

08 Apr '25

In case anyone hasn't seen the news already, OpenInfra Summit Europe has been officially scheduled for October 17-19, 2025, at École Polytechnique near Paris, France: https://openinfra.dev/blog/openinfra-summit-2025 The OpenInfra Foundation is currently recruiting interested individuals to join a Programming Committee kickoff call on Thursday (2025-03-27). This call will be used to plan the conference tracks for the OpenInfra Summit ahead of the CFP, and will greatly benefit from OpenStack's involvement as always. If you would like to participate in the Programming Committee or assist in any other way, please email Helena Spease <helena(a)openinfra.dev>. I am also happy to try and point people in the right direction if you have any questions and prefer to reach out to me. On behalf of the foundation, we’re excited to bring the OpenInfra community together in France, and sincerely hope you'll join us! -- Jeremy Stanley

1 1

[manila]NFS-Ganesha exports
by Mikhail Okhrimenko 06 Apr '25

06 Apr '25

Hello community, Trying to setup the Manila version of Caracal with NFS-Ganesha backend. Ganesha version 4.0 is deployed using ceph orch in a ceph cluster version 17. Backend configuration in manila.conf [ganesha_nfs] driver_handles_share_servers = False share_backend_name = ganesha_nfs Share_driver = manila.share.drivers.cephfs.driver.CephFSDriver cephfs_protocol_helper_type = NFS cephfs_conf_path = /etc/ceph/ceph.conf cephfs_auth_id = manila cephfs_cluster_name = ceph cephfs_filesystem_name = {{ cephfs_filesystem_name }} cephfs_nfs_cluster_id = {{ cephfs_nfs_cluster_id }} I see the subvolume being created in ceph ceph fs subvolume ls <volume_name> but no export ceph nfs export ls <cluster_id> what am I doing wrong? what am I missing? in the manila-share logs I see a request to create a subvolume, but I don't see a request to create an export, for some reason manila doesn't even try to create an export

3 6

bug report
by Xavier Benjamin 05 Apr '25

05 Apr '25

whenever i try to connect to trove pts servers it would usually say fails to authenticate or something. Can you fix that please?

2 2

[octavia][nova][neutron] failed to failover loadbalancer
by Winicius Allan 02 Apr '25

02 Apr '25

Hello stackers! release: zed deploy-tool: kolla-ansible After an outage, all the load balancers from my cluster came to provisioning status ERROR because the o-hm0 interface was unavailable on controller nodes. I've created them again and tried a failover on load balancers. The octavia-worker logs show that failover was completed successfully, but the provisioning status remains with ERROR. Looking into nova-compute logs, I see that an exception was raised os_vif.exception.NetworkInterfaceNotFound: Network interface qvob8ac0f5f-46 not found The instance id for that interface matches the new amphora id. I'll attach the nova-compute logs here[1]. In neutron-server logs, there is no ERROR or WARNING that I can found, the only suspicious entry is Port b8ac0f5f-4613-4b95-9690-7cf59f739fd0 cannot update to ACTIVE because it is not bound. _port_provisioned /var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/plugin.py:360 Can anyone give me a light? [1] https://pastebin.com/hV3xgu0e

2 6

[nova] nova-manage fails when upgrading to Antelope
by Roby G 01 Apr '25

01 Apr '25

Hello everyone, I´m trying to upgrade to Antelope and have a problem when executing the db migrations. `nova-manage db sync` fails when upgrading from Zed to Antelope (same for Yoga --> Antelope) - Nova: 27.4.0 - Database: PostgreSQL 16 - Operating System: Ubuntu 22.04 - OpenStack Zed --> Antelope Full Log. /var/log/nova-manage.log: ``` 2025-03-04 11:42:14.963 2519974 INFO alembic.runtime.migration [None req-0e1f6874-3bac-49db-b525-94aebf9e0b51 - - - - - -] Context impl PostgresqlImpl. 2025-03-04 11:42:14.964 2519974 INFO alembic.runtime.migration [None req-0e1f6874-3bac-49db-b525-94aebf9e0b51 - - - - - -] Will assume transactional DDL. 2025-03-04 11:42:14.967 2519974 INFO alembic.runtime.migration [None req-0e1f6874-3bac-49db-b525-94aebf9e0b51 - - - - - -] Running upgrade ccb0fa1a2252 -> 960aac0e09ea, de-duplicate_indexes_in_instances__console_auth_tokens 2025-03-04 11:42:14.970 2519974 WARNING oslo_db.sqlalchemy.exc_filters [None req-0e1f6874-3bac-49db-b525-94aebf9e0b51 - - - - - -] DBAPIError exception wrapped.: psycopg2.errors.DependentObjectsStillExist: cannot drop index uuid because other objects depend on it DETAIL: constraint block_device_mapping_instance_uuid_fkey on table block_device_mapping depends on index uuid constraint consoles_instance_uuid_fkey on table consoles depends on index uuid constraint fixed_ips_instance_uuid_fkey on table fixed_ips depends on index uuid constraint instance_actions_instance_uuid_fkey on table instance_actions depends on index uuid constraint instance_extra_instance_uuid_fkey on table instance_extra depends on index uuid constraint instance_faults_instance_uuid_fkey on table instance_faults depends on index uuid constraint instance_info_caches_instance_uuid_fkey on table instance_info_caches depends on index uuid constraint instance_metadata_instance_uuid_fkey on table instance_metadata depends on index uuid constraint instance_system_metadata_instance_uuid_fkey on table instance_system_metadata depends on index uuid constraint migrations_instance_uuid_fkey on table migrations depends on index uuid constraint security_group_instance_association_instance_uuid_fkey on table security_group_instance_association depends on index uuid constraint virtual_interfaces_instance_uuid_fkey on table virtual_interfaces depends on index uuid HINT: Use DROP ... CASCADE to drop the dependent objects too. 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last): 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters self.dialect.do_execute( 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters) 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters psycopg2.errors.DependentObjectsStillExist: cannot drop index uuid because other objects depend on it 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters DETAIL: constraint block_device_mapping_instance_uuid_fkey on table block_device_mapping depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint consoles_instance_uuid_fkey on table consoles depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint fixed_ips_instance_uuid_fkey on table fixed_ips depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint instance_actions_instance_uuid_fkey on table instance_actions depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint instance_extra_instance_uuid_fkey on table instance_extra depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint instance_faults_instance_uuid_fkey on table instance_faults depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint instance_info_caches_instance_uuid_fkey on table instance_info_caches depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint instance_metadata_instance_uuid_fkey on table instance_metadata depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint instance_system_metadata_instance_uuid_fkey on table instance_system_metadata depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint migrations_instance_uuid_fkey on table migrations depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint security_group_instance_association_instance_uuid_fkey on table security_group_instance_association depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters constraint virtual_interfaces_instance_uuid_fkey on table virtual_interfaces depends on index uuid 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters HINT: Use DROP ... CASCADE to drop the dependent objects too. 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters 2025-03-04 11:42:14.970 2519974 ERROR oslo_db.sqlalchemy.exc_filters ``` Any suggestions? Thanks!

3 3

[Octavia]
by Vincent Godin 01 Apr '25

01 Apr '25

Hello Octavia Team, We have an Openstack infrastructure with 3 Nova AZ and 3 Cinder AZ. Furthermore, Octavia will use cinder for Amphora disk. Is there a way to create an Amphora so that the Compute AZ match the Cinder AZ ? As the storage is AZ specific, it's not possible to launch an Amphora on a Compute AZ with a storage on an alternate AZ. As a client, there is no need to fix where the Amphora will be launched, we just want to be sure that if the Amphora is planned to be launched on a specific AZ, its storage will be selected on the corresponding AZ too ! Vincent

3 2

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R+0)
by Goutham Pacha Ravi 01 Apr '25

01 Apr '25

Hello Stackers, The release team will be tagging the GA release for all cycle-with-rc projects for 2025.1 ("Epoxy") on 2025-04-02 [1][2]. Remarkably, this will be the 31st release of OpenStack and one of two that will mark 15 years of the community! There will be an OpenInfra Live event on 2025-04-03, where a few project maintainers will reflect on how the release cycle went, what it will bring to users, and project priorities for the 2025.2 ("Flamingo") release cycle [3]. Alongside project teams, the OpenStack Technical Committee has been preparing for the upcoming virtual Project Teams Gathering. We're still gathering topics, so please add yours if you're interested in seeding a relevant discussion. You can find all the discussion etherpads linked on the PTG Website [4]. In the past week, we completed the community testing goal to upgrade CI integration testing ("migrate-to-noble") [5]. Kudos to our goal champion, Ghanshyam Mann (gmann), and to everyone who helped with the effort! === Weekly Meeting === The last weekly meeting of the OpenStack TC was held over IRC in OFTC's #openstack-tc channel on 2025-03-25 [6]. We added a discussion topic on operator engagement to the os-operators' PTG slot. We caught up on CI issues that needed attention from several quarters, including a kernel regression that prevented release-critical patches from merging in Neutron and Kolla and warranted a workaround. A breaking change in Setuptools 78 impacted many projects on Zuul; however, OpenStack was minimally affected before the issue was resolved in Setuptools. A broken CentOS 9 mirror briefly caused problems due to an inconsistent state from an upstream source. It was brought to the TC's attention that the Skyline project commits pre-built static files (e.g., JavaScript bundles generated using Yarn) directly into its repository. This manual intervention creates a potential security risk similar to the XZ backdoor attack since there is no verifiable way to check whether these builds contain malicious code. The TC brainstormed that it would be better to offload the build process to Zuul, preventing manual intervention, and to ensure the yarn.lock file is always included in releases for better traceability. Operators and project contributors must be aware of this issue and the fact that builds are currently not reproducible, complicating packaging. There was a suggestion that OpenStack governance should enforce standardized project security practices. Since we haven't identified concrete steps to resolve the current issue, we'll revisit this topic in a future discussion. The next meeting of the OpenStack TC is today, 2025-04-01, at 1700 UTC. Please note the change in timing—it is an hour earlier than our past meetings. This meeting will be held simultaneously on Video (Zoom) and IRC (OFTC's #openstack-tc channel). Please find the conference details and agenda on the meeting's wiki page [7]. I hope you'll be able to join us there. === Governance Proposals === ==== Merged ==== - Mark CI migration to Ubuntu Noble goal complete | https://review.opendev.org/c/openstack/governance/+/945641 ==== Open for Review ==== - [resolution] Extend scope of VMT to cover all project teams | https://review.opendev.org/c/openstack/governance/+/944817 === Upcoming Events === - 2025-04-01 - 2025-04-04: KubeCon+CloudNativeCon EU: https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/ - 2025-04-02: OpenStack 2025.1 ("Epoxy") Release - 2025-04-03: OpenInfra LIVE - Epoxy release [3] - 2025-04-08: OpenInfra Board Meeting: https://board.openinfra.org/ - 2025-04-07 - 2025-04-12: OpenInfra PTG Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] 2025.1 Release tags: https://review.opendev.org/c/openstack/releases/+/945949 [3] Introducing OpenStack 2025.1 Epoxy on OpenInfra Live: https://openinfra.org/live/ [4] OpenInfra Flamingo vPTG: https://ptg.opendev.org/ptg.html [5] "migrate to noble" community goal: https://governance.openstack.org/tc/goals/completed/2025.1/migrate-ci-jobs-… [6] TC Meeting IRC Log 2025-03-25: https://meetings.opendev.org/meetings/tc/2025/tc.2025-03-25-18.00.log.html [7] TC Meeting Agenda 2025-04-01: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[scientific-sig] Wiki update
by Jimmy McArthur 31 Mar '25

31 Mar '25

Hi Scientific Folks! I was trying to direct someone to your meetings, but noticed the wiki page is quite out of date: https://wiki.openstack.org/wiki/Scientific_SIG I know you all are meeting at the PTG, is this something that could be added to the agenda? I’ve got a few orgs that I’d like to send your way. Cheers, Jimmy

1 0

[openstack-helm] Merge openstack-helm and openstack-helm-infra git repos
by Vladimir Kozhukalov 31 Mar '25

31 Mar '25

Hi dear Openstack-Helmers, I would like to suggest merging the two project git repos: https://opendev.org/openstack/openstack-helm-infra.git https://opendev.org/openstack/openstack-helm.git At some point it seemed reasonable to have separate git repos for openstack and infrastructure charts. But now it looks like we are experiencing more cons than pros. When working on some features we are forced to prepare sets of interdependent patches. Some code can be deduplicated. Also when we publish charts we append two commit hashes as package metadata. Also we can reduce the number of test jobs if we have all chats in one repo. Other people also proposed this earlier. So if there are no objections let's make it done. I understand this is going to affect many of our users. Please express your opinions so we don't break any downstream. If we all agree about this, then I would say we can do this in the mid of March/2025 before 2025.2 PTG. The procedure is going to be smth. like 1. Freeze the infra repo and prepare retirement PSs for it 2. Merge openstack-helm-infra repo into openstack-helm ``` git clone https://opendev.org/openstack/openstack-helm.git cd openstack-helm git remote add infra https://opendev.org/openstack/openstack-helm-infra.git git fetch infra git checkout -b infra infra/master git checkout master git merge --allow-unrelated-histories infra # resolve not so many conflicts git review ``` 3. Accept merger PS 4. Update documentation 5. Accept infra retirement PSs (openstack-helm-infra and project-config) -- Best regards, Kozhukalov Vladimir

4 10

[neutron] Bug deputy report March 24 - March 30
by Oleg Bondarev 31 Mar '25

31 Mar '25

Hello Neutron Team, Bug report for the week of March 24 is below: *High:* - https://bugs.launchpad.net/neutron/+bug/2104019 - random failures in L3 HA fullstack failover tests - Confirmed - assigned to Miguel Lavalle - https://bugs.launchpad.net/neutron/+bug/2104134 - ovs/linuxbridge jobs running on ubuntu noble broken with latest kernel 6.8.0-56.58 - Confirmed - unassigned (looks like yatin is tracking it) - https://bugs.launchpad.net/neutron/+bug/2105383 - [OVN] The filter to add the MTU to a LRP changes the options dictionary population - In progress - patch from Rodolfo Alonso: https://review.opendev.org/c/openstack/neutron/+/945905 - https://bugs.launchpad.net/neutron/+bug/2104160 - [DB] Stop expand migrations with field drop exceptions - Confirmed - unassigned ---------- *Low:* - https://bugs.launchpad.net/neutron/+bug/2104927 - [OVN] HA_Chassis_Group creation returns an exception in the logs Edit - In progress - patch from Rodolfo Alonso: https://review.opendev.org/c/openstack/neutron/+/945837 - https://bugs.launchpad.net/neutron/+bug/2104979 - Possible deadlock on closing ip_monitor thread - Triaged - unassigned ---------- *Undecided:* - https://bugs.launchpad.net/neutron/+bug/2104037 - Unable to open stream to ssl::6641 to retrieve schema: Exec format error - Incomplete Thanks, Oleg

1 0

PCI passthrough - flavor not found
by Francesco Di Nucci 31 Mar '25

31 Mar '25

Hi all, I am trying to setup PCI passthrough with Nova, following guides such as this <https://superuser.openinfra.org/articles/a-comprehensive-guide-to-configuri…> and this <https://medium.com/@thomasal14/gpu-passthrough-in-openstack-da2a98a16f7b>: * installed a GPU on a compute node, configured kernel etc and now it is using the vfio-pci driver * set [pci]/device_spec = { "vendor_id": "10de", "product_id": "1ff2" } on compute node (formerly [pci]/passthrough_whitelist) and restarted openstack-nova-* * On the controller node set [pci]/alias: { "vendor_id":"10de", "product_id":"1ff2", "device_type":"type-PCI", "name":"nvidia-t400" }, [filter_scheduler]/enabled_filters = PciPassthroughFilter, [filter_scheduler]/available_filters = nova.scheduler.filters.all_filters and restarted openstack-nova-* * Created a flavor with openstack flavor create --vcpus 4 --ram 8192 --disk 40 --property "pci_passthrough:alias"="nvidia-t400:1" gpu_flavor, it is shown with openstack flavor list * Created an image with --property img_hide_hypervisor_id=true * Tried to create an instance with openstack server create --flavor gpu_flavor --image Almalinux_GPU --key-name "My Key" --network my_network test-gpu (also with --availability-zone nova:my-gpu-node.example.com) * VM creation fails with this errors on the controller node /nova-api.log/ HTTP exception thrown: Flavor gpu_flavor could not be found. /nova-scheduler.log/ Filter PciPassthroughFilter returned 0 hosts Filtering removed all hosts for the request with instance ID '131d5c1c-1927-42c0-bb48-618c05d31c2a'. Filter results: ['PciPassthroughFilter: (start: 19, end: 0)'] /nova-conductor.log/ Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. If I understand correctly it's a cascading error generating from the flavor not found, even though it exists... Anyone has encountered something like this/has suggestions? Am I missing some config? Thanks in advance -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email:francesco.dinucci@na.infn.it

6 10

[nova][manila] How to deploy the new Manila virtio-fs driver?
by Thomas Goirand 31 Mar '25

31 Mar '25

Hi, I've quickly discussed with Shean on IRC, and he told me the new virtio-fs driver for Manila was part of Epoxy. This is very exciting. Thanks to everyone who worked on it. I'd like to deploy it, however, I have no clue how. Note that I already implemented the generic driver using (puppet-manila). But I haven't been able to read any doc about it. Is there anything I should know? Also, is there any pending patches that should get on top of Epoxy? Shean already told me about these 2: https://review.opendev.org/c/openstack/python-openstackclient/+/881540 https://review.opendev.org/c/openstack/openstacksdk/+/880056 It'd be IMO super nice if we could them in Epoxy proper (even though I could manage to sneak them as Debian specific patches into my Debian packages). Cheers, Thomas Goirand (zigo)

2 1

[openstack][magnum][keystone] Fail to create clusters with Openstack 2025.1
by Nguyễn Hữu Khôi 29 Mar '25

29 Mar '25

Hello. I am testing magnum_cluster_api v0.29.3 with OPS 2025.1.but I encounter this error:keystoneauth1.exceptions.http.BadRequest: Invalid input for field/attribute trust. Value: {'roles': [{'id': '00c3da32a3384e298256b4293d7d0b9a', 'name': 'load-balancer_member', 'domain_id': None, 'description': None, 'options': {}, 'links': {'self': ' https://k.com:5000/v3/roles/00c3da32a3384e298256b4293d7d0b9a'}}, {'id': '32e8b1f999ae4d2f8d50ffd5a7065e38', 'name': 'manager', 'domain_id': None, 'description': None, 'options': {'immutable': True}, 'links': {'self': ' https://k.com:5000/v3/roles/32e8b1f999ae4d2f8d50ffd5a7065e38'}}, {'id': '4dc61b50010540c4a0e614a1fb23d1ef', 'name': 'admin', 'domain_id': None, 'description': None, 'options': {'immutable': True}, 'links': {'self': ' https://k.com:5000/v3/roles/4dc61b50010540c4a0e614a1fb23d1ef'}}, {'id': '2d722357d74f4c39aab4bc18b5cbe391', 'name': 'member', 'domain_id': None, 'description': None, 'options': {'immutable': True}, 'links': {'self': ' https://k.com:5000/v3/roles/2d722357d74f4c39aab4bc18b5cbe391'}}, {'id': 'efa3d956f74e41b6aa999262d456063b', 'name': 'heat_stack_owner', 'domain_id': None, 'description': None, 'options': {}, 'links': {'self': ' https://k.com:5000/v3/roles/efa3d956f74e41b6aa999262d456063b'}}, {'id': '30b6b07e181e414099a79d0d83cec4a1', 'name': 'reader', 'domain_id': None, 'description': None, 'options': {'immutable': True}, 'links': {'self': ' https://k.com:5000/v3/roles/30b6b07e181e414099a79d0d83cec4a1'}}], 'delegation_depth': 0, 'id': '65fc7dbe686c4efca6f8501ca2b0f4c9', 'trustor_user_id': '36e0dee5ff45410e89ee85c403131b92', 'trustee_user_id': 'b3a05e45b78b4ad1bfd537c84f2ebfa2', 'project_id': '16d98639e9154413843795fd1a4a9bcb', 'impersonation': True, 'expires_at': None, 'remaining_uses': None, 'deleted_at': None, 'redelegated_trust_id': None, 'redelegation_count': 0, 'roles_links': {'self': ' https://k.com:5000/v3/65fc7dbe686c4efca6f8501ca2b0f4c9/roles', 'next': None, 'previous': None}, 'links': {'self': ' https://k.com:5000/v3/OS-TRUST/trusts/65fc7dbe686c4efca6f8501ca2b0f4c9'}}. Additional properties are not allowed ('delegation_depth' was unexpected) (HTTP 400) (Request-ID: req-7520e8ee-8073-4164-b023-4f353721a6c5) 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/magnum/conductor/handlers/common/trust_manager.py", line 34, in create_trustee_and_trust 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server trust = osc.keystone().create_trust( 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/magnum/common/keystone.py", line 218, in create_trust 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server raise exception.TrustCreateFailed( 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server magnum.common.exception.TrustCreateFailed: Failed to create trust for trustee b3a05e45b78b4ad1bfd537c84f2ebfa2. 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/rpc/server.py", line 174, in _process_incoming 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server ^^^^^^^^^^^^^^^^^^^^^^ 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/osprofiler/profiler.py", line 159, in wrapper 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server ^^^^^^^^^^^^^^^^^^ 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/magnum/conductor/handlers/cluster_conductor.py", line 66, in cluster_create 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server trust_manager.create_trustee_and_trust(osc, cluster) 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.12/site-packages/magnum/conductor/handlers/common/trust_manager.py", line 42, in create_trustee_and_trust 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server raise exception.TrusteeOrTrustToClusterFailed( 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.server magnum.common.exception.TrusteeOrTrustToClusterFailed: Failed to create trustee or trust for Cluster: 65eec76d-a811-420a-8d9f-7cefdd050f35 2025-03-29 21:03:54.879 1081 ERROR oslo_messaging.rpc.serverCould I have some ideas for this? Thank you. [image: white_check_mark][image: eyes][image: raised_hands] NEW Nguyen Huu Khoi

2 1

[openstack-ansible]: openstack-ansible upgrade from Yoga to Zed
by Rambo Rambo 28 Mar '25

28 Mar '25

Hi Team We are planning openstack-ansible upgrade from Yoga to Zed. Distribution:openstack-ansible Release: Yoga OS - Ubuntu 20.04 I was checking the release notes and can see below point related to neutron linux bridge. We are having linux bridge at the moment. https://docs.openstack.org/releasenotes/openstack-ansible/zed.html =========================================================== - With marking ML2/LinuxBridge driver as ‘Experimental’ in the upstream Neutron project OpenStack-Ansible has switched a default mechanism driver to ML2/OVN. In order to upgrade any existing deployment that was relying on defaults to the new OpenStack-Ansible version you must ensure that following variables are defined explicitly to ensure parity with existing functionality: neutron_plugin_type: ml2.lxbneutron_ml2_drivers_type: "flat,vlan,vxlan,local"neutron_plugin_base: - router - metering Failure to define any of these variables will result in playbook failures and neutron misconfiguration. We have covered this step with upgrade script that will create a user_neutron_migration.yml file with assumed defaults. =================================================== As per above point we need to add new parameters into user_variables file . also it is written "We have covered this step with upgrade script" *Current Configuration:* /etc/neutron/plugins/ml2# *cat ml2_conf.ini* [ml2] type_drivers = flat,vlan,vxlan,local tenant_network_types = vxlan,flat *mechanism_drivers = linuxbridge,l2population* # ML2 flat networks extension_drivers = port_security,subnet_dns_publish_fixed_ip *user_variable file* only has below related parameters : neutron_l2_population: true neutron_plugin_base: - router - neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin neutron_ml2_conf_ini_overrides: ml2: extension_drivers: - port_security - subnet_dns_publish_fixed_ip Can someone please confirm what should be done to take care of above point of the release notes? Regards

2 1

[neutron] Next two weeks CI meeting cancelled
by Yatin Karel 28 Mar '25

28 Mar '25

Hello Neutron Team !! As communicated in last week's CI meeting i will not be around upcoming Monday, so cancelling the meeting. Also proposing to cancel the follow up week as that's PTG week. So we will have our next CI meeting on 14th April. Thanks and Regards Yatin Karel

1 0

[release] Release countdown for week R+0
by Thierry Carrez 28 Mar '25

28 Mar '25

Development Focus ----------------- We will be releasing the coordinated OpenStack 2025.1 Epoxy release next week, on April 2nd, 2025. Thanks to everyone involved in the 2025.1 Epoxy cycle! We are now in pre-release freeze, so no new deliverable will be created until final release, unless a release-critical regression is spotted. Otherwise, teams attending the virtual PTG should start to plan what they will be discussing there, by creating and filling team etherpads. You can access the list of PTG etherpads at: https://ptg.opendev.org/etherpads.html General Information ------------------- On release day, the release team will produce final versions of deliverables following the cycle-with-rc release model, by re-tagging the commit used for the last RC. A patch doing just that will be proposed soon. PTLs and release liaisons should watch for that final release patch from the release team. While not required, we would appreciate having an ack from each team before we approve it on Wednesday, so that their approval is included in the metadata that goes onto the signed tag. Upcoming Deadlines & Dates -------------------------- Final 2025.1 Epoxy release: April 2nd, 2025 Project Teams Gathering (virtual): April 7th-11th, 2025 -- Thierry Carrez (ttx)

1 0

Fwd: PCI passthrough - flavor not found
by Francesco Di Nucci 28 Mar '25

28 Mar '25

Thanks, forwarding to the list as information; tried it and still got the same error -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email: francesco.dinucci(a)na.infn.it -------- Forwarded Message -------- Subject: Re: PCI passthrough - flavor not found Date: Fri, 28 Mar 2025 11:10:25 +0200 From: Mike Currin <mike(a)idia.ac.za> To: Francesco Di Nucci <francesco.dinucci(a)na.infn.it> Hi, Forgot to reply to the list as well so ignore the one specifically to you, it's the same content though Regards Mike On Fri, Mar 28, 2025 at 11:08 AM Mike Currin <mike(a)idia.ac.za> wrote: Hi Francesco, We had something similar for our A100 GPUs, we had to add "device_type":"type-PF" vs PCI and then it worked. Maybe give that a go. I've seen it mentioned on the list before. Regards Mike On Fri, Mar 28, 2025 at 11:00 AM Francesco Di Nucci <francesco.dinucci(a)na.infn.it> wrote: Hi all, I am trying to setup PCI passthrough with Nova, following guides such as this <https://superuser.openinfra.org/articles/a-comprehensive-guide-to-configuri…> and this <https://medium.com/@thomasal14/gpu-passthrough-in-openstack-da2a98a16f7b>: * installed a GPU on a compute node, configured kernel etc and now it is using the vfio-pci driver * set [pci]/device_spec = { "vendor_id": "10de", "product_id": "1ff2" } on compute node (formerly [pci]/passthrough_whitelist) and restarted openstack-nova-* * On the controller node set [pci]/alias: { "vendor_id":"10de", "product_id":"1ff2", "device_type":"type-PCI", "name":"nvidia-t400" }, [filter_scheduler]/enabled_filters = PciPassthroughFilter, [filter_scheduler]/available_filters = nova.scheduler.filters.all_filters and restarted openstack-nova-* * Created a flavor with openstack flavor create --vcpus 4 --ram 8192 --disk 40 --property "pci_passthrough:alias"="nvidia-t400:1" gpu_flavor, it is shown with openstack flavor list * Created an image with --property img_hide_hypervisor_id=true * Tried to create an instance with openstack server create --flavor gpu_flavor --image Almalinux_GPU --key-name "My Key" --network my_network test-gpu (also with --availability-zone nova:my-gpu-node.example.com <http://my-gpu-node.example.com>) * VM creation fails with this errors on the controller node /nova-api.log/ HTTP exception thrown: Flavor gpu_flavor could not be found. /nova-scheduler.log/ Filter PciPassthroughFilter returned 0 hosts Filtering removed all hosts for the request with instance ID '131d5c1c-1927-42c0-bb48-618c05d31c2a'. Filter results: ['PciPassthroughFilter: (start: 19, end: 0)'] /nova-conductor.log/ Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. If I understand correctly it's a cascading error generating from the flavor not found, even though it exists... Anyone has encountered something like this/has suggestions? Am I missing some config? Thanks in advance -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email:francesco.dinucci@na.infn.it -- Mike Currin email: mike(a)idia.ac.za Cloud Computing System Support Inter-University Institute for Data Intensive Astronomy (IDIA) University of Cape Town

1 0

[all][qa] QA tools (devstack/grenade/tempest) setup for new stable/2025.1 and current master
by Ghanshyam Mann 28 Mar '25

28 Mar '25

Hi All, The QA tools setup[1] for the new stable/2015.1 and the current master is completed. I would like to highlight a couple of things: * Recheck if devstack/gernade jobs on master or any backport were not running/testing correctly. * By default, grenade-skip-level-always is added in integrated gate templates as voting. As the current master is non-SLURP, Testing N-2 -> N upgrade is not mandatory, so the project can opt to make it non-voting or remove it from their integrated template[2]. [1] https://review.opendev.org/q/topic:%22qa-2025-1-release%22 [2] https://github.com/openstack/tempest/blob/f4a8698b97a5433b8e64df408aace2fb1… -gmann

1 0

[openstack-helm] health-probes timeout for neutron agents (l3 and dhcp) and nova compute
by Alberto Molina Coballes 27 Mar '25

27 Mar '25

Hi, We’re facing an issue when deploying OpenStack 2024.1 with OpenStack Helm and OVS, the health probes fail for neutron-l3-agent, neutron-dhcp-agent and nova-compute while the internal RPC communication between the nova and neutron components look fine: Neutron: openstack network agent list +--------------------------------------+--------------------+--------------------------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------------------------+-------------------+-------+-------+---------------------------+ | 433c2b07-5c82-4545-991d-6d0de6044ae9 | DHCP agent | fig-virt-intdev-alberto-node-0 | nova | :-) | UP | neutron-dhcp-agent | | 56c050fa-e58e-46a6-8191-0d0d481de246 | Open vSwitch agent | fig-virt-intdev-alberto-node-0 | None | :-) | UP | neutron-openvswitch-agent | | 5ea28388-3fac-4ebb-bce4-6b210f8a3766 | Metadata agent | fig-virt-intdev-alberto-node-0 | None | :-) | UP | neutron-metadata-agent | | 6a4506ca-6fe4-489a-b370-6e426db40d50 | DHCP agent | fig-virt-intdev-alberto-node-1 | nova | :-) | UP | neutron-dhcp-agent | | 772a1d3a-b432-4ac3-8dfe-d3affc6f226d | Metadata agent | fig-virt-intdev-alberto-node-1 | None | :-) | UP | neutron-metadata-agent | | 7bd8360c-468a-4473-a470-abfb1ab435da | Open vSwitch agent | fig-virt-intdev-alberto-node-1 | None | :-) | UP | neutron-openvswitch-agent | | 9d362a63-5740-44ed-ac77-4548eee59205 | L3 agent | fig-virt-intdev-alberto-node-0 | nova | :-) | UP | neutron-l3-agent | | a2cc9dc4-1600-4c5d-a250-2e43cf062cf3 | L3 agent | fig-virt-intdev-alberto-node-1 | nova | :-) | UP | neutron-l3-agent | +--------------------------------------+--------------------+--------------------------------+-------------------+-------+-------+---------------------------+ The neutron-l3 and neutron-dhcp pods are not ready: k -n openstack get pods -l application=neutron |grep 'l3\|dhcp' neutron-dhcp-agent-default-fw5rm 0/1 Running 0 127m neutron-dhcp-agent-default-vh7c4 0/1 Running 0 127m neutron-l3-agent-default-4h5tp 0/1 Running 0 24m neutron-l3-agent-default-bdngv 0/1 Running 0 24m Because the health probes fail: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Unhealthy 4m38s (x119 over 127m) kubelet Readiness probe failed: Health probe timed out. Agent is down or response timed out When trying to execute the health probe manually from the container, the probe fails with a timeout after 60 seconds (RPC_PROBE_TIMEOUT=60) neutron@fig-virt-intdev-alberto-node-1:/$ python /tmp/health-probe.py --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/dhcp_agent.ini --agent-queue-name dhcp_agent --use-fqdn Health probe timed out. Agent is down or response timed out The situation for nova is quite similar: openstack compute service list +--------------------------------------+----------------+---------------------------------+----------+---------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +--------------------------------------+----------------+---------------------------------+----------+---------+-------+----------------------------+ | eaf4684c-88a4-47dd-b340-b7dbebf148ba | nova-conductor | nova-conductor-679d57f977-zbq92 | internal | enabled | up | 2025-03-27T13:11:33.000000 | | 9164d63f-608c-44cc-8a92-2f2bac07c622 | nova-scheduler | nova-scheduler-845f87b9b5-vq9p5 | internal | enabled | up | 2025-03-27T13:11:36.000000 | | 464cd394-8b26-49ad-8c82-f014ca00ec7e | nova-compute | fig-virt-intdev-alberto-node-1 | nova | enabled | up | 2025-03-27T13:11:33.000000 | | aa7e5f6d-2533-4953-bb8d-9f9bfb7fc6bd | nova-compute | fig-virt-intdev-alberto-node-0 | nova | enabled | up | 2025-03-27T13:11:33.000000 | +--------------------------------------+----------------+---------------------------------+----------+---------+-------+----------------------------+ And the nova-compute pods are not ready because the health probes fail: k -n openstack get pods -l component=compute NAME READY STATUS RESTARTS AGE nova-compute-default-t8w7x 1/2 Running 1 (70m ago) 3h31m nova-compute-default-vz7nf 1/2 Running 1 (70m ago) 3h31m To be sure that the OpenStack RPC is working fine, the health probes have been deleted for nova-compute, then nova-cell-setup is executed and the hypervisors are ready. Some VMs have been launched in this environment successfully (nova-compute), getting IP from the DHCP agent or adding floating IPs (l3-agent) and the OpenStack behaviour is normal. So I’m confused about the health-probes failing. Any idea on what can be wrong or what to check is welcome. Thanks! Alberto Note: This is a Kubernetes cluster running on two VMs (OpenStack). This virtual environment has been redeployed several times with some modifications and the issue is fully reproducible: the probes for nova-compute, neutron-l3-agent and neutron-dhcp-agent always fail.

1 0

[Glance]Cancelling Weekly Meeting (March 27)
by Mridula Joshi 27 Mar '25

27 Mar '25

Hello, Cancelling today's weekly meeting, As most of the members are not around. Apologies for the last min update. Thanks, Mridula Joshi

1 0

[designate] multiple zone catalogs with multiple zones
by Benedikt Trefzer 27 Mar '25

27 Mar '25

Hi List I'm trying to create an additional pool with a separate catalog zone. For the default pool, I can do an AXFR to the catalog domain, this works (either without tsig key or with a tsig). For the catalog of the second pool cannot be queried (Log sais Zone not found). In the DB I can see the zone. Questions: - Is it possible to have multiple pools with separate catalogs, does anyone have a running setup ? - Reading this [1] line in the code, I assume It's not possible having multiple zones without using tsig, correct ? Thanks a lot. Benedikt [1] https://opendev.org/openstack/designate/src/branch/master/designate/mdns/ha…

1 1

Nova features
by engineer2024 26 Mar '25

26 Mar '25

Where can I find the nova feature requests which are yet to be developed ? I am thinking of contributing to adding new features.

3 3

[largescale-sig] Next meeting: March 26, 16utc
by Thierry Carrez 26 Mar '25

26 Mar '25

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 16UTC to facilitate America timezones participation. In these troubled DST times, you should doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20250326T16 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

2 1

#eventlet-removal Flamingo PTG
by Herve Beraud 26 Mar '25

26 Mar '25

Hello folks, We’re meeting again on the Project Teams Gathering in April (7-11th). The slot for the Eventlet Removal Initiative PTG is: Tuesday (Apr 8) 1400-1600 UTC The etherpad is: https://etherpad.opendev.org/p/apr2025-ptg-eventlet Please feel free to insert your topic proposals. Additional details can be found at https://ptg.opendev.org/ Regards, -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[release][ptl][tc] OpenInfra Live for OpenStack 2025.1 ‘Epoxy’
by helena＠openinfra.dev 26 Mar '25

26 Mar '25

Hi everyone, Following the release of OpenStack 2025.1 ‘Epoxy’ next Wednesday, we'd like to go ahead and schedule the OpenInfra Live for Thursday, April 3 at 1400 UTC. If you would like to represent your project team’s updates in the Epoxy release as well as their plans for Flamingo, please let me know. If you are not familiar, you can find reference examples like 2024.2 Dalmation [1] and 2024.1 Caracal [2]. [1] https://www.youtube.com/watch?v=6igJNIJ9yFE [2] https://www.youtube.com/watch?v=3BgAXbFaVqw&t=1707s Thanks! Helena helena(a)openinfra.dev Community Programs Manager The OpenInfra Foundation [openinfraLogo-sm.png]

3 2

[neutron][ops][upgrade] l3-agent failure during upgrade to Zed
by Eugen Block 25 Mar '25

25 Mar '25

Hi *, maybe I missed some announcement or something, but usually, I read the release notes [0] before upgrading our OpenStack cloud. I didn't notice anything regarding DB schema upgrades. And after the upgrade from Yoga to Zed in a test environment went well, I tried the same in our production today. Note that I didn't have a router in my test cloud, so that's probably why I didn't notice anything. Unfortunately, there has been a schema change, that's why the l3-agent failed to start properly with this error: 2025-03-21 12:29:14.527 846393 CRITICAL neutron [None req-e225ff0a-82e1-473b-9eba-9a11caa7ace7 - - - - - -] Unhandled error: oslo_messaging.rpc.client.RemoteError: Remote error: OperationalError (pymysql.err.OperationalError) (1054, "Unknown column 'portforwardings.external_port' in 'SELECT'") Indeed, the upgraded control node didn't have "external_port" anymore in /usr/lib/python3/dist-packages/neutron/db/models/port_forwarding.py, while the not yet upgraded control node did. So the situation could only be resolved by proceeding with the upgrade. But that meant an interruption for our virtual routers, causing floating IPs to be unreachable for a couple of minutes. Note that we're using highly-available routers. I thought about setting "no-ha" for each router, but that can only be done for disabled routers, which is not an option, of course. And it doesn't really fit into the "rolling upgrade" concept, which has worked great so far. Since we moved to Ubuntu last September (while still on Victoria), we've been able to upgrade to Yoga without any issues. And while the interruption today was not too critical, I was still surprised that such an important change didn't even make it into the Zed release notes. Was that a mistake or did I miss something? Are there other places I need to check before attempting an upgrade? Thanks, Eugen [0] https://docs.openstack.org/releasenotes/neutron/zed.html

3 9

[dev] New contributing organization help
by Dominik Danelski 25 Mar '25

25 Mar '25

Hello community, I work at a silver member of OpenInfra and will make code contributions on behalf of my employer. I am aware about the requirement to sign Contributor License Agreement listing me as the responsible person. Other than that, are there any additional requirements on identifying me as a corporate developer? For example contributor guide lists creating INDIVIDUAL Member account as one of the setup steps and it got me a bit worried. Kind regards Dominik Danelski @ CloudFerro

4 3

[kolla] Flamingo PTG
by Michał Nasiadka 25 Mar '25

25 Mar '25

Hello Koalas, We’re meeting again on the Project Teams Gathering in April (7-11th). The slots for Kolla/Kolla-Ansible/Kayobe PTG are: Tuesday - 07:00-08:00 UTC (general, Kolla and Kolla-Ansible) Wednesday - 07:00-08:00 UTC (general, Kolla and Kolla-Ansible) Friday - 14:00-15:00 UTC (Kayobe) If there’s a need of an additional afternoon (UTC) slot - please let us know. The ether pad is at the usual location: https://etherpad.opendev.org/p/kolla-flamingo-ptg Please insert your topic proposals and see you on the PTG. Additional details can be found at https://ptg.opendev.org/ Cheers, Michal

1 0

[publiccloud-sig] Next meeting of Public Cloud SIG upcoming - March 26th at 0800 UTC
by Felix Kronlage-Dammers 25 Mar '25

25 Mar '25

Hi everyone, This upcoming Wednesday (March 26th) the next meeting of the publiccloud-sig is going to happen. We’ll be meeting at 0800 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG I'll not be around but Artem (gtema) will be hosting the call. Thanks Artem! cheers felix -- Felix Kronlage-Dammers Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[openstack][rabbitmq]computes cannot register to nova 2025.0master
by Nguyễn Hữu Khôi 25 Mar '25

25 Mar '25

Hello. I deployed OpenStack 2025.1 Master with Kolla-Ansible, but the compute nodes cannot register with Nova. Error: Cannot update service status on host "compute05" due to an unexpected exception.: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 3dad7dbe8ea9429c83958b98bccc7094 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver Traceback (most recent call last): 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/virt/libvirt/driver.py", line 5687, in _set_host_enabled 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver service = objects.Service.get_by_compute_host(ctx, CONF.host) 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_versionedobjects/base.py", line 175, in wrapper 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver result = cls.indirection_api.object_class_action_versions( 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/nova/conductor/rpcapi.py", line 240, in object_class_action_versions 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver return cctxt.call(context, 'object_class_action_versions', 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/rpc/client.py", line 190, in call 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver result = self.transport._send( 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/transport.py", line 123, in _send 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver return self._driver.send(target, ctxt, message, 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 800, in send 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver return self._send(target, ctxt, message, wait_for_reply, timeout, 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 789, in _send 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver result = self._waiter.wait(msg_id, timeout, 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in wait 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver message = self.waiters.get(msg_id, timeout=timeout) 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver File "/var/lib/kolla/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 520, in get 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver raise oslo_messaging.MessagingTimeout( 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 3dad7dbe8ea9429c83958b98bccc7094 2025-03-23 10:31:26.974 7 ERROR nova.virt.libvirt.driver All services are in a "down" state, but this issue only occurs when using a three-controller cluster. When I use a single controller, everything works fine. My Env: OS: Ubuntu 22.04 OpenStack Version: 2025.1 Master Deployment Tool: Kolla-Ansible 19 Thank you. Nguyen Huu Khoi

3 7

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-1)
by Goutham Pacha Ravi 25 Mar '25

25 Mar '25

Hello Stackers, We're one week away from the coordinated release of OpenStack 2025.1 ("Epoxy") [1]. Project teams shipped release candidates over the past week and will keep these updated if any release-critical bugs are identified. Teams are also preparing for the virtual OpenInfra Project Teams Gathering (PTG) [2], which begins on April 7. OpenStack project teams are encouraged to schedule cross-project meetings during the first two days. The OpenStack Technical Committee will meet with contributors to chart out our tasks to aid the next release, 2025.2 "Flamingo." The TC sessions are scheduled at the beginning and end of the PTG. We hope to discuss governance, community goals, and other work affecting the entire community. Please find the etherpad of the OpenStack TC's discussions [3], and feel free to propose new topics! In the past week, the TC appointed its chair (me!) and vice-chair (Dmitriy Rabotyagov / noonedeadpunk). We thank the members of the TC and the community for their continued faith in our work. Additionally, a proposal has been made to change the Vulnerability Management Team's (VMT) work, which will affect all project teams [4]. The proposal aims to place all project teams under VMT's oversight. Your feedback on this proposal is welcome! === Weekly Meeting === The OpenStack TC met on OFTC's #openstack-tc channel for its weekly meeting on 2025-03-18 [5]. We began with a discussion around the former openstack-operators mailing list. This mailing list was merged with the openstack-discuss mailing list after extensive prior discussion. The TC received some feedback in recent months from operators who wished to revive a dedicated list for operator discussions. Some operators feel project maintainers don’t engage with their problems, and discussions now occur on other platforms. Mailing list engagement among operators is still present but could be improved. There will be an Operators Discussion at the PTG, and the TC will raise this concern to better understand operator feedback. The TC appreciated the work project teams did in conjunction with the release team to get release candidates posted across the board. We reviewed the state of the CI and welcomed the donation of cloud resources that aided in Zuul's scalability around this critical time of the release. The next meeting of the OpenStack Technical Committee is today, 2025-03-25. It will be held at 18:00 UTC in the #openstack-tc channel on OFTC. Please find the agenda on the meeting's wiki page [6]. I hope you'll be able to join us. === Governance Proposals === ==== Merged ==== - Add 2025.2 TC chair and vice-chair | https://review.opendev.org/c/openstack/governance/+/944174 - Add separate freezer-api deliverable | https://review.opendev.org/c/openstack/governance/+/944781 ==== Open for Review ==== - [resolution] Extend scope of VMT to cover all project teams | https://review.opendev.org/c/openstack/governance/+/944817 - Add Cinder Huawei charm | https://review.opendev.org/c/openstack/governance/+/867588 === Upcoming Events === - 2025-04-01 - 2025-04-04: KubeCon+CloudNativeCon EU: https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/ - 2025-04-02: OpenStack 2025.1 ("Epoxy") Release - 2025-04-03: OpenInfra LIVE - Epoxy release - 2025-04-08: OpenInfra Board Meeting: https://board.openinfra.org/ - 2025-04-07 - 2025-04-12: OpenInfra PTG Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] 2025.2 "Flamingo" PTG: https://openinfra.org/ptg/ [3] TC PTG Etherpad: https://etherpad.opendev.org/p/apr2025-ptg-os-tc [4] VMT proposal: https://review.opendev.org/c/openstack/governance/+/944817 [5] TC Meeting IRC Log 2025-03-18: https://meetings.opendev.org/meetings/tc/2025/tc.2025-03-18-18.00.log.html [6] TC Meeting Agenda, 2025-03-25: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

Fwd: [Bug 2103448] Re: Extending `hosts` to also accept new `target_hosts` variable
by Dominic Dumrauf 24 Mar '25

24 Mar '25

Hi all, Could I interest someone in reviewing the *bifrost* - *bug* https://bugs.launchpad.net/bifrost/+bug/2103448 and its - *proposed solution* https://review.opendev.org/c/openstack/bifrost/+/944810 ? Many thanks and feel free to let me know how I can help. Cheers, Dominic ---------- Forwarded message --------- From: Afonne-CID <2103448(a)bugs.launchpad.net> Date: Wed, Mar 19, 2025 at 4:30 PM Subject: [Bug 2103448] Re: Extending `hosts` to also accept new `target_hosts` variable To: <dominic.dumrauf.git(a)gmail.com> Thanks for the bug and also proposing a change. You can get some eyes on your proposed change by asking about it in the #openstack-bifrost channel on IRC (if you haven't already). I personally think it LGTM. On the change, instead of "Bug #2103448: ", use 'Closes-Bug: #2103448' so that it directly references this report and gets automatically tracked and closed by the bot once resolved. ** Changed in: bifrost Status: New => Triaged ** Changed in: bifrost Importance: Undecided => Medium ** Changed in: bifrost Assignee: (unassigned) => Dominic Dumrauf (dumrauf) -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/2103448 Title: Extending `hosts` to also accept new `target_hosts` variable Status in Bifrost: Triaged Bug description: Many thanks for providing a single `install.yaml` playbook which installs `bifrost` in one go. Great work! It seems the hosts are fixed to target in the playbook. Would it be possible to make the playbook work with other inventories by introducing a new variable `target_hosts` as in PR https://review.opendev.org/c/openstack/bifrost/+/944810 ? The proposed change should be backwards compatible. To manage notifications about this bug go to: https://bugs.launchpad.net/bifrost/+bug/2103448/+subscriptions

1 0

[neutron] Bug deputy report for week 12 of 2025
by Jakub Libosvar 24 Mar '25

24 Mar '25

Hi all, I was the deputy for the last week. There are two bugs I left in “Undecided" that might not be in Neutron and need deeper understanding of what is going on. Undecided --------- - Race condition - Error creating instance if network is deleted by different process https://bugs.launchpad.net/neutron/+bug/2103500 It needs further investigation as it is unknown if the problem is with Neutron or Nova - Open vSwitch (Version 3.3.0) goes into deadlocked state https://bugs.launchpad.net/neutron/+bug/2103641 Based on the description of the bug it sounds like a problem with OVS rather than Neutron. High ---- - [FT] Random error in ``test_ovn_sb_sync_host_with_other_agent_type_not_deleted`` https://bugs.launchpad.net/neutron/+bug/2103424 Needs an assignee Wishlist -------- - [RFE][OVN] A L3 router tool https://bugs.launchpad.net/neutron/+bug/2103521

1 0

[heat][ptg] Topics for PTG April 2025
by Takashi Kajinami 24 Mar '25

24 Mar '25

Hi, I started updating the etherpard[1] for the heat sessions in upcoming PTG. If you are interested in joining the sessions then please add your name and your time zone. Also, please add any topics you want to discuss during these sessions. I'll reserve the slots based on these inputs, likely next week. [1] https://etherpad.opendev.org/p/apr2025-ptg-heat -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

1 0

[openstack][rabbitmq]rabbitmq is broken
by Nguyễn Hữu Khôi 23 Mar '25

23 Mar '25

Hello. I am using rabbitmq quorum queue in my system. After I reboot the controllers, Now it has this debug message and my system runs under error. "repairing leader record" Nguyen Huu Khoi

2 3

[mistral][heat][watcher][aodh] croniter being abandoned
by Thierry Carrez 22 Mar '25

22 Mar '25

Hi everyone, It was brought to my attention that the croniter maintainer has decided to abandon his project, due to not wanting to deal with his perceived exposure to the EU Cyber-Resilience act (CRA). https://pypi.org/project/croniter/#disclaimer The maintainer indicates that you should no longer use it and it may be "unpublished" any time after March 15, 2025. Croniter is being used in at least Mistral, Heat, Watcher, and Aodh as a dependency. While this is clearly an overreaction (the CRA only applies to "economic operators" putting "products with digital elements" on the market "in the course of a commercial activity"), we should probably prepare to migrate away from this dependency, or maintain a fork of it. -- Thierry Carrez (ttx)

5 8

[openstack-ansible] Weird install failure
by Sarah Thompson 22 Mar '25

22 Mar '25

Hi all, I think I'm close to getting things working at this point, but I'm seeing a weird install failure during openstack.osa.setup_openstack: FAILED - RETRYING: [ostk-controller-heat-api-container-7d1e8599 -> ostk-controller-utility-container-db5068d8]: Add service users to roles (1 retries left).Result was: attempts: 5 censored: 'the output has been hidden due to the fact that ''no_log: true'' was specified for this result' changed: false retries: 6 Using module file /etc/ansible/ansible_collections/openstack/cloud/plugins/modules/role_assignment.py Pipelining is enabled. <ostk-controller> ESTABLISH SSH CONNECTION FOR USER: root <ostk-controller> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=300) <ostk-controller> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no) <ostk-controller> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=22) <ostk-controller> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no) <ostk-controller> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root") <ostk-controller> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=5) <ostk-controller> SSH: Set ssh_extra_args: (-o)(UserKnownHostsFile=/dev/null)(-o)(StrictHostKeyChecking=no)(-o)(ServerAliveInterval=64)(-o)(ServerAliveCountMax=1024)(-o)(Compression=no)(-o)(TCPKeepAlive=yes)(-o)(VerifyHostKeyDNS=no)(-o)(ForwardX11=no)(-o)(ForwardAgent=yes)(-T) <ostk-controller> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath="/root/.ansible/cp/d3dfd6cdfc") <ostk-controller> SSH: EXEC ssh -vvvvv -C -o ControlMaster=auto -o ControlPersist=300 -o StrictHostKeyChecking=no -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=5 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ServerAliveInterval=64 -o ServerAliveCountMax=1024 -o Compression=no -o TCPKeepAlive=yes -o VerifyHostKeyDNS=no -o ForwardX11=no -o ForwardAgent=yes -T -o 'ControlPath="/root/.ansible/cp/d3dfd6cdfc"' ostk-controller 'sudo lxc-attach --clear-env --name ostk-controller-utility-container-db5068d8 -- su - root -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'/openstack/venvs/utility-30.1.0.dev46/bin/python && sleep 0'"'"'"'"'"'"'"'"''"'"'' <ostk-controller> rc=1, stdout and stderr censored due to no log <ostk-controller> Failed to connect to the host via ssh: <error censored due to no log> failed: [ostk-controller-heat-api-container-7d1e8599 -> ostk-controller-utility-container-db5068d8(192.168.100.117)] (item=None) => attempts: 5 censored: 'the output has been hidden due to the fact that ''no_log: true'' was specified for this result' changed: false fatal: [ostk-controller-heat-api-container-7d1e8599 -> {{ _service_setup_host }}]: FAILED! => censored: 'the output has been hidden due to the fact that ''no_log: true'' was specified for this result' changed: false These logs came from running ansible with -vvvvv as a parameter. Looking at the failed ssh, this root causes pretty straightforwardly: ssh -vvvvv -C -o ControlMaster=auto -o ControlPersist=300 -o StrictHostKeyChecking=no -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=5 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ServerAliveInterval=64 -o ServerAliveCountMax=1024 -o Compression=no -o TCPKeepAlive=yes -o VerifyHostKeyDNS=no -o ForwardX11=no -o ForwardAgent=yes -T -o 'ControlPath="/root/.ansible/cp/d3dfd6cdfc"' ostk-controller 'sudo lxc-attach --clear-env --name ostk-controller-utility-container-db5068d8 -- su - root -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'/openstack/venvs/utility-30.1.0.dev46/bin/python && sleep 0'"'"'"'"'"'"'"'"''"'"'' If I manually ssh to ostk-controller and lxc-attach -n ostk-controller-utility-container-db5068d8, then run the command explicitly: root@ostk-controller-utility-container-db5068d8:~# /openstack/venvs/utility-30.1.0.dev46/bin/python && sleep 0 Python 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> KeyboardInterrupt >>> exit() So basically it's invoking an interactive python session, which of course times out, and after 5 retries the installation aborts. Something is obviously missing somewhere, most likely a parameter after the python command which specifies a python script to be run. Any ideas? Thanks all, Sarah

2 1

[cinder][nova] force detach volume
by Arnaud Morin 22 Mar '25

22 Mar '25

Hey team, Is there any way to force detach a volume when the nova compute service is down? Like when the hypervisor is off, we want to detach the volume and attach it to another instance. Maybe I'm missing the way to do it, but detach action on nova side seems a synchronous task that I can't force without admin creds. Thanks for any help, Regards, Arnaud

4 6

[release] Release countdown for week R-1, Mar 24 - 28
by Elõd Illés 21 Mar '25

21 Mar '25

(re-sent this mail with the right week number in subject, sry :S) Development Focus ----------------- We are on the final mile of the 2025.1 Epoxy development cycle! Remember that the 2025.1 Epoxy final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. March 28th, 2025 is the deadline for final 2025.1 Epoxy release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on April 2nd, 2025. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the 2025.2 Flamingo development cycle, including discussing Forum and PTG sessions content, in preparation of Project Teams Gathering (virtual), April 7th-11th, 2025. Actions ------- Watch for any translation patches coming through on the stable/2025.1 branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/2025.1 branch before triggering a new release. Please drop by #openstack-release with any questions or concerns about the upcoming release ! Upcoming Deadlines & Dates -------------------------- Final 2025.1 Epoxy release: April 2nd, 2025 Project Teams Gathering (virtual): April 7th-11th, 2025 Előd Illés irc: elodilles @ #openstack-release

1 0

[release] Release countdown for week R-4, Mar 24 - 28
by Elõd Illés 21 Mar '25

21 Mar '25

Development Focus ----------------- We are on the final mile of the 2025.1 Epoxy development cycle! Remember that the 2025.1 Epoxy final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. March 28th, 2025 is the deadline for final 2025.1 Epoxy release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on April 2nd, 2025. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the 2025.2 Flamingo development cycle, including discussing Forum and PTG sessions content, in preparation of Project Teams Gathering (virtual), April 7th-11th, 2025. Actions ------- Watch for any translation patches coming through on the stable/2025.1 branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/2025.1 branch before triggering a new release. Please drop by #openstack-release with any questions or concerns about the upcoming release ! Upcoming Deadlines & Dates -------------------------- Final 2025.1 Epoxy release: April 2nd, 2025 Project Teams Gathering (virtual): April 7th-11th, 2025 Előd Illés irc: elodilles @ #openstack-release

1 0

[neutron] Drivers meeting canceled Friday, March 21st
by Brian Haley 21 Mar '25

21 Mar '25

Hello Neutrinos: I'm going to cancel tomorrow's drivers meeting as I have a conflict and cannot attend. Let's just add any new RFE items to the agenda for the upcoming PTG [0] - I'll try and group them all together in our Thursday meeting slots if possible. Just a reminder there are still open specs that need review, list is at [1], would be good to get them merged. Have a nice weekend! -Brian [0] https://etherpad.opendev.org/p/apr2025-ptg-neutron [1] https://review.opendev.org/q/status:open+project:openstack/neutron-specs

1 0

[ironic] [operators] Inspector service deprecated; scope assumed by Ironic
by Jay Faulkner 19 Mar '25

19 Mar '25

OpenStack developers and deployers, In 2014, Dmitry Tantsur created a project to allow deployers of Ironic to inspect nodes and discover and enroll new nodes -- ironic-inspector. This project has become a staple, deployed alongside other Ironic services in most common deployments. Over the last couple of years, we've worked on -- and now completed -- a migration all existing ironic-inspector functionality into the primary Ironic service. As a result of that, the service/project at openstack/ironic-inspector and it's client at openstack/python-ironic-inspector-client have both been in maintenance mode since 2024. Users of Ironic should not expect future releases of ironic-inspector past this SLURP release of OpenStack 2025.1. If you have any issues migrating to the in-tree inspection functionality in Ironic, please reach out to the Ironic team on the mailing list, in IRC, or wherever else you may find us. Thanks, Jay Faulkner Ironic Security Liason

1 0

[neutron][l3 agent] pymysql.err.IntegrityError (Duplicate entry) while updating router information
by Viacheslav Klenov 19 Mar '25

19 Mar '25

Hello, We ran into a strange issue with neutron l3_agent after failed live-migration, which failed due to pyroute2/iproute2 version incompatibility issue During live-migration l3 agent starts to update router information but then logs a lot of errors 2025-03-13 12:28:18.041 43002 ERROR neutron.agent.l3.agent [-] Failed to fetch router information for '4b89508c-beaf-43cf-92a6-9e080c03554d': oslo_messaging.rpc.client.RemoteError: Remote error: PendingRollbackError This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (pymysql.err.IntegrityError) (1062, "Duplicate entry '506ac36d-4e5c-47a1-96ca-36f739018dfe-hv-017' for key 'PRIMARY'") [SQL: INSERT INTO ml2_distributed_port_bindings (port_id, host, router_id, vif_type, vif_details, vnic_type, profile, status) VALUES (%(port_id)s, %(host)s, %(router_id)s, %(vif_type)s, %(vif_details)s, %(vnic_type)s, %(profile)s, %(status)s)] [parameters: {'port_id': '506ac36d-4e5c-47a1-96ca-36f739018dfe', 'host': 'hv-017', 'router_id': '4b89508c-beaf-43cf-92a6-9e080c03554d', 'vif_type': 'unbound', 'vif_details': '', 'vnic_type': 'normal', 'profile': '', 'status': 'DOWN'}] (log attached) Before the migration there is no records in neutron ml2_distributed_port_bindings table with a hostname from the error after the errors the migrated VM is seemingly functional The error reproduces every time on live-migration even after pyroute2/iproute2 incompatibility was fixed Any insight into why is this happening or a way to fix it would be greatly appreciated -- Viacheslav ----------------- Viacheslav Klenov vklenov(a)sardinasystems.com | www.sardinasystems.com | @sardinasystems | https://www.linkedin.com/company/sardina-systems "We build technology so that others can build more technology.”

2 2

Guidance on Deploying Octavia with VXLAN in Nested OpenStack Setup
by Thamanna Farhath 19 Mar '25

19 Mar '25

Dear OpenStack Community, I am currently using OpenStack 2023.1 and looking to deploy Octavia using VXLAN networking. My setup consists one parent openstack environment, and within this, I have an nested OpenStack deployment in which I want to configure Octavia with VXLAN. I would appreciate any guidance on the following: Network Configuration: Best practices for setting up VXLAN tunnels in a nested OpenStack environment. Octavia Deployment: Steps or references for installing Octavia with VXLAN networking. Integration with Neutron: Any specific settings required to ensure proper communication between the parent network and the nested OpenStack deployment. Reference Documentation: Any official or community-driven guides that can help with this setup. If anyone has experience with a similar deployment or has documentation to refer to, I would be grateful for your insights. Thank you in advance for your help! Associate Engineer-R&D Thamanna Farhath N Cn: (+91) 9344093591 , thamanna.f(a)zybisys.com Zybisys Consulting LLP | Srinivasa Tower' No.29, 39th E Cross, 4th T Block, Jayanagar , Bangalore India – 560041 http://zybisys.com/ Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

1 0

Spice, NoVNC and TLS
by Jani Heikkinen 19 Mar '25

19 Mar '25

Dear Openstack community, I would be happy to hear if anybody has configured Spice or noVNC console with TLS. Is this possible with using spicehtml5proxy? As far I know, it is easy to set up the TLS until the server running spiche5html(or noVNC). And according to Websockets documentation, it will only establish itself over existing http connection. How the proxy actually forms this connection? Is it possible to pass TLS certificates to it? My requirement would also be, that self-signed certificates are no-go. Best, and thanks, Jani -- Berner Fachhochschule / Bern University of Applied Sciences IT-Services / Team Linux & Infrastructure Services Jani Heikkinen IT Linux Engineer ___________________________________________________________ Dammweg 3, CH-3013 Bern Telefon direkt +41 31 848 68 14 Telefon Servicedesk +41 31 848 48 48 jani.heikkinen(a)bfh.ch

4 5

MAC address associated with the FIP (Floating IP) appears as "INCOMPLETE"
by ussefelboukhari＠gmail.com 19 Mar '25

19 Mar '25

Hello ! I hope this message finds you well. I am reaching out to share a problem I’m currently facing, and I would greatly appreciate your assistance or any suggestions you might have. I deployed a bare metal EC2 instance on AWS to set up an all-in-one deployment using Kolla Ansible, but I’ve encountered some issues, which are as follows: Initially, we noticed that our instances were unable to access the metadata stored on nova-compute to retrieve essential information such as the private IP address and SSH keys. For example, when running the `ip a` command on an instance, the private IP address did not appear. After some research, we discovered that this bug was already known. The solution we implemented was to use OVN instead of OVS. This change allowed our instances to access the metadata on nova-compute and retrieve the private IP addresses. However, there is still an anomaly: when we run the `arp -n` command on the host machine, the MAC address associated with the FIP (Floating IP) appears as "INCOMPLETE". I would be grateful if you could provide any insights or suggestions to help resolve this issue. Thank you in advance for your support!

1 0

[cinder]Need some help for merging the bug fixing commits
by hao wang 18 Mar '25

18 Mar '25

Hi, guys It is possible that get your help reviewing and merging those two bug-fixing commits? They have been there for a long time and have discussed for many rounds. If there are no further concerns, could we merge it in the new cycle? Thanks a lot. [1]https://review.opendev.org/c/openstack/cinder/+/740146 [2]https://review.opendev.org/c/openstack/cinder/+/734415 Regards, Hao Wang -- Best Wishes To You!

1 0

[administrivia] Recent change in moderation for new subscribers
by Jeremy Stanley 18 Mar '25

18 Mar '25

Due to a spate of spammers subscribing to openstack-discuss over the last several days in order to send off-topic messages to the list, I've altered the default behavior for posts from new subscribers. For now, if someone subscribes, their initial post to the list will be held for moderation in order for our moderators to make sure it's a legitimate post, before setting their subscription to bypass moderation for any subsequent posts they make. Note that this only affects new subscriptions as of now, anyone who has already subscribed to the list before now should see no change in how things work for them. As for the spam to the list for the past few days, I've blocked those subscribers from sending any new messages and deleted their off-topic posts from the public archive. Please let me know if you have any questions. -- Jeremy Stanley

1 0

[nova][cinder][glance] Support for rbd namespaces
by Eugen Block 18 Mar '25

18 Mar '25

Hi, I've been searching and testing for a while, but I haven't found anything helpful yet. Someone in the Ceph community brought up the question about rbd namespaces for nova, cinder and glance. It would make sense to use only one pool for those services with regards to PG numbers. Having multiple pools means to manage their pg_num individually, and scaling can be limited due to the max number of PGs per OSD. Having only one pool would make that easier. I mean, we've been using a single pool for those three services from the beginning, but without a permission segregation. Is there anything on the road map to support rbd namespaces? Or has there been something and it was decided not to go down that road? I'd appreciate any insights! Thanks! Eugen

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-2)
by Goutham Pacha Ravi 18 Mar '25

18 Mar '25

Hello Stackers, We're two weeks away from the release date for the 2025.1 ("Epoxy") coordinated release [1]. OpenStack project teams worked with the release team to make the first release candidate (RC) available for wider testing. Over the next couple of weeks, any release-critical bugs may require revised RCs. Now's a good time to review release highlights [2] that project teams have added to reflect major changes made during this release. Development focus has shifted to the next release, 2025.2 "Flamingo" [3]. Last week was eventful. The OpenInfra Foundation board unanimously approved resolutions that further its move to become a sub-foundation of the Linux Foundation [4]. For many of you who are members of the foundation, you will have an opportunity to ratify the board's decisions through a vote next month. In the past week, the OpenStack Technical Committee didn't merge any new governance updates; however, several are open for community review. === Weekly Meeting === The TC's weekly meeting in OFTC's #openstack-tc channel on 2025-03-11 was well attended [5]. We were joined by several attendees of the OpenInfra Days North America event to discuss key takeaways and gather action items for the TC. There was broad acknowledgment that many presentations and discussions at the event revolved around Running AI workloads on OpenStack and aiding the migration of enterprises that are transitioning to OpenStack from existing commercial/proprietary solutions We expressed interest in participating in working group discussions at the upcoming PTG on both these topics. Additionally, we revisited community pain points that were brainstormed at the event. Two notable issues were: - The need for an official presence on communication platforms beyond OFTC/IRC - The lack of sufficient engagement from operators on the openstack-discuss mailing list No immediate actions were taken on these issues during the meeting, but both topics generated significant discussion. We expect to gather input from the wider community during the upcoming PTG. The next OpenStack TC meeting is today, 2025-03-18, at 1800 UTC on OFTC's #openstack-tc channel. Please find the agenda on the meeting’s wiki page [6]. I hope you'll be able to join us. I'd encourage the community to bring any topics of interest to this meeting. The synchronous attention of the TC and the lively discussions that follow are a great way to demonstrate our commitment to OpenStack’s "Open Community" values. === Governance Proposals === ==== Open for Review ==== - [resolution] Extend scope of VMT to cover all project teams | https://review.opendev.org/c/openstack/governance/+/944817 - Add separate freezer-api deliverable | https://review.opendev.org/c/openstack/governance/+/944781 - Fix links to chair responsibilities in TC Guide | https://review.opendev.org/c/openstack/governance/+/944802 - Add Cinder Huawei charm | https://review.opendev.org/c/openstack/governance/+/867588 - Resolve to adhere to non-biased language | https://review.opendev.org/c/openstack/governance/+/934907 === Upcoming Events === 2025-04-01 – 2025-04-04: KubeCon+CloudNativeCon EU: https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/ 2025-04-02: OpenStack 2025.1 ("Epoxy") Release 2025-04-08: OpenInfra Board Meeting: https://board.openinfra.org/ 2025-04-07 – 2025-04-12: OpenInfra PTG === How to Contact the TC === You can reach the TC in several ways: - Email: Send an email with the [tc] tag on this mailing list. - IRC: Ping us using the tc-members keyword in the #openstack-tc IRC channel on OFTC. - Weekly Meetings: The Technical Committee meets every Tuesday at 1800 UTC [6]. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] 2025.1 "Epoxy" Release Highlights: https://releases.openstack.org/epoxy/highlights.html [3] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [4] OIF Board March '25 Meeting Recap: https://lists.openinfra.org/archives/list/foundation@lists.openinfra.org/th… [5] TC Meeting IRC Log 2025-03-11: https://meetings.opendev.org/meetings/tc/2025/tc.2025-03-11-18.00.log.html [6] TC Meeting Agenda, 2025-03-18: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[ovs] [neutron] Egress traffic is not reaching table 71
by yardalgedal＠gmail.com 18 Mar '25

18 Mar '25

Hi, all! We are experimenting with bluefield-2 and neutron-openvswith-agent on it and have a big problem – all our egress traffic (from stateless point of view) are not leaving the bluefield. Symptoms are the next: the baremetal to which bluefield-2 is connected successfully responds on ping, but does not return any response for TCP traffic and can not send anything from inside. We have some security group with fully allowed egress rules. +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ | ID | IP Protocol | Ethertype | IP Range | Port Range | Direction | Remote Security Group | Remote Address Group | +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ | 14604165-e0c1-4bce-ae21-d20bf4165fa2 | None | IPv6 | ::/0 | | ingress | None | None | | 16f3f44f-1c9a-48f1-bcd0-7ff8df0cd30a | None | IPv4 | 0.0.0.0/0 | | ingress | None | None | | c7054cde-72ce-4dc8-a4e0-361df199204e | None | IPv6 | ::/0 | | egress | None | None | | d8bc8f82-f181-443d-a0ca-67cbae78b47f | None | IPv4 | 0.0.0.0/0 | | egress | None | None | +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ and port to which it is assigned has port_security_enabled: True (if we are disabling port security, there is no problem anymore, it is actual only for enabled port security) we have disabled DVR and VxLAN tunnel here, we have native openvswitch firewall also [agent] tunnel_types = l2_population = true arp_responder = true extensions = qos baremetal_smartnic = True [securitygroup] firewall_driver = openvswitch [ovs] bridge_mappings = br-ex:br-int datapath_type = system ovsdb_connection = tcp:127.0.0.1:6640 #local_ip = tunnel_bridge = of_inactivity_probe = 60 ovsdb_timeout = 60 when we're sending some traffic, all flows in table 71 (BASE_EGRESS_TABLE) has n_packets = 0, so they are not reaching it. in table 60 (TRANSIENT_TABLE), which should send traffic to the table 71, on flow which should do it we can see n_packets=0 as well cookie=0x6cc97f12f173f538, duration=2086.129s, table=60, n_packets=0, n_bytes=0, priority=100,in_port=pf0hpf actions=load:0x2b->NXM_NX_REG5[],load:0x1->NXM_NX_REG6[],resubmit(,71) at the same time, the counter for NORMAL flow is growing, so packets are probably going to here (not sure cookie=0x6cc97f12f173f538, duration=2126.547s, table=60, n_packets=139340, n_bytes=6817502, priority=3 actions=NORMAL on trace we can see that the packet was directly sent to port 1 instead of going via the table ovs-appctl ofproto/trace br-int 'dl_src=a0:88:c2:eb:15:4e,dl_dst=00:1c:73:00:57:ff,ip,nw_dst=DST_IP,nw_src=SRC_IP,in_port=43,nw_ttl=64,nw_proto=6,tp_dst=80' Flow: tcp,in_port=43,vlan_tci=0x0000,dl_src=a0:88:c2:eb:15:4e,dl_dst=00:1c:73:00:57:ff,nw_src=SRC_IP,nw_dst=DST_IP,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,tp_src=0,tp_dst=80,tcp_flags=0 bridge("br-int") ---------------- 0. priority 0, cookie 0x699a137a0a73d6a8 NORMAL -> forwarding to learned port bridge("br-int") ---------------- 0. in_port=5,dl_vlan=1, priority 4, cookie 0x699a137a0a73d6a8 set_field:4297->vlan_vid NORMAL -> forwarding to learned port Final flow: unchanged Megaflow: recirc_id=0,eth,ip,in_port=43,dl_src=a0:88:c2:eb:15:4e,dl_dst=00:1c:73:00:57:ff,nw_frag=no Datapath actions: push_vlan(vid=201,pcp=0),1 any ideas why it happens or any proposals for future debugging? Thank you in advance!

1 0

Controller node openstack LXC containers clashing on IP subnets
by Sarah Thompson 17 Mar '25

17 Mar '25

Hi all, I've got much further now, and am now tracking down networking issues. I'm pretty sure everything is now installing, but I'm seeing a systematic issue. My test network that I'm running the openstack instances on is 10.0.1.x -- the VMs all have fixed IP addresses, can talk to each other, etc, nothing weird going on. haproxy is installing and running, but throwing 503 errors. Digging into this, it seems that there are some issues with the network configs of at least some of the LXC containers. The one I'm seeing that's preventing Ansible from completing the infrastructure setup is the repo server container. If I attach to the container, the keepalive comes back correctly. Externally to the container, the HTTP connection is rejected. Looking into the reasons, it looks like there are 3 networks visible from inside the container: ens18: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.1.38 netmask 255.255.255.0 broadcast 10.0.1.255 inet6 fe80::216:3eff:feab:ab45 prefixlen 64 scopeid 0x20<link> ether 00:16:3e:ab:ab:45 txqueuelen 1000 (Ethernet) RX packets 96 bytes 6952 (6.9 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 16 bytes 1236 (1.2 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.3.113 netmask 255.255.255.0 broadcast 10.0.3.255 inet6 fe80::216:3eff:fe45:64d6 prefixlen 64 scopeid 0x20<link> ether 00:16:3e:45:64:d6 txqueuelen 1000 (Ethernet) RX packets 351 bytes 425285 (425.2 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 263 bytes 19114 (19.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 2939 bytes 208134 (208.1 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 2939 bytes 208134 (208.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 The 10.0.1.38 address seems to be the problem. I think this is an internally routed subnet, not the actual physical subnet (note the lack of a gateway address, and 10.0.1.38 is definitely not being allocated via the DCHP server). Looking at some other containers, I'm seeing 10.0.2.x and 10.0.3.x there, so obviously this is being allocated on the fly either by lxc or the container(s). TL;DR: The internal 10.0.1.x is clashing with the physical 10.0.1.x network, which is almost certainly why the keepalive is failing. Does anyone have any idea how to fix the configuration to use some other CIDR block for this? I'd like to avoid the extreme pain of remapping my physical network (this is a way more complicated problem than a few test nodes unfortunotely!) Thank you in advance, Sarah Thompson -- [s]

3 4

PCI alias rtx is not defined
by Felipe Mogollon 17 Mar '25

17 Mar '25

Dear all, I am suffering a problem in my OpenStack servers. I have a 3 server configuration: - controller + compute - 2 x compute nodes The controller node has an Nvidia L4 configured in vGPU mode. One of the compute nodes has another Nvidia L4 in vGPU mode. Both servers are running fine, and I can deploy instances with vGPU capabilities. The problem comes with my 2nd compute node; that node has a Nvidia RTX4000 that can only work in passthrough mode. , I am pretty sure that in past times I managed to deploy this compute node with an Nvidia RTX 4000 in passthrough mode as a compute node of another OpenStack server configuration, but nowadays I am not able to use that GPU in passthrough mode and I don't know why: - I have configured IOMMU in compute node - I have configured nova.conf with pci alias and whitelist in compute node - I have configured nova.conf with pci alias and enabled filters in controller node - I have created a flavor and set up pci_passthrough:rtx:1 to it's configuration As far as I know, those are the steps to configure a GPU passthrough in a compute node in a Openstack multi-node configuration... But when I try to deploy a new instance I am always getting a "PCI alias rtx is not defined (HTTP 400)" error. Any ideas about what could be the problem? Is there any incompatibility with vGPU and passthrough working modes? Thanks in advance. Felipe -- [image: Vicomtech] <https://www.vicomtech.org> Industry Division Juan Felipe Mogollón Rodríguez Senior Researcher Digital Media and Communications fmogollon(a)vicomtech.org +(34) 943 30 92 30 The information contained in this electronic message is intended only for the personal and confidential use of the recipients. If you have received this e-mail by mistake, please, notify us and delete it. Avoid printing this message if it is not strictly necessary.

1 0

Routed provider networks
by Alvin Starr 17 Mar '25

17 Mar '25

I have a couple of small flat networks that I am using for a POC project. I am trying to use the routed provider networks features to create a single public net. I delete the old networks and try to install the following network and it looks like it works in that I see the 2 subnets as part of the single public-net. The trouble is that I cannot create a router with an interface in the public-net. Is there anything special that I need to do to create a router on the public-net? I changed the addresses so the subnets are likely wrong. I am installing with openstack-ansible using stable/2023.2 onto Centos-Stream. # Create the parent network with first physical segment PUBNET_ID=$(openstack network create --share \ --provider-physical-network pub204 \ --provider-network-type flat \ --external \ public-net \ -f value -c id ) FIRST_SEG=$(openstack network segment list --network ${PUBNET_ID} -f value -c ID) openstack network segment set --name segment-pub204 ${FIRST_SEG} # Add second physical segment to the network openstack network segment create \ --physical-network pub203 \ --network-type flat \ --network public-net \ segment-pub203 # Create subnet for pub203 segment openstack subnet create --network public-net \ --network-segment segment-pub203 \ --ip-version 4 \ --subnet-range 1.2.50.192/28 \ --allocation-pool start=1.2.50.196,end=1.2.50.206 \ --gateway 1.2.50.193 \ --dhcp \ public-subnet203 # Create subnet for pub204 segment openstack subnet create --network public-net \ --ip-version 4 \ --network-segment segment-pub204 \ --subnet-range 1.2.48.128/29 \ --allocation-pool start=1.2.48.132,end=1.2.48.134 \ --gateway 1.2.48.129 \ --dhcp \ public-subnet204 -- Alvin Starr || land: (647)478-6285 Netvel Inc. || home: (905)513-7688 alvin(a)netvel.net ||

1 0

[openstack-ansible]Playbooks failing after upgrading lxc containers from ubuntu18 to ubuntu20 due to python version mismatch.
by Danish Khan 17 Mar '25

17 Mar '25

Dear All, We are upgrading one of our openstack cluster from ussuri to wallaby and for this we are first upgrading OS of lxc container from ubuntu to 18 to 20(hosts OS is also upgraded to 20). We are destroying LXC containers(which don't form quorum like nova,neutron,heat,cinder,horizon) of one infra node and then creating it again. The new containers are getting created with Ubuntu 20 as the host OS is 20. Playbook for cinder, nova neutron were completed successfully for new containers of ubuntu 20. But Playbook is failing for heat, placement and memcached due to python version mismatch: fatal: [host5_heat_api_container-25f90e52]: FAILED! => {"attempts": 5, "changed": false, "cmd": ["/openstack/venvs/heat-21.2.5/bin/pip3", "install", "-U", "--constraint", "/openstack/venvs/heat-21.2.5/global-constraints.txt", "--constraint", "/openstack/venvs/heat-21.2.5/constraints.txt", "--pre", "--log", "/var/log/python_venv_build.log", "cryptography", "keystonemiddleware", "openstack-heat", "osprofiler", "pymemcache", "PyMySQL", "python-ceilometerclient", "python-glanceclient", "python-heatclient", "python-keystoneclient", "python-memcached", "python-neutronclient", "python-novaclient", "python-openstackclient", "python-swiftclient", "python-troveclient", "systemd-python"], "msg": "stdout: Ignoring python-saharaclient: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring python-saharaclient: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring python-saharaclient: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring oslo.limit: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring oslo.limit: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring oslo.limit: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring sphinxcontrib-actdiag: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring sphinxcontrib-actdiag: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring sphinxcontrib-actdiag: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring os-api-ref: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring os-api-ref: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring os-api-ref: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring oslo.concurrency: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring oslo.concurrency: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring oslo.concurrency: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring osprofiler: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring osprofiler: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring osprofiler: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring os-resource-classes: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring os-resource-classes: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring os-resource-classes: markers I understand that 2.7 version is depricated but where should make changes to fix it? https://docs.openstack.org/releasenotes/python-heatclient/ussuri.html I tried updating few variable to fix python version like: /etc/ansible/roles/os_heat/defaults/main.yml #heat_venv_python_executable: "{{ openstack_venv_python_executable | default('python3') }}" updated it from python2 to python 3 Tried few more variables but that didn't work. Any idea how we can fix this version mismatch issue ? Regards, Danish

2 1

[Devstack] Trying to setup development environment on VM
by Ishan Shanware (ishanwar) 17 Mar '25

17 Mar '25

Hello Everyone, I am currently trying to setup in all-in-one development environment for OpenStack on a Debian VM. The VM has 16 GB RAM and 4 CPUs. However, I face an error while trying to run FORCE=yes ./stack.sh. I get an error that keystone does not start. This causes the entire process to fail. Requesting assistance in solving this issue. I am following the this guide: https://docs.openstack.org/devstack/latest/guides/single-machine.html PFA the error log and error message on the terminal. [Call Trace] ./stack.sh:1126:start_keystone /opt/stack/devstack/lib/keystone:549:die [ERROR] /opt/stack/devstack/lib/keystone:549 keystone did not start Error on exit World dumping... see /opt/stack/logs/worlddump-2024-11-18-150808.txt for details ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) Thanks in advance, Ishan

3 5

[neutron] Bug deputy report for week 11 of 2025
by Miguel Lavalle 16 Mar '25

16 Mar '25

I was the Neutron bug deputy last week. Following is a brief summary of the reported bugs: High bugs -------------- https://bugs.launchpad.net/neutron/+bug/2101839 [fullstack] Random errors during the execution of ``wait_until_true``. Assignee Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/943942 https://bugs.launchpad.net/neutron/+bug/2101857 ``neutron-tempest-plugin-ovn-ubuntu-jammy`` job unstable. No owner. Jobs neutron-tempest-plugin-*-ubuntu-jammy`` were made no voting: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/944119 Medium bugs ------------------ https://bugs.launchpad.net/neutron/+bug/2101840 ovs jobs randomly failing with guest vms not getting ip from dhcp. Assignee Lajos Katona https://bugs.launchpad.net/neutron/+bug/2101998 [OVN][Placement] Deleting a configured resource provider does not remove the Placement entry. Assignee Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/944053 https://bugs.launchpad.net/neutron/+bug/2102083 ``SingletonDecorator`` class to allow destroying instances. Assignee Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/944099 https://bugs.launchpad.net/neutron/+bug/2101871 Use of revision_number for router update on 2023.2 Bobcat cause error 500. Waiting for reproducer info from submitter Low bugs ------------- https://bugs.launchpad.net/neutron/+bug/2102095 [OVN][Placement] Change the message level when a Placement exception happens. Assignee Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/944101 RFEs ----- https://bugs.launchpad.net/neutron/+bug/2102090 [RFE][OVN][Placement] With Neutron API WSGI module, the initial chassis configuration is not executed. https://bugs.launchpad.net/neutron/+bug/2102165 [RFE] Forbid usage of certain IP addresses in subnet pools https://bugs.launchpad.net/neutron/+bug/2102184 [RFE] Project-Specific QoS Controls for Granular Resource Management Incomplete --------------- https://bugs.launchpad.net/neutron/+bug/2102044 Nova-API problem - cannot list instances anymore

1 0

[ci][all] Migrating doc job back to ubuntu jammy
by Takashi Kajinami 16 Mar '25

16 Mar '25

Hi, I noticed that doc job in taskflow repo was broken recently. Example: https://review.opendev.org/c/openstack/taskflow/+/944495?tab=change-view-ta… Digging into the failure, it seems the failure is caused by api doc build, trying to parse a module importing asyncore, because asyncore is no longer available in Python 3.12. IIUC Doc job was recently migrated from Ubuntu Jammy to Ubuntu Noble, which means that the default python version was bumped to 3.12. We are aware of the deprecation/removal of asyncore and already deprecated the implementation in taskflow. However deprecation was merged during 2024.2 cycle so we can't remove it until 2025.2 cycle to follow SLURP policy. There might be a way to drop the asyncore-related implementations from docs to unblock the doc job, I'm not sure if that is the right approach given the fact the implementation is deprecated but is still valid in older python versions. Thus, I wonder if we can keep running doc job in Jammy for 2025.1, which still supports Ubuntu Jammy, and then migrate the job again during 2025.2 . An alternative approach is to remove the asyncore implementation during this cycle, breaking SLURP policy, but I'm not sure if this can be the preferred option. Thank you, Takashi -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 4

Deploiment in 2025
by Albert Shih 16 Mar '25

16 Mar '25

Hi everyone Long time ago I deploy a openstack mostly manually, then I integrated in our deployment tool (puppet). Now I need to re-install everything. I'm searching the «best» method to do that. My deployment would be on bare-metal, I don't need/want a deployment of CEPH (I already have my ceph cluster). Just need the cinder to use the ceph. The main focus is the easy way to upgrade. I don't have a lot of hardware so I can afford to have a «test» cluster for upgrading. I'm thinking of kolla, what do you think ? I see they are now openstack-helm project, which If I understand correctly use a k8s to manage the deployment of openstack. Is that correct ? With those tools is they are any limitation about the result ? Or can I got the most the main features of openstack ? What would you recommend ? Regards. -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 05 mars 2025 17:13:54 CET

8 16

[all][tc][ptl][qa] Migrating upstream CI/CD jobs to Ubuntu Noble (24.04)
by Ghanshyam Mann 14 Mar '25

14 Mar '25

Hello Everyone, As per the 2025.2 cycle testing runtime, we need to run upstream CI/CD jobs on Ubuntu Noble (24.04) [1]. For better tracking and speeding up this work, it is the community-wide goal for this cycle[2]. This is the etherpad which has the migration plan and will be used for the tracking: - https://etherpad.opendev.org/p/migrate-to-noble I have prepared the change(WIP) for base jobs migration on Noble (openstack tox jobs, devstack, and tempest) which will migrate most of the projects jobs. Tempest change[3] include dependency on openstack tox, devstack base jobs changes so testing project gate using the the tempest change will test the complete set. Refer to the etherpad for the complete migration plan/steps. * Deadline: Nov 29th 2024 (R18) is the deadline to test/fix the projects jobs and to merge the base jobs running on Noble (openstck tox, devstack, and tempest). [1] https://governance.openstack.org/tc/reference/runtimes/2025.1.html [2] https://governance.openstack.org/tc/goals/selected/migrate-ci-jobs-to-ubunt… [3] https://review.opendev.org/c/openstack/tempest/+/932156 -gmann

2 9

[release] Release countdown for week R-2
by Thierry Carrez 14 Mar '25

14 Mar '25

Development Focus ----------------- At this point we should have release candidates (RC1 or recent intermediary release) for all the 2025.1 Epoxy deliverables. Teams should be working on any release-critical bugs that would require another RC or intermediary release before the final release. Actions ------- Early in the week, the release team will be proposing stable/2025.1 branch creation for all deliverables that have not branched yet, using the latest available 2025.1 Epoxy release as the branch point. If your team is ready to go for creating that branch, please let us know by leaving a +1 on these patches. If you would like to wait for another release before branching, you can -1 the patch and update it later in the week with the new release you would like to use. By the end of the week the release team will merge those patches though, unless an exception is granted. Once stable/2025.1 branches are created, if a release-critical bug is detected, you will need to fix the issue in the master branch first, then backport the fix to the stable/2025.1 branch before releasing out of the stable/2025.1 branch. After all of the cycle-with-rc projects have branched we will branch devstack, grenade, and the requirements repos. This will effectively open them up for 2025.2 Flamingo development, though the focus should still be on finishing up 2025.1 Epoxy until the final release. Opening up the new development cycle is also a good opportunity to teams to revise their zuul jobs and used nodesets and clean up all unnecessary ones. For projects with translations, watch for any translation patches coming through and merge them quickly. A new release should be produced so that translations are included in the final 2025.1 Epoxy release. Finally, now is a good time to finalize release notes. In particular, consider adding any relevant "prelude" content. Release notes are targeted for the downstream consumers of your project, so it would be great to include any useful information for those that are going to pick up and use or deploy the 2025.1 Epoxy version of your project. Upcoming Deadlines & Dates -------------------------- Final RC deadline: March 28th, 2025 (R-1 week) Final 2025.1 Epoxy release: April 2nd, 2025 Project Teams Gathering (virtual): April 7th-11th, 2025 -- Thierry Carrez (ttx)

1 0

[neutron] Drivers meeting canceled Friday, March 14th
by Brian Haley 14 Mar '25

14 Mar '25

Hello Neutrinos: I'm going to cancel tomorrow's drivers meeting as there is currently nothing listed in the agenda and we should be focusing on getting last-minute changes for RC1 merged. Just a reminder there are still open specs that need review, list is at [0]. Have a nice weekend! -Brian [0] https://review.opendev.org/q/status:open+project:openstack/neutron-specs

1 0

weighing compute nodes by real memory available when filtering
by Jiatong Shen 14 Mar '25

14 Mar '25

Hello Experts, I would like to ask if there is an available implementation to achieve filtering or weighing compute nodes by the real memory usage. From my observation, nova filters the compute-nodes by computing the summation of all instances' flavor ram and reserved ram. But since reserved ram is only an empirical value, sometimes compute nodes could potentially use more ram. Making things even worse is the ram allocation ratio could be more than 1. So I would like to know if there is an available implementation such that when the real memory available (from procfs) is low, the compute-node could be given a smaller weight or simply filtered out. Thank you. -- Best Regards, Jiatong Shen

2 2

Building a test cluster with openstack ansible
by Sarah Thompson 13 Mar '25

13 Mar '25

Hi all, I've been experimenting with openstack through a devstack instance, but I need to start looking at a more realistic deployment architecture. I have a 5 node Proxmox cluster available, so I've created 6 VMs, one to serve as a deployment host for Ansible etc, one as a controller node, and four compute nodes. These are all currently set up with Ubuntu 22 LTS and configured with 128GB of disk, 20GB of RAM and 12 cores. I've verified that Ansible works, LXC works correctly, etc. I'm getting a failure when I'm running openstack-ansible openstack.osa.setup_infrastructure Specifically I get errors like: PLAY [Gather repo facts] ******************************************************************************************************* TASK [Gathering Facts] ********************************************************************************************************* [WARNING]: Unhandled error in Python interpreter discovery for host openstack-controller-repo-container-e44299d5: unexpected output from Python interpreter discovery fatal: [openstack-controller-repo-container-e44299d5]: FAILED! => ansible_facts: {} changed: false failed_modules: ansible.legacy.setup: ansible_facts: discovered_interpreter_python: /usr/bin/python3 failed: true module_stderr: |- lxc-attach: openstack-controller-repo-container-e44299d5: attach.c: get_attach_context: 405 Connection refused - Failed to get init pid lxc-attach: openstack-controller-repo-container-e44299d5: attach.c: lxc_attach: 1469 Connection refused - Failed to get attach context module_stdout: '' msg: |- MODULE FAILURE See stdout/stderr for the exact error rc: 1 warnings: - Platform unknown on host openstack-controller-repo-container-e44299d5 is using the discovered Python interpreter at /usr/bin/python3, but future installation of another Python interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-core/2.17/reference_appendices/interpreter… for more information. msg: |- The following modules failed to execute: ansible.legacy.setup Inspecting the openstack-controller node manually with lxc-ls, there are no containers present. (I have manually created one and tested it successfully, so it's definitely not something weird with not being able to create/run containers). Thank you in advance for your insights. :-) Sarah PS: When I have this working, next step is to put this on bare metal in a colo, assuming I can get it working.

3 8

[infra][tc][trove] Freeing the reddwarf project on PyPI
by Jeremy Stanley 13 Mar '25

13 Mar '25

I've received a request from a representative of The Computational Democracy Project (pol.is) who have a project named red-dwarf and would like to release software to PyPI under that name. For those of you who have been around long enough to remember, the Trove project was called Red Dwarf during incubation, but was renamed roughly 12 years ago owing to trademark concerns. The https://pypi.org/p/reddwarf project was reserved but no release files were uploaded to it. I don't expect our community has any reason to continue holding onto that name as I don't see that we would ever actually use it, but I'm posting this for the sake of transparency. Does anyone object to letting go of the reddwarf project on PyPI so that it can be reused by another open source community effort? -- Jeremy Stanley

2 3

[kolla] [nova] API flags ignored in Wallaby
by Albert Braden 12 Mar '25

12 Mar '25

After rebuilding our Train clusters on Wallaby we are seeing some API issues. It appears that flags that work in Train are still documented in Wallaby, but are ignored. Do we have something setup wrong, or are we encountering bugs? The first thing is the “locked” list: Train: openstack server list --all-projects --locked --os-compute-api-version 2.73 <empty list because nothing is locked> openstack server list --all-projects --unlocked --os-compute-api-version 2.73 <list of every VM in the cluster> Wallaby (locked flag is ignored): openstack server list --all-projects --locked --os-compute-api-version 2.73 <list of every VM in the cluster> openstack server list --all-projects --unlocked --os-compute-api-version 2.73 <list of every VM in the cluster> The second problem is the hypervisor list. We can’t get “long” output in Wallaby: Train: openstack hypervisor list --long | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | vCPUs Used | vCPUs | Memory MB Used | Memory MB | Wallaby (long flag is ignored): openstack hypervisor list --long | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |

6 14

[keystone] Keystone tries to get project by selecting user_id
by İzzettin Erdem 12 Mar '25

12 Mar '25

Hi, I faced to an odd issue; while migrating some servers, keystone threw an exception and migration aborted - Could not find project: 6008439893ca4d80ab933cf3b0b5fef0 When I checked, a project with this ID does not exist, but the key point is it's not a project ID, it's an ID of a user. I googled this problem but couldn't find more information except this [1] <https://bugs.launchpad.net/keystone/+bug/2062247>. PS: _ mysql logs were fine, didn't see weird queries _ keystone works fine _ openstack version kolla-ansible/train Do you have any ideas? Thanks, İzzettin

2 1

[all] Important discussion about the OpenInfra Foundation's future
by Jeremy Stanley 12 Mar '25

12 Mar '25

The OpenInfra Foundation's board of directors is considering a significant organizational change, and your feedback is critical to assist them in their decision. Please read this announcement on the foundation mailing list: https://lists.openinfra.org/archives/list/foundation@lists.openinfra.org/th… You can follow up publicly on that list with any questions or input you have for the board and staff, or in one of the public feedback conference calls mentioned there, as well as privately to Jonathan Bryce or myself if you prefer to give feedback in confidence. Thanks as always for your interest in the future of our community! -- Jeremy Stanley -- Jeremy Stanley

4 11

[ovn][neutron] ha_chassis make priority permanent
by Satish Patel 12 Mar '25

12 Mar '25

Folks, I am dealing with a very strange DHCP issue because of the priority of ha_chassis. I have the following ha_chassis group and I want os-chn-gen-comp-001 and os-chn-gen-comp-002 to always service DHCP for sriov interface. I have used following command to give them high priority $ ovn-nbctl --no-leader-only set ha_chassis 97077f76-46ae-4e15-9502-24fdb4ad0214 priority=32767 $ ovn-nbctl --no-leader-only set ha_chassis 97077f76-46ae-4e15-9502-24fdb4ad0214 priority=32767 But suddenly today something happened and all priority got reset causing small outage and cause all sriov VM started looking IP address. Solution was I have again set high priority again by hand to fix it.. How do I make priority permanently so it will never reset again. $ (ovn-nb-db)[root@os-chn-ctrl01 /]# ovn-nbctl --no-leader-only list HA_Chassis 784ec6e9-9af8-45c6-86e0-75ae7ccf9278 7fa04343-4471-4707-b5e9-23bd7f29ca6e 97077f76-46ae-4e15-9502-24fdb4ad0214 c75de7ab-b346-4aa5-886b-31b0fd12401e dd6ce2d4-9746-4429-862a-e78750bb4909 _uuid : 784ec6e9-9af8-45c6-86e0-75ae7ccf9278 chassis_name : os-chn-ctrl03 external_ids : {} priority : 32764 _uuid : 7fa04343-4471-4707-b5e9-23bd7f29ca6e chassis_name : os-chn-gen-comp-001 external_ids : {} priority : 32766 _uuid : 97077f76-46ae-4e15-9502-24fdb4ad0214 chassis_name : os-chn-gen-comp-002 external_ids : {} priority : 32767 _uuid : c75de7ab-b346-4aa5-886b-31b0fd12401e chassis_name : os-chn-ctrl01 external_ids : {} priority : 32765 _uuid : dd6ce2d4-9746-4429-862a-e78750bb4909 chassis_name : os-chn-ctrl02 external_ids : {} priority : 32763

1 1

Confidential VM and TDX support in upstream Openstack
by Ildiko Vancsa 12 Mar '25

12 Mar '25

Hi, I'm reaching out about a topic that came up in the Kata Containers / Confidential Containers communities related to a feature in Kata that is called remote hypervisor and also referred to as peer pods. Kata provides an API that allows users to implement a function to run Kata Containers pods in a remote location, which can include a different host machine or cloud environment. The Cloud API Adaptor in the Confidential Containers project is an implementation of the Kata remote hypervisor interface: https://github.com/confidential-containers/cloud-api-adaptor It recently came up to add support for OpenStack to be a cloud option to run Kata remote VMs in. In order to implement that, OpenStack would need to support the creation of Confidential VMs (CVM) along with things like TDX. I looked around and found 3rd-party implementation to add TDX support to Nova, but the repo has been archived: https://github.com/intel/secured-cloud-management-stack/blob/main/scm3.0/no… Does anyone know if the CVM support along with supporting Intel TDX or related technologies have been added to OpenStack already? Or if there's anyone in the community planning for adding these in the future? Thanks and Best Regards, Ildikó

4 4

[publiccloud-sig] Next meeting of Public Cloud SIG upcoming - March 12th at 0800 UTC
by Felix Kronlage-Dammers 11 Mar '25

11 Mar '25

Hi everyone, This upcoming Wednesday (March 12th) the next meeting of the publiccloud-sig is going to happen. We’ll be meeting at 0800 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG We've had a few new faces in the SIG lately which is awesome. Onboarding is easy - just come by and join the discussion :) cheers felix -- Felix Kronlage-Dammers Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

about using sriov-agent
by Jiatong Shen 11 Mar '25

11 Mar '25

Hello Experts, I am learning using SR-IOV to enable faster network access. I do not have a smartNIC so I am only talking about regular VF device passthrough. Right now, I configure Nova and use regular device passthrough flavor to create an instance. It works but since the PF device is a trunk network port, I have to configure sub interface manually in the guest. I find that I can only use neutron to configure SR-IOV VF devices (bandwidth, spoofcheck etc.). I am not trying it yet but I would like to ask if I use SR-IOV through neutron, does neutron's sriovagent will configure vlan id automatically such that I do not need to configure it inside the guest? if answer is yes, could you show me the related code? Thank you for your help. -- Best Regards, Jiatong Shen

4 8

[tempest] dynamically defined tests
by jerome.defavereau＠uclouvain.be 11 Mar '25

11 Mar '25

Hi, I'm currently trying to use tempest to run a simple series of tests on each of my hypervisors. I would like to have a separate test for each hypervisor instead of a single test for all. For that I need to dynamically define tests. I wrote a simple plugin to run the tests and then I use "sertattr" in a loop to define one "test_" method per hypervisor. The plugin is listed properly, as well as the tests, but when I run them using a regexp or the full test path I get "The test run didn't actually run any tests". I was able to run them using: "python -m unittest discover -s ./multi_hypervisor_tempest_plugin" but tempest run fails. Would anyone have wisdom to share ? Thanks !

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-3)
by Goutham Pacha Ravi 11 Mar '25

11 Mar '25

Hello Stackers, We're three weeks away from the coordinated 2025.1 "Epoxy" release [1]. The release will be published on 2025-04-02. This week, we're expecting to publish release candidates for all project deliverables that follow the "cycle-with-rc" release model [2]. In the past week, the OpenStack TC appointed project team leaders for Barbican (Mauricio Harley/mharley), Masakari (Suzhengwei/suzhengwei), and Zaqar (Hao Wang/wanghao) project teams. There's a pending appointment to the OpenStack Charms project [3] that we'll wrap up this week. The Oslo team re-nominated DPL liaisons, and the TC is going through the process of naming its chair and vice-chair for its current term. The community convened alongside Southern California Linux Expo (SCaLE) 2025 for an OpenInfra Days North America event. The event was well attended and featured close to 40 sessions spread across two days. You can watch most of the recorded proceedings as they were streamed live on both days [4][5][6]. We also celebrated OpenStack's 15th birthday with cake! If you missed this one, please check out upcoming birthday celebrations organized by different local user groups around the world [7]. === Weekly Meeting === The OpenStack Technical Committee (TC) held its weekly meeting primarily via video call [8], with action items and minutes documented in IRC [9]. The agenda covered past action items, PTL election results, leaderless projects, and a discussion on the format of TC meetings. A number of new Project Team Leads who will take charge of several OpenStack project teams next week are not core reviewers of the deliverables their teams manage. The general consensus was that a PTL does not have to be a core reviewer, as long as they collaborate effectively with the cores. PTLs should focus on driving consensus rather than making unilateral decisions—whether technical or otherwise. The TC trusts that if no one else nominates or objects, a lone PTL nominee has the support of the team. I personally believe that new and relatively new contributors can bring energy to the PTL role by being inquisitive and enthusiastic. I hope that the core team nurtures these qualities to integrate them into the fold sooner rather than later. The TC discussed why certain project teams that were deemed "leaderless" at the end of this past election consistently require appointments. We're learning from volunteers who have stepped up to lead each of these teams whether they would suggest any changes to the election process. Finally, we spent some time discussing whether the TC's monthly video meetings were useful and what the downsides were. We'll revisit this topic at the upcoming Project Teams Gathering with a wider audience. The next meeting of the OpenStack Technical Committee is today, 2025-03-11. Please find the agenda on the meeting's wiki page [10]. I hope you'll be able to join us. === Governance Proposals === ==== Merged ==== Add suzhengwei as PTL for Masakari | https://review.opendev.org/c/openstack/governance/+/943226 Appoint Hao Wang as PTL for OpenStack Zaqar | https://review.opendev.org/c/openstack/governance/+/943338 Switch back Oslo to DPL model | https://review.opendev.org/c/openstack/governance/+/942793 Appoint Mauricio Harley as PTL for Barbican | https://review.opendev.org/c/openstack/governance/+/942907 Add gouthamr's nomination for 2025.2 TC chair | https://review.opendev.org/c/openstack/governance/+/943204 Add TC chair nomination folder for 2025.2 | https://review.opendev.org/c/openstack/governance/+/943203 ==== Open for Review ==== Appoint Felipe Reyes as PTL for OpenStack Charms | https://review.opendev.org/c/openstack/governance/+/943254 Add Cinder Huawei charm | https://review.opendev.org/c/openstack/governance/+/867588 Resolve to adhere to non-biased language | https://review.opendev.org/c/openstack/governance/+/934907 === Upcoming Events === 2025-03-11: OpenInfra Board Meeting: https://board.openinfra.org/ 2025-04-02: OpenStack 2025.1 ("Epoxy") Release 2025-04-07 - 2025-04-12: OpenInfra PTG Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] cycle-with-rc release model: https://releases.openstack.org/reference/release_models.html#cycle-with-rc [3] Appoint Felipe Reyes as PTL for OpenStack Charms: https://review.opendev.org/c/openstack/governance/+/943254 [4] OpenInfraDays @ SCaLE 2025, Thursday Stream 1: https://www.youtube.com/watch?v=W9OmGdtJAAE [5] OpenInfraDays @ SCaLE 2025, Thursday Stream 2: https://www.youtube.com/watch?v=prIwZshr55U [6] OpenInfraDays @ SCaLE 2025, Friday: https://www.youtube.com/watch?v=Fv0Bdw6pBlk [7] Celebrating 15 years of OpenStack: https://www.openstack.org/blog/celebrating-15-years-of-openstack/ [8] TC Meeting Video Recording, 2025-03-04: https://youtu.be/NMDrz1N6ogo [9] TC Meeting IRC Log 2025-03-04: https://meetings.opendev.org/meetings/tc/2025/tc.2025-03-04-18.00.log.html [10] TC Meeting Agenda, 2025-03-11: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

openstack ansible lxc containers
by engineer2024 10 Mar '25

10 Mar '25

Using the lxc-container-create playbook, how to. create a container with a specific name and specific release ? openstack-ansible lxc-containers-create.yml -e "container_name=testing" -e "lxc_container_distribution=ubuntu" -e "lxc_container_release=jammy" --limit 'test' is throwing an error. it is trying to attach to the 'testing' container when it does not exist in the first place. I want the playbook to create one with that name ? So, what's the workaround ? --- module_stderr": "lxc-attach: testing: attach.c: get_attach_context: 405 Connection refused - Failed to get init pid\nlxc-attach: testing: attach.c: lxc_attach: 1469 Connection refused - Failed to get attach context\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", ----

2 1

[telemetry] Nominate Juan Larriba to Telemetry core
by Takashi Kajinami 10 Mar '25

10 Mar '25

Hi, As some of you may know, current telemetry core team is quite small due to some cores shifting their focus recently. (As far as I see Matthius and I are the cores around the community recently). On the other hand we have seen revived interest in adding new features to telemetry services. To make sure these effort may receive appropriate feedback, I'd like to nominate Juan as a new Telemetry project core. He has been contributing the project for some time, and I've recently seen his consistent presence to review patches. In addition he volunteered to be a Telemetry PTL for the next cycle with his continued interest to keep the project active. If I don't hear any objections for one week, I'll add Juan to the group. Also, I'd like to take this chance to mention that we are really happy to see anyone who is interested in helping the project and appreciate any help to review proposed changes or maintain the project (so that we can invite them to the core group as well). Thank you, Takashi Kajinami -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 4

[neutron] bug deputy report for 2025w10
by Bence Romsics 10 Mar '25

10 Mar '25

Hi Neutron Team! Last week we had the following new bug reports: Gate failures: * https://bugs.launchpad.net/neutron/+bug/2100776 test_securitygroup(ovs-openflow) fullstack test random failure with Time out exception unassigned * https://bugs.launchpad.net/neutron/+bug/2101165 Random failure in test_established_tcp_session_after_re_attachinging_sg unassigned * https://bugs.launchpad.net/neutron/+bug/2101166 Random failure in dvr ovs grenade jobs to ping vm post upgrade unassigned * https://bugs.launchpad.net/neutron/+bug/2100806 Intermittent test_batch_notifier failure fix proposed: https://review.opendev.org/c/openstack/neutron/+/943212 High: * https://bugs.launchpad.net/neutron/+bug/2101150 User who is not owner of the SG can create/delete rules in the shared SG assigned to Slawek * https://bugs.launchpad.net/neutron/+bug/2101063 [OVN] min-bw QoS policy cannot be defined alone, exception raised fix proposed: https://review.opendev.org/c/openstack/neutron/+/943664 Medium: * https://bugs.launchpad.net/neutron/+bug/2101140 [OVN] Make LSP up/down events processing resilient to IDL disconnections fix proposed: https://review.opendev.org/c/openstack/neutron/+/943404 * https://bugs.launchpad.net/neutron/+bug/2101162 [OVN] Use HA_Chassis_Group for tunnelled gateway network LRPs WIP fix: https://review.opendev.org/c/openstack/neutron/+/938134 * https://bugs.launchpad.net/neutron/+bug/2100938 Assign HA_Chassis_Group to all LRPs for tunnelled gateway assigned to Rodolfo Eventlet removal: * https://bugs.launchpad.net/neutron/+bug/2101076 [eventlet-removal] Remove the usage of eventlet in the MacVTap agent unassigned RFEs: * https://bugs.launchpad.net/neutron/+bug/2100770 [RFE] Make alembic_migration scripts in neutron idempotent * https://bugs.launchpad.net/neutron/+bug/2073254 nova-compute optional wait for vif-plugged in ovn usecase Further triage needed: * https://bugs.launchpad.net/neutron/+bug/2101077 neutron-bgp-agent dies with debug log + json Info requested from reporter on reproduction method * https://bugs.launchpad.net/neutron/+bug/2100752 Severe Packet Loss During Live Migration Due to Premature Route Advertisement in Neutron BGP Dynamic Routing Edit Lajos promised to triage this Cheers, Bence

1 0

using compute_monitors option for nova
by Jiatong Shen 08 Mar '25

08 Mar '25

Hello Experts, I hope this email finds you well. I am researching the "compute_monitors" option for nova and would like to find out what it can do. By some googling I find that by enabling this option, the compute_nodes table will store extra metrics and reflect the vcpu usage on the compute node. My question is is it possible to also save memory usage? Besides, is this option still supported in the long term? Thank you very much for the help. -- Best Regards, Jiatong Shen https://docs.openstack.org/ocata/config-reference/compute/config-options.ht…

2 2

[dev][ptl][release] Cycle highlights reminder (2025.1/Epoxy)
by Jeremy Stanley 07 Mar '25

07 Mar '25

Just a reminder in case anyone missed it: we've officially reached the deadline for merging 2025.1/Epoxy release highlights, but if anyone has something they want to mention then now is the time to write, push and approve it! The OpenInfra marketing machine is getting ready to digest what they find at https://releases.openstack.org/epoxy/highlights.html and put together a press release and package for news outlets. They're interested in whatever you're excited to include, but especially anything related to AI, digital sovereignty, security, or VMware feature parity. Please get your remaining highlights in ASAP!!! -- Jeremy Stanley

1 0

[horizon] Propose Jan Jašek to Horizon core
by Tatiana Ovchinnikova 07 Mar '25

07 Mar '25

Hello team, As it was discussed at our weekly meeting, I would like to add Jan Jašek to the OpenStack Horizon core team. Jan has been contributing to the Horizon project since 2022, and his impact is tremendous. His improvement and maintenance of Horizon test jobs made them significantly more stable, and Jan's reviews are always fast and valuable. Thank you Jan! Regards, Tatiana

1 0

[release] Release countdown for week R-3
by Thierry Carrez 07 Mar '25

07 Mar '25

Development Focus ----------------- The Release Candidate (RC) deadline is next Friday, March 14th, 2025. Work should be focused on fixing any release-critical bugs. General Information ------------------- All deliverables released under a cycle-with-rc model should have a first release candidate by the end of the week, from which a stable/2025.1 branch will be cut. This branch will track the 2025.1 Epoxy release. Once stable/2025.1 has been created, the master branch will be ready to switch to 2025.2 Flamingo development. While the master branch will no longer be feature-frozen, please prioritize any work necessary for completing 2025.1 Epoxy plans. Release-critical bugfixes will need to be merged in the master branch first, then backported to the stable/2025.1 branch before a new release candidate can be proposed. Actions ------- Early in the week, the release team will be proposing RC1 patches for all cycle-with-rc projects, using the latest commit from the master branch. If your team is ready to go for cutting RC1, please let us know by leaving a +1 on these patches. If there are still a few more patches needed before RC1, you can -1 the patch and update it later in the week with the new commit hash you would like to use. Remember, stable/2025.1 branches will be created with this, so you will want to make sure you have what you need included to avoid needing to backport changes from the master branch (which will technically then be Flamingo development) to this stable branch for any additional RCs before the final release. The release team will also be proposing releases for any deliverable following a cycle-with-intermediary model that has not produced any 2025.1 Epoxy release so far. Finally, now is a good time to finalize release highlights. Release highlights help shape the messaging around the release and make sure that your work is properly represented. Upcoming Deadlines & Dates -------------------------- RC1 deadline: March 14th, 2025 (R-3 week) Final RC deadline: March 28th, 2025 (R-1 week) Final 2025.1 Epoxy release: April 2nd, 2025 Project Teams Gathering (virtual): April 7th-11th, 2025 -- Thierry Carrez (ttx)

1 0

[openstack-helm] Inquiry: OpenStack-Helm Deployment in Production Environments
by Thamanna Farhath 07 Mar '25

07 Mar '25

Hello OpenStack Community, I hope this email finds you well. I am currently exploring OpenStack-Helm for deploying OpenStack and would like to understand its adoption in live production environments. Specifically, I am interested in knowing if any cloud providers have successfully deployed and are operating OpenStack using OpenStack-Helm in production. If there are any case studies, references, or experiences shared by operators who have implemented this deployment method, I would greatly appreciate any insights or guidance. Looking forward to your valuable input. Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

4 5

SRIOV on 2024.1 caracal
by Manohar Kundapura (makundap) 07 Mar '25

07 Mar '25

Hi Everyone, I have the 2024.1 Caracal multi node setup working. Now I am trying to enable SRIOV on this setup by following the below link, https://docs.openstack.org/neutron/2024.1/admin/config-sriov.html Observing the below issue while running the “neutron-sriov-agent” service on compute node and it is restarting on every 2 seconds. Kindly let me know if I need to enable any configs specific to “neutron-sriov-agent” service. # tail -f /var/log/neutron/neutron-sriov-agent.log 2025-03-06 15:02:42.968 9311 INFO neutron.common.config [-] Logging enabled! 2025-03-06 15:02:42.969 9311 INFO neutron.common.config [-] /usr/bin/neutron-sriov-nic-agent version 24.0.0 2025-03-06 15:02:42.969 9311 DEBUG neutron.common.config [-] command line: /usr/bin/neutron-sriov-nic-agent --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/plugins/ml2/sriov_agent.ini --log-file=/var/log/neutron/neutron-sriov-agent.log setup_logging /usr/lib/python3/dist-packages/neutron/common/config.py:123 2025-03-06 15:02:42.970 9311 DEBUG neutron.agent.common.utils [-] Std library socket module is patched by eventlet. Requesting std library socket module from eventlet. __init__ /usr/lib/python3/dist-packages/neutron/agent/common/utils.py:91 2025-03-06 15:02:42.971 9311 INFO neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [-] Physical Devices mappings: {'vf_inside': ['enp216s0f0'], 'vf_outside': ['enp216s0f1']} 2025-03-06 15:02:42.971 9311 INFO neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [-] Exclude Devices: {} 2025-03-06 15:02:42.972 9311 INFO neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [-] Resource provider bandwidths: {} 2025-03-06 15:02:42.972 9311 INFO neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [-] Resource provider inventory defaults: {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0} 2025-03-06 15:02:42.972 9311 INFO neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [-] Resource provider hypervisors: {'enp216s0f1': 'u36c01p06-ostest-cmp2.openstack.local', 'enp216s0f0': 'u36c01p06-ostest-cmp2.openstack.local'} 2025-03-06 15:02:42.973 9311 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.eswitch_manager [-] Number of VFs configured on device enp216s0f0: 8 get_numvfs /usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/eswitch_manager.py:134 2025-03-06 15:02:42.973 9311 DEBUG neutron.plugins.ml2.drivers.mech_sriov.agent.eswitch_manager [-] Number of VFs configured on device enp216s0f1: 8 get_numvfs /usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/eswitch_manager.py:134 2025-03-06 15:02:42.976 9311 DEBUG neutron.agent.securitygroups_rpc [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] Init firewall settings (driver=noop) init_firewall /usr/lib/python3/dist-packages/neutron/agent/securitygroups_rpc.py:124 2025-03-06 15:02:42.976 9311 INFO neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] RPC agent_id: nic-switch-agent.u36c01p06-ostest-cmp2 2025-03-06 15:02:42.981 9311 INFO neutron.agent.agent_extensions_manager [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] Loaded agent extensions: [] 2025-03-06 15:02:43.037 9311 WARNING oslo.messaging._drivers.impl_rabbit [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true': amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true' 2025-03-06 15:02:43.038 9311 INFO oslo.messaging._drivers.impl_rabbit [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] [7d2d8221-cd72-4203-b717-46c9d4fac7a9] Retrying to declare the exchange (q-agent-notifier-port-update_fanout) as non durable 2025-03-06 15:02:43.041 9311 ERROR oslo.messaging._drivers.impl_rabbit [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] Failed to declare consumer for topic 'q-agent-notifier-port-update': Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true': amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true' 2025-03-06 15:02:43.046 9311 ERROR oslo.messaging._drivers.impl_rabbit [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] Unable to connect to AMQP server on 10.10.4.19:5671 after inf tries: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true': amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true' 2025-03-06 15:02:43.047 9311 CRITICAL neutron [None req-617911dd-c075-4154-967b-a8ad1c334d34 - - - - - -] Unhandled error: oslo_messaging.exceptions.MessageDeliveryFailure: Unable to connect to AMQP server on 10.10.4.19:5671 after inf tries: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true' 2025-03-06 15:02:43.047 9311 ERROR neutron Traceback (most recent call last): 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 421, in declare 2025-03-06 15:02:43.047 9311 ERROR neutron self.queue.declare() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/entity.py", line 605, in declare 2025-03-06 15:02:43.047 9311 ERROR neutron self._create_exchange(nowait=nowait, channel=channel) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/entity.py", line 612, in _create_exchange 2025-03-06 15:02:43.047 9311 ERROR neutron self.exchange.declare(nowait=nowait, channel=channel) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/entity.py", line 180, in declare 2025-03-06 15:02:43.047 9311 ERROR neutron return (channel or self.channel).exchange_declare( 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/channel.py", line 612, in exchange_declare 2025-03-06 15:02:43.047 9311 ERROR neutron self.send_method( 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 66, in send_method 2025-03-06 15:02:43.047 9311 ERROR neutron return self.wait(wait, returns_tuple=returns_tuple) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 86, in wait 2025-03-06 15:02:43.047 9311 ERROR neutron self.connection.drain_events(timeout=timeout) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/connection.py", line 522, in drain_events 2025-03-06 15:02:43.047 9311 ERROR neutron while not self.blocking_read(timeout): 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/connection.py", line 528, in blocking_read 2025-03-06 15:02:43.047 9311 ERROR neutron return self.on_inbound_frame(frame) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/method_framing.py", line 53, in on_frame 2025-03-06 15:02:43.047 9311 ERROR neutron callback(channel, method_sig, buf, None) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/connection.py", line 534, in on_inbound_method 2025-03-06 15:02:43.047 9311 ERROR neutron return self.channels[channel_id].dispatch_method( 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 143, in dispatch_method 2025-03-06 15:02:43.047 9311 ERROR neutron listener(*args) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/channel.py", line 281, in _on_close 2025-03-06 15:02:43.047 9311 ERROR neutron raise error_for_code( 2025-03-06 15:02:43.047 9311 ERROR neutron amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true' 2025-03-06 15:02:43.047 9311 ERROR neutron 2025-03-06 15:02:43.047 9311 ERROR neutron During handling of the above exception, another exception occurred: 2025-03-06 15:02:43.047 9311 ERROR neutron 2025-03-06 15:02:43.047 9311 ERROR neutron Traceback (most recent call last): 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1036, in ensure 2025-03-06 15:02:43.047 9311 ERROR neutron ret, channel = autoretry_method() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/connection.py", line 524, in _ensured 2025-03-06 15:02:43.047 9311 ERROR neutron return fun(*args, **kwargs) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/connection.py", line 600, in __call__ 2025-03-06 15:02:43.047 9311 ERROR neutron return fun(*args, channel=channels[0], **kwargs), channels[0] 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1025, in execute_method 2025-03-06 15:02:43.047 9311 ERROR neutron method() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1262, in _declare_consumer 2025-03-06 15:02:43.047 9311 ERROR neutron consumer.declare(self) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 431, in declare 2025-03-06 15:02:43.047 9311 ERROR neutron self._declare_fallback(err, conn, consumer_arguments) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 396, in _declare_fallback 2025-03-06 15:02:43.047 9311 ERROR neutron self.queue.declare() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/entity.py", line 605, in declare 2025-03-06 15:02:43.047 9311 ERROR neutron self._create_exchange(nowait=nowait, channel=channel) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/entity.py", line 612, in _create_exchange 2025-03-06 15:02:43.047 9311 ERROR neutron self.exchange.declare(nowait=nowait, channel=channel) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/kombu/entity.py", line 180, in declare 2025-03-06 15:02:43.047 9311 ERROR neutron return (channel or self.channel).exchange_declare( 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/channel.py", line 612, in exchange_declare 2025-03-06 15:02:43.047 9311 ERROR neutron self.send_method( 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 66, in send_method 2025-03-06 15:02:43.047 9311 ERROR neutron return self.wait(wait, returns_tuple=returns_tuple) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 86, in wait 2025-03-06 15:02:43.047 9311 ERROR neutron self.connection.drain_events(timeout=timeout) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/connection.py", line 522, in drain_events 2025-03-06 15:02:43.047 9311 ERROR neutron while not self.blocking_read(timeout): 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/connection.py", line 528, in blocking_read 2025-03-06 15:02:43.047 9311 ERROR neutron return self.on_inbound_frame(frame) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/method_framing.py", line 53, in on_frame 2025-03-06 15:02:43.047 9311 ERROR neutron callback(channel, method_sig, buf, None) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/connection.py", line 534, in on_inbound_method 2025-03-06 15:02:43.047 9311 ERROR neutron return self.channels[channel_id].dispatch_method( 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/abstract_channel.py", line 143, in dispatch_method 2025-03-06 15:02:43.047 9311 ERROR neutron listener(*args) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/amqp/channel.py", line 281, in _on_close 2025-03-06 15:02:43.047 9311 ERROR neutron raise error_for_code( 2025-03-06 15:02:43.047 9311 ERROR neutron amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true' 2025-03-06 15:02:43.047 9311 ERROR neutron 2025-03-06 15:02:43.047 9311 ERROR neutron During handling of the above exception, another exception occurred: 2025-03-06 15:02:43.047 9311 ERROR neutron 2025-03-06 15:02:43.047 9311 ERROR neutron Traceback (most recent call last): 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/bin/neutron-sriov-nic-agent", line 10, in <module> 2025-03-06 15:02:43.047 9311 ERROR neutron sys.exit(main()) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/plugins/sriov_nic_neutron_agent.py", line 27, in main 2025-03-06 15:02:43.047 9311 ERROR neutron agent_main.main() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py", line 568, in main 2025-03-06 15:02:43.047 9311 ERROR neutron agent = SriovNicSwitchAgent(device_mappings, 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py", line 181, in __init__ 2025-03-06 15:02:43.047 9311 ERROR neutron self.connection.consume_in_threads() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/rpc.py", line 345, in consume_in_threads 2025-03-06 15:02:43.047 9311 ERROR neutron server.start() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 267, in wrapper 2025-03-06 15:02:43.047 9311 ERROR neutron states[state].run_once(lambda: fn(self, *args, **kwargs), 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 188, in run_once 2025-03-06 15:02:43.047 9311 ERROR neutron post_fn = fn() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 267, in <lambda> 2025-03-06 15:02:43.047 9311 ERROR neutron states[state].run_once(lambda: fn(self, *args, **kwargs), 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 413, in start 2025-03-06 15:02:43.047 9311 ERROR neutron self.listener = self._create_listener() 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 150, in _create_listener 2025-03-06 15:02:43.047 9311 ERROR neutron return self.transport._listen(self._target, 1, None) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 142, in _listen 2025-03-06 15:02:43.047 9311 ERROR neutron return self._driver.listen(target, batch_size, 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 709, in listen 2025-03-06 15:02:43.047 9311 ERROR neutron conn.declare_fanout_consumer(target.topic, listener) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1398, in declare_fanout_consumer 2025-03-06 15:02:43.047 9311 ERROR neutron self.declare_consumer(consumer) 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1273, in declare_consumer 2025-03-06 15:02:43.047 9311 ERROR neutron return self.ensure(_declare_consumer, 2025-03-06 15:02:43.047 9311 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1058, in ensure 2025-03-06 15:02:43.047 9311 ERROR neutron raise exceptions.MessageDeliveryFailure(msg) 2025-03-06 15:02:43.047 9311 ERROR neutron oslo_messaging.exceptions.MessageDeliveryFailure: Unable to connect to AMQP server on 10.10.4.19:5671 after inf tries: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'q-agent-notifier-port-update_fanout' in vhost 'neutron': received 'false' but current is 'true' 2025-03-06 15:02:43.047 9311 ERROR neutron Thanks, Manohar

3 2

[neutron] Drivers meeting canceled Friday, March 7th
by Brian Haley 06 Mar '25

06 Mar '25

Hello Neutrinos: I'm going to cancel tomorrow's drivers meeting as discussed at this week's team meeting, since we are past feature freeze and should be focusing on getting in-flight things merged and bug fixes. Just a reminder there are still open specs that need review, list is at [0]. And just a reminder I am offline until next Tuesday. Have a nice weekend! -Brian [0] https://review.opendev.org/q/status:open+project:openstack/neutron-specs

1 0

Assistance Required: OpenStack Helm Deployment (Version 2023.1) Compatibility Issues
by Thamanna Farhath 05 Mar '25

05 Mar '25

Hello OpenStack Community, I hope you are doing well. I am currently deploying OpenStack on top of a Kubernetes cluster using OpenStack Helm and aiming to install the 2023.1 (Antelope) version. However, I am facing difficulties in installing this specific version via Helm charts. Despite my attempts, I have not been able to successfully deploy the required OpenStack services for this release. Additionally, I am ensuring that the dependent services, such as RabbitMQ, align with the compatibility requirements of OpenStack 2023.1. As per the OpenStack documentation, RabbitMQ 3.11 is the recommended version, but I encountered issues while deploying it. Instead, I found that RabbitMQ 3.12 works without any issues. I would like to know: Is there a way to install OpenStack 2023.1 properly using Helm charts, or should I consider using a newer OpenStack version? Will using RabbitMQ 3.12 instead of 3.11 cause any potential issues with OpenStack services, or is it safe to proceed with this version? Are there any known workarounds or recommendations for deploying OpenStack 2023.1 using Helm? I would appreciate any guidance, best practices, or documentation references that could help resolve these challenges. Looking forward to your valuable insights. Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

4 6

[nova][libvirt][qemu] Internal error: process exited while connecting to monitor
by Roby G 04 Mar '25

04 Mar '25

Hello everyone, After upgrading to Antelope i´m getting following errors when creating new volume-backed instance. Volumes are stored in a ceph pool. - nova: 27.4.0 - libvirt: 8.0.0 - qemu: 6.2.0 - Operating system: Ubuntu 22.04 ``` ERROR nova.virt.libvirt.driver [instance: f8fa30f3-3dc8-43bd-bc29-8ed23b15b1f2] Failed to start libvirt guest: libvirt.libvirtError: internal error: process exited while connecting to monitor: 2025-03-04T15:15:34.416946Z qemu-system-x86_64: -blockdev {"driver":"rbd","pool":"cinder","image":"volume-6dfe3084-ef9d-4107-8b69-b0036fefbed0","server":[{"host":"1.2.3.4","port":"1234"},{"host":"1.2.3.5","port":"1234"},{"host":"1.2.3.6","port":"1234"}],"user":"cinder","auth-client-required":["cephx","none"],"key-secret":"libvirt-1-storage-auth-secret0","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error connecting: Permission denied libvirtd[50229]: internal error: process exited while connecting to monitor: 2025-03-04T15:15:34.416946Z qemu-system-x86_64: -blockdev {"driver":"rbd","pool":"cinder","image":"volume-6dfe3084-ef9d-4107-8b69-b0036fefbed0","server":[{"host":"1.2.3.4","port":"1234"},{"host":"1.2.3.5","port":"1234"},{"host":"1.2.3.6","port":"1234"}],"user":"cinder","auth-client-required":["cephx","none"],"key-secret":"libvirt-1-storage-auth-secret0","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error connecting: Permission denied libvirtd[50229]: internal error: qemu unexpectedly closed the monitor: 2025-03-04T15:15:34.416946Z qemu-system-x86_64: -blockdev {"driver":"rbd","pool":"cinder","image":"volume-6dfe3084-ef9d-4107-8b69-b0036fefbed0","server":[{"host":"1.2.3.4","port":"1234"},{"host":"1.2.3.5","port":"1234"},{"host":"1.2.3.6","port":"1234"}],"user":"cinder","auth-client-required":["cephx","none"],"key-secret":"libvirt-1-storage-auth-secret0","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error connecting: Permission denied os-cmt7 libvirtd[50229]: Unable to read from monitor: Connection reset by peer ``` Any fixes/tips? From The compute node accessing the ceph pool using `rbd` client, there are no permission errors. Are this errors related to qemu monitor?

1 0

PTG April 2025 Teams List
by Kendall Nelson 04 Mar '25

04 Mar '25

Hello Everyone! The list of teams participating in the upcoming virtual PTG (April 7-11, 2025) is now official! Projects + Teams - OpenInfra Foundation Teams - - OpenInfra Diversity and Inclusion Working Group - Kata Containers - OpenStack - - Services - - Blazar - Cinder - CloudKitty - Designate - Glance - Heat - Horizon - Ironic - Keystone - Kolla - Manila - Neutron - Nova - Octavia - OpenStack-Ansible - OpenStack SDK/CLI - Sunbeam - Swift - Tacker - Watcher - Other OpenStack Teams - Eventlet Removal - i18n SIG - OpenStack for AI Working Group - OpenStack Security SIG - OpenStack Technical Committee - Public Cloud SIG - VMware Migration Working Group - StarlingX - Adjacent Open Source Communities - - OceanBase If your team was planning to meet and isn’t in this list, please contact ptg(a)openinfra.dev IMMEDIATELY. Soon I will be contacting moderators to sign up for time via the PTGBot[1] once it is configured for the teams that are signed up to attend the event. Etherpads for agenda planning and note taking will be automatically generated as a part of that configuration. Otherwise, please don't forget to register[2] — it's FREE! Also, it does help us with event coordination and communication, so please take a moment to register. As usual, feel free to let me know if you have any questions! Thanks! -Kendall (diablo_rojo) [1] PTGBot Docs: https://opendev.org/openstack/ptgbot#open-infrastructure-ptg-bot [2] PTG Registration: http://ptg.openinfra.org/

1 0

[neutron] LXB Removal and OVN migration
by Ricardo Cano 04 Mar '25

04 Mar '25

Hello openstack-discuss, With the removal of the linuxbridge mechanism driver, has anyone successfully completed an in-place migrationfrom LinuxBridge to OVN? I did come across this https://www.jimmdenton.com/migrating-lxb-to-ovn/ and that looks good as a one-shoit while biting the bullet and taking a full outage. However, I was wondering if it was possible to run lxb and ovn side-by-side and cut them over, say one network at a time. This particular deployment was stood up using openstack-ansible back in the rocky days (now running antelope) and it appears that openstack ansible's default was lxb up until Zed. Any suggestions or recommendations on the best path forward? Also, will the dynamic routing agent work with ovn?

7 8

LXB Removal and OVN migration
by Ricardo Cano 04 Mar '25

04 Mar '25

Hello openstack-discuss, With the removal of the linuxbridge mechanism driver, has anyone successfully completed an in-place migrationfrom LinuxBridge to OVN? I did come across this https://www.jimmdenton.com/migrating-lxb-to-ovn/ and that looks good as a one-shoit while biting the bullet and taking a full outage. However, I was wondering if it was possible to run lxb and ovn side-by-side and cut them over, say one network at a time. This particular deployment was stood up using openstack-ansible back in the rocky days (now running antelope) and it appears that openstack ansible's default was lxb up until Zed. Any suggestions or recommendations on the best path forward? Also, will the dynamic routing agent work with ovn?

2 1

Contributing to Openstack [Looking for Mentorship]
by jb03229422＠student.ku.edu.np 04 Mar '25

04 Mar '25

Dear Mentors, I have good experience in building openstack in multiple environments bare-metal, kolla-ansible and helm project for minimal deployment to good HA clusters. Now, I'm looking to get into code level and contribute to open-infra foundation. I'm not a someone beginner to coding, I have done programming in a good amount and hold a knowledge of paradigm and principles. Here's my github: https://github.com/jamesii-b I'm looking for mentor who can guide me what to do as a Contribution.. Even deploying first openstack is overwhelming on its own, Coding is more. Please have me. Trust me, I'll have utmost respect to your time and energy and get most of it. Knowledge I have: HPC, Openstack Deployment, Network & System, Parallel Programming, Python, Rust.

2 1

[tacker] Today's weekly meeting
by Yasufumi Ogawa 04 Mar '25

04 Mar '25

Hi tacker team, I might not be able to join today's meeting due to something urgent come up. Let me join the today's discussion by adding my comment on the etherpad later. Thanks, Yasufumi

1 0