March 2025 - openstack-discuss - lists.openstack.org

[administrivia] Recent change in moderation for new subscribers
by Jeremy Stanley 18 Mar '25

18 Mar '25

Due to a spate of spammers subscribing to openstack-discuss over the last several days in order to send off-topic messages to the list, I've altered the default behavior for posts from new subscribers. For now, if someone subscribes, their initial post to the list will be held for moderation in order for our moderators to make sure it's a legitimate post, before setting their subscription to bypass moderation for any subsequent posts they make. Note that this only affects new … [View More]

1 0

[nova][cinder][glance] Support for rbd namespaces
by Eugen Block 18 Mar '25

18 Mar '25

Hi, I've been searching and testing for a while, but I haven't found anything helpful yet. Someone in the Ceph community brought up the question about rbd namespaces for nova, cinder and glance. It would make sense to use only one pool for those services with regards to PG numbers. Having multiple pools means to manage their pg_num individually, and scaling can be limited due to the max number of PGs per OSD. Having only one pool would make that easier. I mean, we've been using a … [View More]

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.1/R-2)
by Goutham Pacha Ravi 18 Mar '25

18 Mar '25

Hello Stackers, We're two weeks away from the release date for the 2025.1 ("Epoxy") coordinated release [1]. OpenStack project teams worked with the release team to make the first release candidate (RC) available for wider testing. Over the next couple of weeks, any release-critical bugs may require revised RCs. Now's a good time to review release highlights [2] that project teams have added to reflect major changes made during this release. Development focus has shifted to the next release, … [View More]2025.2 "Flamingo" [3]. Last week was eventful. The OpenInfra Foundation board unanimously approved resolutions that further its move to become a sub-foundation of the Linux Foundation [4]. For many of you who are members of the foundation, you will have an opportunity to ratify the board's decisions through a vote next month. In the past week, the OpenStack Technical Committee didn't merge any new governance updates; however, several are open for community review. === Weekly Meeting === The TC's weekly meeting in OFTC's #openstack-tc channel on 2025-03-11 was well attended [5]. We were joined by several attendees of the OpenInfra Days North America event to discuss key takeaways and gather action items for the TC. There was broad acknowledgment that many presentations and discussions at the event revolved around Running AI workloads on OpenStack and aiding the migration of enterprises that are transitioning to OpenStack from existing commercial/proprietary solutions We expressed interest in participating in working group discussions at the upcoming PTG on both these topics. Additionally, we revisited community pain points that were brainstormed at the event. Two notable issues were: - The need for an official presence on communication platforms beyond OFTC/IRC - The lack of sufficient engagement from operators on the openstack-discuss mailing list No immediate actions were taken on these issues during the meeting, but both topics generated significant discussion. We expect to gather input from the wider community during the upcoming PTG. The next OpenStack TC meeting is today, 2025-03-18, at 1800 UTC on OFTC's #openstack-tc channel. Please find the agenda on the meeting’s wiki page [6]. I hope you'll be able to join us. I'd encourage the community to bring any topics of interest to this meeting. The synchronous attention of the TC and the lively discussions that follow are a great way to demonstrate our commitment to OpenStack’s "Open Community" values. === Governance Proposals === ==== Open for Review ==== - [resolution] Extend scope of VMT to cover all project teams | https://review.opendev.org/c/openstack/governance/+/944817 - Add separate freezer-api deliverable | https://review.opendev.org/c/openstack/governance/+/944781 - Fix links to chair responsibilities in TC Guide | https://review.opendev.org/c/openstack/governance/+/944802 - Add Cinder Huawei charm | https://review.opendev.org/c/openstack/governance/+/867588 - Resolve to adhere to non-biased language | https://review.opendev.org/c/openstack/governance/+/934907 === Upcoming Events === 2025-04-01 – 2025-04-04: KubeCon+CloudNativeCon EU: https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/ 2025-04-02: OpenStack 2025.1 ("Epoxy") Release 2025-04-08: OpenInfra Board Meeting: https://board.openinfra.org/ 2025-04-07 – 2025-04-12: OpenInfra PTG === How to Contact the TC === You can reach the TC in several ways: - Email: Send an email with the [tc] tag on this mailing list. - IRC: Ping us using the tc-members keyword in the #openstack-tc IRC channel on OFTC. - Weekly Meetings: The Technical Committee meets every Tuesday at 1800 UTC [6]. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [2] 2025.1 "Epoxy" Release Highlights: https://releases.openstack.org/epoxy/highlights.html [3] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [4] OIF Board March '25 Meeting Recap: https://lists.openinfra.org/archives/list/foundation@lists.openinfra.org/th… [5] TC Meeting IRC Log 2025-03-11: https://meetings.opendev.org/meetings/tc/2025/tc.2025-03-11-18.00.log.html [6] TC Meeting Agenda, 2025-03-18: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [View Less]

1 0

[ovs] [neutron] Egress traffic is not reaching table 71
by yardalgedal＠gmail.com 18 Mar '25

18 Mar '25

Hi, all! We are experimenting with bluefield-2 and neutron-openvswith-agent on it and have a big problem – all our egress traffic (from stateless point of view) are not leaving the bluefield. Symptoms are the next: the baremetal to which bluefield-2 is connected successfully responds on ping, but does not return any response for TCP traffic and can not send anything from inside. We have some security group with fully allowed egress rules. +--------------------------------------+-----… [View More]--------+-----------+-----------+------------+-----------+-----------------------+----------------------+ | ID | IP Protocol | Ethertype | IP Range | Port Range | Direction | Remote Security Group | Remote Address Group | +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ | 14604165-e0c1-4bce-ae21-d20bf4165fa2 | None | IPv6 | ::/0 | | ingress | None | None | | 16f3f44f-1c9a-48f1-bcd0-7ff8df0cd30a | None | IPv4 | 0.0.0.0/0 | | ingress | None | None | | c7054cde-72ce-4dc8-a4e0-361df199204e | None | IPv6 | ::/0 | | egress | None | None | | d8bc8f82-f181-443d-a0ca-67cbae78b47f | None | IPv4 | 0.0.0.0/0 | | egress | None | None | +--------------------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+ and port to which it is assigned has port_security_enabled: True (if we are disabling port security, there is no problem anymore, it is actual only for enabled port security) we have disabled DVR and VxLAN tunnel here, we have native openvswitch firewall also [agent] tunnel_types = l2_population = true arp_responder = true extensions = qos baremetal_smartnic = True [securitygroup] firewall_driver = openvswitch [ovs] bridge_mappings = br-ex:br-int datapath_type = system ovsdb_connection = tcp:127.0.0.1:6640 #local_ip = tunnel_bridge = of_inactivity_probe = 60 ovsdb_timeout = 60 when we're sending some traffic, all flows in table 71 (BASE_EGRESS_TABLE) has n_packets = 0, so they are not reaching it. in table 60 (TRANSIENT_TABLE), which should send traffic to the table 71, on flow which should do it we can see n_packets=0 as well cookie=0x6cc97f12f173f538, duration=2086.129s, table=60, n_packets=0, n_bytes=0, priority=100,in_port=pf0hpf actions=load:0x2b->NXM_NX_REG5[],load:0x1->NXM_NX_REG6[],resubmit(,71) at the same time, the counter for NORMAL flow is growing, so packets are probably going to here (not sure cookie=0x6cc97f12f173f538, duration=2126.547s, table=60, n_packets=139340, n_bytes=6817502, priority=3 actions=NORMAL on trace we can see that the packet was directly sent to port 1 instead of going via the table ovs-appctl ofproto/trace br-int 'dl_src=a0:88:c2:eb:15:4e,dl_dst=00:1c:73:00:57:ff,ip,nw_dst=DST_IP,nw_src=SRC_IP,in_port=43,nw_ttl=64,nw_proto=6,tp_dst=80' Flow: tcp,in_port=43,vlan_tci=0x0000,dl_src=a0:88:c2:eb:15:4e,dl_dst=00:1c:73:00:57:ff,nw_src=SRC_IP,nw_dst=DST_IP,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,tp_src=0,tp_dst=80,tcp_flags=0 bridge("br-int") ---------------- 0. priority 0, cookie 0x699a137a0a73d6a8 NORMAL -> forwarding to learned port bridge("br-int") ---------------- 0. in_port=5,dl_vlan=1, priority 4, cookie 0x699a137a0a73d6a8 set_field:4297->vlan_vid NORMAL -> forwarding to learned port Final flow: unchanged Megaflow: recirc_id=0,eth,ip,in_port=43,dl_src=a0:88:c2:eb:15:4e,dl_dst=00:1c:73:00:57:ff,nw_frag=no Datapath actions: push_vlan(vid=201,pcp=0),1 any ideas why it happens or any proposals for future debugging? Thank you in advance! [View Less]

1 0

Controller node openstack LXC containers clashing on IP subnets
by Sarah Thompson 17 Mar '25

17 Mar '25

Hi all, I've got much further now, and am now tracking down networking issues. I'm pretty sure everything is now installing, but I'm seeing a systematic issue. My test network that I'm running the openstack instances on is 10.0.1.x -- the VMs all have fixed IP addresses, can talk to each other, etc, nothing weird going on. haproxy is installing and running, but throwing 503 errors. Digging into this, it seems that there are some issues with the network configs of at least some of the LXC … [View More]containers. The one I'm seeing that's preventing Ansible from completing the infrastructure setup is the repo server container. If I attach to the container, the keepalive comes back correctly. Externally to the container, the HTTP connection is rejected. Looking into the reasons, it looks like there are 3 networks visible from inside the container: ens18: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.1.38 netmask 255.255.255.0 broadcast 10.0.1.255 inet6 fe80::216:3eff:feab:ab45 prefixlen 64 scopeid 0x20<link> ether 00:16:3e:ab:ab:45 txqueuelen 1000 (Ethernet) RX packets 96 bytes 6952 (6.9 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 16 bytes 1236 (1.2 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.3.113 netmask 255.255.255.0 broadcast 10.0.3.255 inet6 fe80::216:3eff:fe45:64d6 prefixlen 64 scopeid 0x20<link> ether 00:16:3e:45:64:d6 txqueuelen 1000 (Ethernet) RX packets 351 bytes 425285 (425.2 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 263 bytes 19114 (19.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 2939 bytes 208134 (208.1 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 2939 bytes 208134 (208.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 The 10.0.1.38 address seems to be the problem. I think this is an internally routed subnet, not the actual physical subnet (note the lack of a gateway address, and 10.0.1.38 is definitely not being allocated via the DCHP server). Looking at some other containers, I'm seeing 10.0.2.x and 10.0.3.x there, so obviously this is being allocated on the fly either by lxc or the container(s). TL;DR: The internal 10.0.1.x is clashing with the physical 10.0.1.x network, which is almost certainly why the keepalive is failing. Does anyone have any idea how to fix the configuration to use some other CIDR block for this? I'd like to avoid the extreme pain of remapping my physical network (this is a way more complicated problem than a few test nodes unfortunotely!) Thank you in advance, Sarah Thompson -- [s] [View Less]

3 4

PCI alias rtx is not defined
by Felipe Mogollon 17 Mar '25

17 Mar '25

Dear all, I am suffering a problem in my OpenStack servers. I have a 3 server configuration: - controller + compute - 2 x compute nodes The controller node has an Nvidia L4 configured in vGPU mode. One of the compute nodes has another Nvidia L4 in vGPU mode. Both servers are running fine, and I can deploy instances with vGPU capabilities. The problem comes with my 2nd compute node; that node has a Nvidia RTX4000 that can only work in passthrough mode. , I am pretty sure that in past times … [View More]

1 0

Routed provider networks
by Alvin Starr 17 Mar '25

17 Mar '25

I have a couple of small flat networks that I am using for a POC project. I am trying to use the routed provider networks features to create a single public net. I delete the old networks and try to install the following network and it looks like it works in that I see the 2 subnets as part of the single public-net. The trouble is that I cannot create a router with an interface in the public-net. Is there anything special that I need to do to create a router on the public-net? I changed … [View More]

1 0

[openstack-ansible]Playbooks failing after upgrading lxc containers from ubuntu18 to ubuntu20 due to python version mismatch.
by Danish Khan 17 Mar '25

17 Mar '25

Dear All, We are upgrading one of our openstack cluster from ussuri to wallaby and for this we are first upgrading OS of lxc container from ubuntu to 18 to 20(hosts OS is also upgraded to 20). We are destroying LXC containers(which don't form quorum like nova,neutron,heat,cinder,horizon) of one infra node and then creating it again. The new containers are getting created with Ubuntu 20 as the host OS is 20. Playbook for cinder, nova neutron were completed successfully for new containers of … [View More]ubuntu 20. But Playbook is failing for heat, placement and memcached due to python version mismatch: fatal: [host5_heat_api_container-25f90e52]: FAILED! => {"attempts": 5, "changed": false, "cmd": ["/openstack/venvs/heat-21.2.5/bin/pip3", "install", "-U", "--constraint", "/openstack/venvs/heat-21.2.5/global-constraints.txt", "--constraint", "/openstack/venvs/heat-21.2.5/constraints.txt", "--pre", "--log", "/var/log/python_venv_build.log", "cryptography", "keystonemiddleware", "openstack-heat", "osprofiler", "pymemcache", "PyMySQL", "python-ceilometerclient", "python-glanceclient", "python-heatclient", "python-keystoneclient", "python-memcached", "python-neutronclient", "python-novaclient", "python-openstackclient", "python-swiftclient", "python-troveclient", "systemd-python"], "msg": "stdout: Ignoring python-saharaclient: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring python-saharaclient: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring python-saharaclient: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring oslo.limit: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring oslo.limit: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring oslo.limit: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring sphinxcontrib-actdiag: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring sphinxcontrib-actdiag: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring sphinxcontrib-actdiag: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring os-api-ref: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring os-api-ref: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring os-api-ref: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring oslo.concurrency: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring oslo.concurrency: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring oslo.concurrency: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring osprofiler: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring osprofiler: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring osprofiler: markers 'python_version == \"3.7\"' don't match your environment\nIgnoring os-resource-classes: markers 'python_version == \"2.7\"' don't match your environment\nIgnoring os-resource-classes: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring os-resource-classes: markers I understand that 2.7 version is depricated but where should make changes to fix it? https://docs.openstack.org/releasenotes/python-heatclient/ussuri.html I tried updating few variable to fix python version like: /etc/ansible/roles/os_heat/defaults/main.yml #heat_venv_python_executable: "{{ openstack_venv_python_executable | default('python3') }}" updated it from python2 to python 3 Tried few more variables but that didn't work. Any idea how we can fix this version mismatch issue ? Regards, Danish [View Less]

2 1

[Devstack] Trying to setup development environment on VM
by Ishan Shanware (ishanwar) 17 Mar '25

17 Mar '25

Hello Everyone, I am currently trying to setup in all-in-one development environment for OpenStack on a Debian VM. The VM has 16 GB RAM and 4 CPUs. However, I face an error while trying to run FORCE=yes ./stack.sh. I get an error that keystone does not start. This causes the entire process to fail. Requesting assistance in solving this issue. I am following the this guide: https://docs.openstack.org/devstack/latest/guides/single-machine.html PFA the error log and error message on the … [View More]

3 5

[neutron] Bug deputy report for week 11 of 2025
by Miguel Lavalle 16 Mar '25

16 Mar '25

I was the Neutron bug deputy last week. Following is a brief summary of the reported bugs: High bugs -------------- https://bugs.launchpad.net/neutron/+bug/2101839 [fullstack] Random errors during the execution of ``wait_until_true``. Assignee Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/943942 https://bugs.launchpad.net/neutron/+bug/2101857 ``neutron-tempest-plugin-ovn-ubuntu-jammy`` job unstable. No owner. Jobs neutron-tempest-plugin-*-ubuntu-jammy`` … [View More]

1 0