January 2025 - openstack-discuss - lists.openstack.org

How to know the openstack client's default microversion?
by berndbausch＠gmail.com 20 Jan '25

20 Jan '25

Without the version option e.g. --os-compute-api-version, the openstack client submits API requests with certain default microversions. The precise microversions vary with the openstack client's version and the specific task. For example, the --debug option reveals that openstack client 7.1.3 uses Nova microversion 2.89 when attaching a volume to an instance. Is there a simpler way than --debug to find out which microversions the client uses? Is there a command, or perhaps a website where this is documented? Thanks for your ideas! Bernd.

2 1

[openstack-ansible] : libvirtd debug logs after Wallaby upgrade
by Rambo Rambo 20 Jan '25

20 Jan '25

Hi Team I have recently done Victoria to Wallaby upgrade on my openstack-ansible installation. After the upgrade we observe that /var/log/libvirt/ size on all the computes was growing due to libvirtd.log file. Further checking this file it can be seen that there are debug logs. 2025-01-19 11:15:20.817+0000: 2943565: *debug *: virNetlinkEventCallback:862 : dispatching to max 0 clients, called from event watch 7 2025-01-19 11:15:20.817+0000: 2943565:* debug* : virNetlinkEventCallback:876 : event not handled. 2025-01-19 11:15:20.821+0000: 2943605: *debug :* udevHandleOneDevice:1492 : udev action: 'add' ... libvirtd configuration file on compute shows log_level=1 cat libvirtd.conf # Ansible managed # enabling debug mode according to http://wiki.libvirt.org/page/DebugLogs log_level = 1 log_filters="3:remote 4:event 3:json 3:rpc" log_outputs="1:file:/var/log/libvirt/libvirtd.log" listen_tls = 0 listen_tcp = 0 unix_sock_group = "libvirt" unix_sock_ro_perms = "0777" unix_sock_rw_perms = "0770" auth_unix_ro = "none" auth_unix_rw = "none" auth_tcp = "sasl" No related parameter was changed/updated in user_variable file during the upgrade? Any information what could have enabled the debug logs for libvirtd on all the computes? Also how can I turn-off this using the openstack-ansible and what could be impact during the procedure? Regards

2 3

Openstack Ansible
by engineer2024 20 Jan '25

20 Jan '25

It seems that the ssh to containers from the deploy node, is removed in the caracal release by default in OSA. The new containers don't have the ssh service inside them. Is there a way to enable it via a variable ?

2 2

[neutron] Bug deputy week 3 2025 (2025-01-13 - 2025-01-19)
by Lajos Katona 20 Jan '25

20 Jan '25

Hi Neutrinos, The bug list for week 3 of 2025. * Critical ** https://bugs.launchpad.net/neutron/+bug/2094970 (rally ovn job timing out with eventlet 0.38.2) *** https://review.opendev.org/c/openstack/neutron/+/939315 Merged * High ** https://bugs.launchpad.net/neutron/+bug/2094736 ([OVN] Missing ``LogicalSwitchPortUpdateUpEvent`` when a port is bound) *** WIP: https://review.opendev.org/c/openstack/neutron/+/939345 ** https://bugs.launchpad.net/neutron/+bug/2094810 (neutron-status upgrade check fails on checking if ovn_alembic_version table exists) *** https://review.opendev.org/c/openstack/neutron/+/939248 ** https://bugs.launchpad.net/neutron/+bug/2094842 ("ovs.db.error.Error: ovsdb error: 0 values when type requires between 1 and 9223372036854775807" when router is attached to network without subnet ) *** Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095000 (ovs-agent: Failed to store metadata for trunk ...: Cannot find Bridge with name=tbr-... ) Assigned ** https://bugs.launchpad.net/neutron/+bug/2095044 (test_ip_lib calls to sudo and fails (sometimes?)) *** https://review.opendev.org/c/openstack/neutron/+/939435 Merged ** https://bugs.launchpad.net/neutron/+bug/2095167 (BW limit not applied to external/provider network) *** https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/939539 ** https://bugs.launchpad.net/neutron/+bug/2095180 (test_agent_metadata_port_dhcp_reenable_event fails when ovsdb-server combines create and later update into a single create event for monitor) *** https://review.opendev.org/c/openstack/neutron/+/939542 ** https://bugs.launchpad.net/neutron/+bug/2095195 ( VM fails to receive IPv4 from DHCP agent because the agent has not updated the network config for dnsmasq after port was created) ** https://bugs.launchpad.net/neutron/+bug/2095196 (functional: test_virtual_port_host_update failed due to ovsdb SB transaction timeout, "Unreasonably long 25512ms poll interval (0ms user, 26ms system)" in ovsdb-server log) *** https://review.opendev.org/c/openstack/neutron/+/939558 ** https://bugs.launchpad.net/neutron/+bug/2095224 (Functional jobs randomly timed out, possible load issues on the host ) *** perhaps duplicate of bug/2095196 ( https://bugs.launchpad.net/neutron/+bug/209519) * Medium ** https://bugs.launchpad.net/neutron/+bug/2094846 (neutron.tests.functional.services.trunk.drivers.ovn.test_trunk_driver.TestOVNTrunkDriver.test_subport_delete fails because remove_subports fails to update OVN db due to revision number inconsistency) *** Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095084 ([OVN] Error in ``_delete_router_remove_external_gateways``: "'NoneType' object has no attribute 'network_id'") *** https://review.opendev.org/c/openstack/neutron/+/939451 ** https://bugs.launchpad.net/neutron/+bug/2095097 (500 error creating resource with emoji for name) Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095152 (ovs-agent: Leftover tpi/spi interfaces after VM boot/delete with trunk port(s)) Unassigned ** https://bugs.launchpad.net/neutron/+bug/2095185 test_legacy_router_conntrack_helper failed as "neutron_lib.exceptions.BridgeDoesNotExist: Bridge test-bre0e2da16 does not exist" when router attempted to plug interface *** Unassigned, Was seen on stable, but can happen on master * Low ** https://bugs.launchpad.net/neutron/+bug/2095091 (test_metadata_ipv6_only_network failed looking for QUERY_MSG) Unassigned Lajos Katona (lajoskatona)

1 0

[glance][ops] proposal to remove the glance-scrubber
by Brian Rosmaita 19 Jan '25

19 Jan '25

Hello Glance users, The Glance project team wants to determine whether anyone is using the glance-scrubber utility [0]. If not, we are thinking of deprecating it this cycle to be removed in the appropriate future cycle. If you have not previously heard of the glance-scrubber, that's most likely because it's only useful for particular deployment configurations of which yours is not one. See the doc [0] for details if you are curious. Our suspicion is that the glance-scrubber may have outlived its usefulness for modern OpenStack configurations. (The last time I remember it being discussed at a PTG was during the snowstorm in Dublin, which I believe was the Rocky development cycle). On the other hand, the team also has some ideas for improving the scrubber, but there's no point doing that if it's not actually being used in the first place. So if removal of the glance-scrubber would negatively affect you, please respond to this email so we can get some data points. Thanks! [0] https://docs.openstack.org/glance/latest/cli/glancescrubber.html

2 2

[magnum] GPU support with magnum
by Satish Patel 18 Jan '25

18 Jan '25

Folks, We are running magnum with openstack to spin up k8s clusters and everything works as expected. Now I want GPU support so we can do some AI/ML stuff with k8s. Does magnum support GPU and how does it work? In the past I used GPU in passthrough mode to expose hardware to VMs but not sure how it works for k8s and what are the challenges?

5 16

[neutron][upgrades] Linuxbridge driver removal and migration
by Brian Haley 17 Jan '25

17 Jan '25

Hi Stackers, In 2022, during the Zed cycle, the Neutron team took a step at marking some items as experimental. One of those was the Linuxbridge driver. The reason was due to it being unmaintained for a number of cycles, and no one was able to take on its ownership. The implication of this decision was that it was consider deprecated, even if not given an official warning message. In doing this, all gate jobs were moved into the experimental queue, feature-parity with other drivers was deemed unnecessary, and bug fixing was considered best-effort. Since then, both major distros have stopped supporting new deployments using Linuxbridge, and have been mainly focused on OVN. So what does this mean? We (the neutron cores) have come to the conclusion that it is time to take the next step and remove the code from the tree [1]. After 5 cycles of being considered experimental without any investment it just makes sense. We plan on doing this in the current Epoxy cycle. We realize that there are still deployments using Linuxbridge, and hopefully they have been looking at migration strategies since we originally marked the code experimental. This email is another push to the user community to start looking at this, especially if you are planning any new deployments. There has been at least one migration guide written at [2] (thanks Jim!), and at this point I would encourage those still using Linuxbridge to look at it and start asking any questions they have so we have the chance to make it better and can make that process easier. Thanks for reading, -Brian, Neutron PTL [0] https://review.opendev.org/c/openstack/neutron/+/845181 [1] https://review.opendev.org/c/openstack/neutron/+/927216 [2] https://www.jimmdenton.com/migrating-lxb-to-ovn/

11 14

openstack ssl
by engineer2024 17 Jan '25

17 Jan '25

The present model for openstack API services uses keystone for authentication mechanism for the REST API. is it possible to test the api calls directly using the server/port of a API service, bypassing the keystone URI ? does this require any modification in the configuration of the openstack service ? Thanks

1 0

[neutron] Using address scopes in a DVR environment breaks floating IPs
by Lukas Steiner 17 Jan '25

17 Jan '25

Hi all, I hope to get some advice here if we’re doing something wrong or ran into a bug. We have an OpenStack environment (wallaby) with Distributed Virtual Router (DVR) enabled (compute nodes use DVR_NO_EXTERNAL agent mode, network nodes use DVR_SNAT) and we would like to onboard our provider network to a subnet pool part of an address scope to allow routed self-service networks. But as soon as the provider network is part of an address scope, the floating IP feature stops working. After debugging we found that adding the network to an address scope results in an additional set of firewall rules in the router and snat network namespaces which blocks cross-scope traffic. But floating IPs should be the exception to that and there are indeed rules that explicitly allow them by setting a mark in the iptables mangle table: -A neutron-l3-agent-floatingip -d <floating-ip>/32 -i qg-<random-id> -j MARK --set-xmark 0x4000000/0xffff0000 We can find these rules in the qrouter namespaces, but they’re missing in the snat namespaces. If we add the rule manually to the snat namespace, the floating IP is working again. If we switch the compute nodes to DVR agent mode, it also works, as the snat namespace is not used. I have already searched for bugs and changes in the DVR router code since wallaby but can’t find anything which is not already patched in wallaby. So, I’m wondering if we have an unsupported setup with combining dynamic routing with DVR and centralized network nodes or facing a long running neutron bug. Maybe someone has a similar setup or can confirm that this use case should be supported? Regards, Lukas

1 0

Re: Dalmatian Devstack: Empty Port Binding section in Horizon
by Lajos Katona 17 Jan '25

17 Jan '25

Hi, I checked and I hoped that this SDK patch : https://review.opendev.org/c/openstack/openstacksdk/+/915902 or SDK 4.2.0 is enough (for binding details at least) but something else is missing. Could you please open a bug report for this issue, I try to check where to fix it. Lajos Katona (lajoskatona) <berndbausch(a)gmail.com> ezt írta (időpont: 2025. jan. 14., K, 13:25): > Thanks much, Lajos! > > > > It was installed by Devstack with git_clone https://opendev.org/openstack/horizon.git > /opt/stack/horizon stable/2024.2. Furthermore, under /opt/stack/horizon/ > horizon.egg-info I find a file named pbr.json, which contains > > > > {"git_version": "4bcc3dcea", "is_release": false} > > > > I hope that makes it clear what I am using! Or is there another way of > obtaining the Horizon release number? > > > > Of the list of patches you shared, the ones that seem relevant (SDK for > neutron ports, SDK for neutron networks) were merged in August and June, > respectively, which makes me think that should be included in my release. > > > > And since this is an HTML message now, I am adding two screenshots. Not > only port binding info, but also network provider info is missing. I can > confirm that this is the admin user. > > > > Kind regards, > > > > Bernd > > > > > > *From:* Lajos Katona <katonalala(a)gmail.com> > *Sent:* Tuesday, January 14, 2025 5:22 PM > *To:* berndbausch(a)gmail.com > *Cc:* openstack-discuss <openstack-discuss(a)lists.openstack.org> > *Subject:* Re: Dalmatian Devstack: Empty Port Binding section in Horizon > > > > Hi Bernd, > > Shall I ask which version of Horizon do you use? Recently some patches > were merged for changing to SDK in the background: > > > https://review.opendev.org/q/project:openstack/horizon+topic:%22bug/1999774… > > And to be sure if you hit an issue from that change series or something > else. > > > > Best wishes > > Lajos Katona (lajoskatona) > > > > <berndbausch(a)gmail.com> ezt írta (időpont: 2025. jan. 14., K, 4:43): > > Is there a known bug, either in Devstack or Horizon, that prevents Neutron > port binding details from being displayed? My googling did not find > anything. > > As admin, I open a port's details, and in the Binding section at the end of > the page, all I get is a bold text "VNIC Type". The CLI command "openstack > port show" does provide the details. Also on a Xena Devstack, Horizon > displays them[1]. Horizon's setup.py seems to be identical on both > releases, > as far as Neutron configuration is concerned. > > Thanks for any input. > > Bernd. > > [1] The Binding details I expect: > > Binding > > VNIC Type Normal > Host xena-202 > Profile None > VIF Type ovs > VIF Details > connectivity l2 > port_filter True > ovs_hybrid_plug False > datapath_type system > bridge_name br-int > >

2 1