January 2025 - openstack-discuss - lists.openstack.org

[neutron] Drivers meeting canceled Friday, January 24th
by Brian Haley 24 Jan '25

24 Jan '25

Hello Neutrinos: I'm going to cancel tomorrow's drivers meeting as there is no agenda. Just a reminder there are still open specs that need review, list is at [0]. Have a nice weekend! -Brian [0] https://review.opendev.org/q/status:open+project:openstack/neutron-specs

1 0

[cinder] 2025.1 Midcycle reminder
by Jon Bernard 23 Jan '25

23 Jan '25

Hello all, this is a reminder that our midcycle meeting will take the place of next week's upstream meeting on 29 Jan. I've allocated 2 hours (as opposed to our usual 1) in case additional time is needed. The etherpad we'll be using is here [1]. See you all then! [1]: https://etherpad.opendev.org/p/cinder-epoxy-midcycles -- Jon

1 0

[Neutron][operators] LinuxBridge to OVN Outreach
by Daniel Failing 23 Jan '25

23 Jan '25

Dear operators of Neutron deployments using LinuxBridge, This mail is to target operators who plan to move away from neutron LinuxBridge as well. In our setup we are currently planing to move away from LinuxBridge and use OVN instead. Based on the last PTGs survey with quite a few deployments using LinuxBridge and it being marked experimental for some time, we feel there could be a common ground and pain points. We were wondering how many other operators have the same problems and want to share knowledge and if necessary consolidate code written for a potential migration. Knowing there are quite some different deployments, we most likely will not be able to find the golden solution that works for everyone, but we could strive for not making the same mistakes multiple times in different deployments. We are located in European timezones, if there are others interested in a zoom/jitsi call, please let me know your timezones/time preferences, so we find a good slot. Otherwise I can propose one of the following dates already: Friday, Jan 24st 2 or 3pm UTC Tuesday, Jan 28st 10am UTC Tuesday, Jan 28st 3:30pm UTC Wednesday, Jan 29st 10am UTC Friday, Jan 31st 2 or 3pm UTC Best, Daniel CERN Cloud Infrastructure [1] original thread: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

3 3

[openstack-ansible]:Wallaby to Xena Upgrade
by Rambo Rambo 23 Jan '25

23 Jan '25

Hi Team I am planning openstack-ansible wallaby to Xena upgrade. Distribution: openstack-ansible Release: Wallaby (current) ==> Xena (Planned) I was checking the release notes of Xena and could see that there is introduction of SSL in several services with nova for live migrations one of them. https://docs.openstack.org/releasenotes/openstack-ansible/xena.html As per the release notes: - Nova now defaults to to using the “QEMU-native TLS” feature for live migrations, rather than the deprecated SSH method. A pre-existing PKI (Public Key Infrastructure) setup is required. QEMU-native TLS requires all compute hosts to accept TCP connections on port 16514 and port range 49152 to 49261. More information can be found here: https://docs.openstack.org/nova/latest/admin/secure-live-migration-with-qem… Can you please confirm what steps are required to setup this PKI before doing the upgrade or it will be taken care by internal-pki using openstack-ansible upgrade method? Is there any option to skip this QEMU-native TLS for live-migrations and keep the orginal ssh based live-migrations? Regards

2 5

[all][oslo] revisiting the proposal to deprecate eventlet from oslo.service
by Mike Bayer 23 Jan '25

23 Jan '25

Hi - We've come to an impasse regarding our efforts to create a new backend selection system for oslo.service which will allow applications based on eventlet to have an incremental migration path where they can select for threads instead of eventlet [1] [2]. Briefly, the change organizes oslo.service into two different "backends", known as "eventlet" and "threaded". A calling application may choose to use either backend exclusively. Applications that use oslo.service right now are essentially using the "eventlet" backend. As these applications seek to migrate away from eventlet, they'd select the "threaded" backend and tune their applications to work under this new mode, where eventually the "eventlet" backend would be removed from oslo.service and ultimately from all of openstack entirely. All of that falls under things that have been agreed upon. The current impasse is among three options of how this "backend" system in oslo.service may be informed of which backend should be used: 1. being selected at runtime via CONF from the .cfg file , which is what was originally agreed upon 2. at start up time via environment variable 3. as a fixed setting in source code. Personally, as someone who has worked for decades on multiple switchable backend-oriented systems, I'm fully in the camp of choice 3, which is that switching the backend is just the beginning of a whole series of changes and tuning that a codebase would need in order to accommodate this backend change, and having the ability to "switch the backend back to eventlet" in CONF or runtime is simply not something anyone would want to do, unless a considerable amount of additional effort were made to support the codebase being reliable under both idioms. Which I would argue is a waste of effort since eventlet is going away. More concretely, if you took nova, which were using oslo.service with eventlet and decided to try the "threaded" backend, that's naturally going to have a lot of implications for other parts of the code. For example, if your code is setting eventlet monkey patching, that has to be turned off when this backend selection is made. If your code has other explicit use of eventlet, this also needs to be changed. Finally, threaded concurrency is naturally going to "feel" different than coroutine-based concurrency - context switches happen at different places and with different frequencies. Typically in a large codebase, the places that are prone to race conditions can shift; areas that had no races under coroutines now have threaded concurrency problems, areas that were sensitive to coroutine issues like being CPU bound are suddenly smooth with threading. The overall number of "working routines" can change, whereas eventlet can spin up 1000 coroutines like popcorn, under threads we now have to work with pools of threads typically a few dozen or so threads large. Overall, if I had my whole application passing under tempest under "threading", I would have had to make changes to get there, which may very well be mutually exclusive vs. test resiliency under eventlet. More generally, getting my application to run under threads represents a change in implementation that I expect to be permanent. If there are continued issues running under threaded mode, those are bugs that have to be fixed in any case. In the discussion at [1], Takashi Kajinami writes that a PTG discussion (for which I dont have any link or background) explicitly decided against the above idea: > In the initial design we considered that developer may make the decision to migrate from eventlet to the new mechanism, and didn't intend to provide any options to switch back to the original eventlet model. > However there was some strong push back during the previous PTG and people were concerned with having no mechanism to switch back to "previously worked well" eventlet model and to address the concern we agreed to introduce the option for operators to switch the backend. > If we don't allow users to configure the option then IMO we should not expose the option and we should use an internal flag instead (though that's basically what was disagreed with during the PTG, AFAIR) So first off, we have already realized that going with option 1. , use CONF, is not tenable, because the CONF collection is not typically in its final populated state when imports occur, and since the "backends" are an import time selection, there's a chicken/egg problem that cannot be resolved unless applications either highly modify the way in which their applications consume CONF vs. do their basic imports, or the way we are writing our backends in [1] needs to be modified so that imports can proceed using proxy objects which then change their loaded implementation when CONF is set up. Both of these options seem deeply complex and overarchitected in order to support the feature of "I want to change the backend in CONF" which I maintain is not going to be a real world use case. As a compromise, I suggested option 2. using an environment variable. This allows the backend selection to remain something that can be theoretically changed from the outside without modifying source code. However, current deployment practices suggest this is not really useful, as again per Takashi the way in which we deploy mod_wsgi does not allow environment variables to be local per virtual host; per [3] it looks like you need a hardcoded python script anyway that sets the env the way you want which makes it not too useful. As we were proceeding on option 2. we received some more pushback on the "env" idea from Arnaud Morin which I don't disagree with, and still more pushback and arguments leading in favor of 3, here from Jay Faulkner: > Using environment variables to set backends means that operators can change what backend is used. This is an easy thing for them to screw up and will end up creating a large amount of operator pain and bug churn. I am -1 in the strongest terms to the use of operator-adjustable values to enable/disable backends. The above was surprising considering that this was supposed to be a decision that had already been made at PTG, yet there still seems to be disagreement as though the decision were not actually made in any final way. So as I am tasked along with Daniel Bengtsson and Herve Beraud with getting [1] merged and moving onto building out the threaded backend, I would like to ask the group here to give me some background on the concerns raised at PTG and if we can all just here revisit the whole issue and hopefully decide that at least to start, let's get this merged without any CONF/env variable process (option 3); an application that's gone through the effort to transition to threads with the help of this backend selector should be assumed to be moving forward with that implementation, and if it has problems, that's just an ordinary bug like any other. thanks for reading! [1] https://review.opendev.org/c/openstack/oslo.service/+/935783 [2] https://review.opendev.org/c/openstack/oslo-specs/+/927503 [3] https://gist.github.com/GrahamDumpleton/b380652b768e81a7f60c

10 11

[kolla] Deprecating swift support
by Michał Nasiadka 23 Jan '25

23 Jan '25

Hello, Kolla and Kolla-Ansible is deprecating Swift deployment support, due to absence of active maintainers in the community for swift Kolla images and swift Kolla-Ansible role. If there are any volunteers to pick up that - please reach out either on #openstack-kolla or in the patch [1]. [1]: https://review.opendev.org/c/openstack/kolla-ansible/+/939968 Best regards, Michal

1 0

[watcher] Bug Triage Meeting - January 14, 2025, 12pm UTC
by Ronelle Landy 23 Jan '25

23 Jan '25

Hello All,We will be holding a IRC/Google Meet meeting on Tuesday, January 14, 2025 at 12pm UTC to start the triage process for the list of current Watcher bugs: <https://bugs.launchpad.net/watcher> <https://bugs.launchpad.net/watcher> - <https://bugs.launchpad.net/watcher>https://bugs.launchpad.net/watcher - https://bugs.launchpad.net/watcher-dashboard - https://bugs.launchpad.net/watcher-tempest-plugin Anyone interested in the Watcher project is invited to join the meeting on: - IRC channel #openstack-watcher - Google Meet link: meet.google.com/drk-bgbx-uyz Notes from the meeting will be captured on the IRC channel.Thank you

1 3

[manila] Removal of the Standalone CephFS NFS protocol helper in Epoxy
by Ashley Rodriguez 23 Jan '25

23 Jan '25

Hi zorillas! As per subject line, we will be deprecating the standalone CephFS NFS protocol helper in this cycle. This is following our deprecation warning we put out in 2024.1 / Caracal [1] [2]. We've supported using a clustered Ceph NFS service since the Zed release. We've also added upgrade helpers to migrate from a standalone NFS-Ganesha service to the ceph-orch-deployed clustered NFS service; this code was backported to stable/2023.1 (Antelope). Although the upgrade path is disruptive, there is a UX improvement in Manila to ease this migration through surfacing “preferred” export paths. Moving forward, deployments must have “cephfs_nfs_cluster_id” in your configuration, otherwise the driver will fail to start. Manila will not make any export changes to NFS-Ganesha, so upgrading manila without an NFS cluster will mean that only the control plane is not going to work; but, data plane will remain unaffected. To assist migration, admins can also configure "cephfs_ganesha_export_ips" (or alternatively, "cephfs_ganesha_server_ip") alongside "cephfs_nfs_cluster_id". Setting these options will allow the CephFS driver to report additional export paths. These additional export paths will have the "preferred" metadata key set to False. Best, Manila team [1] https://review.opendev.org/c/openstack/manila/+/912648 [2] https://docs.openstack.org/manila/2024.2/configuration/shared-file-systems/…

5 5

[Glance]Cancelling Today's Weekly Meeting
by Mridula Joshi 23 Jan '25

23 Jan '25

Hello, As PTL and I are not around today , hence skipping today's weekly meeting . If someone has anything to discuss, ping Abhishek or Cyril on the IRC channel . Thanks, Mridula Joshi

1 0

[security-sig][ops][neutron] OVN Security issue can impact OpenStack users
by Jay Faulkner 22 Jan '25

22 Jan '25

Hi all, A recent security issue with OVN recently came to light and was fixed in a recent release. Full details are here: https://www.openwall.com/lists/oss-security/2025/01/22/5 . OpenStack deployers running OVN are strongly recommended to upgrade to the new fixed version. No CVE number has been issued yet for this resolved vulnerability. As a note, this was originally reported to the VMT as a neutron bug, but after triage it was responsibly reported to OVN and has been held under embargo. Thank you to the folks who participated in this triage process, including the OVN community for fixing the issue. Please see https://bugs.launchpad.net/ossa/+bug/2088280 for full details on OpenStack handling of this security report. Thanks, Jay Faulkner OpenStack VMT

1 0