August 2025 - openstack-discuss - lists.openstack.org

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 02 Oct '25

02 Oct '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 11

[openstack][nova]qemu system wont release memory
by Nguyễn Hữu Khôi 29 Sep '25

29 Sep '25

Hello. I have an OpenStack instance that was using 14GB out of 16GB of memory. Later, when it was using only 4GB, I checked the QEMU process on the compute node, and it was still consuming 14GB, this means memory isn't released when usage decreases. Do we have solutions for this? I use Ubuntu 24.04 instances. Nguyen Huu Khoi

2 4

[Neutron][operators] LinuxBridge to OVN Outreach
by Daniel Failing 28 Sep '25

28 Sep '25

Hi all, Following on the good session we had last time discussing migrating out of LinuxBridge between operators, it was proposed to meet again in June. To find a good time slot, please respond to the following survey: https://newdle.cern.ch/newdle/TtbH8Dbv For reference, the notes from last meeting: https://etherpad.opendev.org/p/neutron-lb-ovn-migration Best, Daniel Failing CERN Cloud Infrastructure

2 3

[nova] Nova API call duration increases after upgrading to Caracal
by Chang Xue 21 Sep '25

21 Sep '25

Hello, we recently upgraded from Bobcat to Caracal. After upgrading, an interesting increment of duration of nova api call is observed. It's involved to multiple API calls, e.g. os-query-sets, /v2.1/servers/detail, etc. And it seems to be accumulated, meaning whenever nova-api is restarted, the duration seems to be fairly fast, giving response within like 80, 90 ms. As it keeps running for like up to 10 days, the duration increases to over 500 ms. I noticed that Caracal is using UnifiedLimitsDriver instead of DbQuotaDriver. I tried to change nova.conf to use DbQuotaDriver but it seems not working well with reducing the duration. I understand that calling to Keystone and Placement (and likely other external APIs) could result to longer duration, but pretty confused why it shows up as accumulated. Wondering if anyone else in Openstack community noticed the similar issue, thanks!

6 14

[oslo][messaging] Container fix of QManager breaks it for containers
by Dmitriy Rabotyagov 16 Sep '25

16 Sep '25

Hey folks, I wanted to raise attention to a relatively old issue, which has had a patch around since January [1]. Basically a regression [2] was introduced and backported to 2024.1 as part of an attempt to fix QManager execution in containers. However, it does not account for the possibility of os.getpgrp() returning 0 in some edge cases, which makes, for instance, impossible to sync database for Nova during upgrades, if QManager is used. Thus I'm asking to kindly review the [1] as it appears to be a blocker for some upgrades involving affected versions of oslo.messaging. I have not spotted this patch right away, so I proposed an alternative [3], but I'm happy to abandon it in favor of [1]. Thanks in advance for upcoming reviews :) [1] https://review.opendev.org/c/openstack/oslo.messaging/+/939540 [2] https://review.opendev.org/c/openstack/oslo.messaging/+/928034 [3] https://review.opendev.org/c/openstack/oslo.messaging/+/958528

2 2

[cinder] IBM SVf driver callhome
by Vivek Pandey 10 Sep '25

10 Sep '25

Hi Joe/ Brian, Since we didn’t get a chance to discuss the patch [IBM SVf driver] Adding support for callhome plugin, we can continue the conversation here. Patch : https://review.opendev.org/c/openstack/cinder/+/951829. We need callhome enabled as default for which we have introduced a parameter “callhome” in cinder.conf. This feature is to fecth the os_version, cinder-manage version and ispbhaenabled. Note: We are not fetching any customer’s sensitive data. Please let me know if you’ve had a chance to review NetApp’s ActiveIQ implementation. Netapp’s ActiveIQ code ref : https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/netap… As we have discussed during the 9th July cinder call, Anoop from NetApp confirmed that they are enabling it by default and EMS details are being captured from the logs. Based on that, do you think we can align or extend our current IBM callhome implementation (from the patch above) to follow a similar approach? Thanks, Vivek

5 8

[oslo] status of eventlet removal for oslo.service?
by Chris Friesen 04 Sep '25

04 Sep '25

Hi everyone, Looking at the "epoxy" spec for removing Eventlet from oslo.service (https://specs.openstack.org/openstack/oslo-specs/specs/epoxy/remove-eventle…) the very top line is a link to search blueprints.launchpad.net. Following that link it seems like there is no blueprint for removing eventlet from oslo.service, so does the spec itself (with the proposed milestones) represent the current plan? Separately, is there somewhere that shows the progress on the threading backend for oslo.service? I see it exists, are there any limitations on the current implementation? Thanks, Chris

2 1

[keystone][ops] Upgrade to 2025.1 and "corner cases"
by Dmitriy Rabotyagov 02 Sep '25

02 Sep '25

Hey folks, I wanted to raise awareness of operators regarding Keystone upgrade to Epoxy (2025.1), as found that existing release notes are slightly more vague about corner case definition. During analysis, it appeared that a lot of users (and under "a lot" I mean 4 figures number) in my really old deployments (which were deployed during Newton), still in use passwords hashed with sha512_crypt which has been dropped in 2025.1 [1]. As this was referenced as a "corner case" in upgrade docs, it may not bring enough attention to this matter reading notes, while sha512_crypt removal is mentioned in "Other" section which I personally missed when analyzing upgrade requirements. So I would highly recommend to ensure that all your users are using supported hashing algorithms before the upgrade to Epoxy, and inform them to reset their passwords if they don't. You can list users who are using unsupported hashing algorithm using following MySQL query: SELECT distinct l_u.user_id FROM keystone.local_user AS l_u JOIN keystone.password AS hash ON l_u.id = hash.local_user_id WHERE (hash.password_hash LIKE '$6$%' OR hash.password_hash LIKE '$5$%') AND hash.expires_at i s null; Hope this helps somebody to save some time in the future. [1] https://opendev.org/openstack/keystone/commit/4a4137d5f77ced3f4f011846fb9c2…

7 13

[Nova&Neutron] pagination returns an http, not https href link
by Chang Xue 02 Sep '25

02 Sep '25

Hello, we just migrated using new openstack CLI tools, and when using pagination for both Nova and Neutron, we noticed that they are returning http href link although they are expecting https, causing failure: ``` openstack port list Failure: Unable to establish connection to http://openstack....example.com:9696/v2.0/ports: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')). Retrying in 0.5s.1 retries left Unable to establish connection to http://openstack.....example.com:9696/v2.0/ports: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) ``` Is this a bug in Nova/Neutron? I tried other services like Heat, Keystone, etc, and they either returning https href links or just take relative path without schema/host. Thanks, Chang

3 10

[neutron][ovn] Logical Port Statistics
by James Denton 02 Sep '25

02 Sep '25

Hello all, Is anyone aware of a method to gather statistics (bytes in/out, packets, etc.) for logical router ports or similar when using ML2/OVN? I understand neutron-metering-agent parity is currently a feature gap, but I’m curious if there’s some OVN-native functionality that can provide those details. The ask is to provide a bytes-in/out counter for tenant routers, specifically on the external interface(s). Thanks, James Get Outlook for Mac <https://aka.ms/GetOutlookForMac>

3 3