- openstack-discuss - lists.openstack.org

[tc][all] No weekly IRC meeting on January 6 2026
by Goutham Pacha Ravi 06 Jan '26

06 Jan '26

Hello Stackers, A number of TC members are still out on PTO or observing public holidays in their respective countries, so we’ll be skipping the weekly IRC meeting this week, and the next meeting will be on January 13th 2026 at 1700 UTC in OFTC’s #openstack-tc channel. Thank you, Goutham Pacha Ravi (gouthamr) Openstack TC Chair

1 0

[openstacksdk][keystoneauth1][nova] Memory leak in nova-scheduler, possibly all services?
by Clippinger, Sam 05 Jan '26

05 Jan '26

Hello everyone, We've been seeing steady memory leaks in some of our OpenStack API services. I think I've found the cause but I'd appreciate some feedback. When a service creates a client object (e.g. placementclient) to use another service's API, the call goes through Proxy.request() in the openstacksdk module. That function calls _report_stats()[1] for every request which calls three other functions for statsd, prometheus and influxdb. The prometheus function records the number of requests and response times in a dict using the full URL of the request.[2] This data is recorded unless the _prometheus_counter or _prometheus_histogram member variables are None.[2] Those member variables are set by config.CloudRegion.get_prometheus_counter()[3] and config.CloudRegion.get_prometheus_histogram()[4], which do not check any config settings. It looks like the only way to prevent this behavior is to uninstall the prometheus_client module.[5] Unfortunately prometheus_client is required by oslo.metrics[6] which is required by oslo.messaging[7]. This is causing a lot of extra memory usage in our production environments. It's worst in our build farm where we create over 500K VMs per week. nova-scheduler queries a lot of unique URLs because the Placement API uses the VM's UUID to get allocations[8] and each URL gets its own counter and histogram. Even though we run 6 copies of nova-scheduler (currently Caracal) after a few weeks each copy will reach 8 GB and the worker processes will start getting OOMkilled, putting the new VM in error state. The nova-scheduler parent process keeps running and restarts the workers, which just keep getting killed and VMs can't be reliably scheduled until nova-scheduler is restarted. This feels like two problems to me. We don't need Prometheus metrics from the OpenStack services but there's no way to turn them off. The Session class in keystoneauth1 supports a collect-timing option[9] (default False) but it's only used in one or two places that I can find. The CloudRegion methods get their config from the Adapter[10] class in keystoneauth1, which does not support collect-timing. Should it? Even if we did use these Prometheus metrics, it seems like they shouldn't be allowed to grow unbounded forever. Should the Proxy._report_stats_prometheus() function[2] limit the metrics by age or quantity? Any thoughts? I've patched our services in-house to turn off these metrics (I'll have results in a week). In the meantime, are we doing something wrong or is every OpenStack service always collecting these metrics constantly? -- Sam Clippinger 1: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/prox… 2: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/prox… 3: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/conf… 4: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/conf… 5: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/conf… 6: https://github.com/openstack/oslo.metrics/blob/stable/2025.2/requirements.t… 7: https://github.com/openstack/oslo.messaging/blob/stable/2025.2/requirements… 8: https://docs.openstack.org/api-ref/placement/#list-allocations 9: https://github.com/openstack/keystoneauth/blob/stable/2025.2/keystoneauth1/… 10: https://github.com/openstack/keystoneauth/blob/stable/2025.2/keystoneauth1/… ________________________________ CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.

4 4

[puppet-cinder,glance,manila] Resigning from core team
by Alan Bishop 05 Jan '26

05 Jan '26

Hello, Due to a change in my work status, I am no longer able to fulfill my obligations as a core reviewer. With this in mind, I am resigning from the core team on the puppet-cinder, puppet-glance and puppet-manila projects. Alan Bishop

3 2

[release] Release countdown for week R-12, Jan 05 - Jan 09
by Elõd Illés 05 Jan '26

05 Jan '26

Development Focus ----------------- The Gazpacho-2 milestone is *this* week, on January 8th, 2026! 2026.1 Gazpacho-related specs should now be finalized so that teams can move to implementation ASAP. Some teams observe specific deadlines on the second milestone (mostly spec freezes): please refer to https://releases.openstack.org/gazpacho/schedule.html for details. General Information ------------------- Libraries need to be released at least once per milestone period. Today, the release team will propose releases for any library that has not been otherwise released since milestone 1. PTL's and release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well. A +1 would be appreciated, but if we do not hear anything at all by the end of the week, we will assume things are OK to proceed. Remember that non-library deliverables that follow the cycle-with-intermediary release model should have an intermediary release before milestone-2. Those who haven't will be proposed to switch to the cycle-with-rc model, which is more suited to deliverables that are released only once per cycle. This week is also the deadline to freeze the contents of the final release. All new '2026.1 Gazpacho' deliverables need to have a deliverable file in https://opendev.org/openstack/releases/src/branch/master/deliverables/gazpa… and need to have done a release by milestone-2. Changes proposing those deliverables for inclusion in 2026.1 Gazpacho have been posted, please update them with an actual release request before the milestone-2 deadline if you plan on including that deliverable in 2026.1 Gazpacho, or -1 if you need one more cycle to be ready. Upcoming Deadlines & Dates -------------------------- Gazpacho-2 Milestone: January 8th, 2026 Előd Illés irc: elodilles @ #openstack-release

1 0

[Keystone][ldap] Keystone bind request to ldap not able to interpret if the user's password is expired.
by Sharath Ck 05 Jan '26

05 Jan '26

Hi, I have configured Keystone with 389ds as backend. 389ds server is configured with password policy and pwdmustchange is set to true. However on first login attempt from keystone to request token for user in 389ds is resulting in successful bind request. Perhaps, 389ds is providing control ID - control: 2.16.840.1.113730.3.4.4 in the response. But keystone is not able to interpret the control ID and provide a token with full access. Is it not possible for keystone to respond token with minimum scope to only change password? Or any response attribute to specify password is expired and needs to be changed? Kindly help with any configuration that may interpret the password expiry of ldap user and can be used in token response. P.S: configuring user_enabled_attribute to check nsAccountLock will not work as ldap user's password would be expired but not locked. user_enabled_attribute = nsAccountLock user_enabled_mask = 0 user_enabled_invert = true user_enabled_default = true Regards, Sharath

3 2

[ops][cinder][show-and-tell] I wrote an open-source tool to automate OpenStack Cinder snapshot lifecycles
by aravindhstanley＠gmail.com 05 Jan '26

05 Jan '26

Hello folks, We have been using OpenStack for a while and needed a reliable way to keep snapshots on hot storage for compliance and quick recovery. While Cinder provides the mechanism to create snapshots, the actual scheduling and rotation are largely left for the operator to figure out. I know there are robust commercial tools like Trilio (awesome tool, by the way), but I wanted a lightweight, open-source alternative to manage the snapshot lifecycle without the extra overhead. So, I built SnapSentry. It is written in Go, completely free, and MIT licensed, so I thought I should share it with the community. Key Features: Metadata-Driven: No central database or config files; just tag your volumes with metadata and it works. (Helper command takes care of metadata updates) Atomic-ish VM Snapshots: Automatically identifies all volumes attached to a specific VM and snapshots them in parallel to minimize consistency gaps. Smart Concurrency: Uses a hybrid model (Parallel for attached volumes, Sequential for unattached) to prevent API throttling. Resilient: Handles transient OpenStack errors (500s) and cleans up orphaned "zombie" snapshots automatically. Secure: Supports restricted Application Credentials for least-privilege operation. Here is the repo: https://github.com/aravindh-murugesan/openstack-snapsentry-go Cheers, and feedback is welcome! Note: I do use AI assistant for docs, but the logic or the app itself is not vibe-coded. I'm not sure if this is the right place to post this, if it's not, I apologize. Just trying to contribute to the openstack ecosystem :)

2 1

[tacker] Cancelling weekly IRC meeting
by Yasufumi Ogawa 04 Jan '26

04 Jan '26

Hi team, Sorry for the late notice. I'd like to cancel the IRC meeting today as we agreed in the previous meeting. Regards, Yasufumi

1 0

[Opensource Tool] Cinder snapshot lifecycle management
by Aravindh M 03 Jan '26

03 Jan '26

Hello folks, We have been using OpenStack for a while and needed a reliable way to keep snapshots on hot storage for compliance and quick recovery. While Cinder provides the mechanism to create snapshots, the actual scheduling and rotation are largely left for the operator to figure out. I know there are robust commercial tools like Trilio (awesome tool, by the way), but I wanted a lightweight, open-source alternative to manage the snapshot lifecycle without the extra overhead. So, I built *SnapSentry*. It is written in Go, completely free, and MIT licensed, so I thought I should share it with the community. *Key Features:* - *Metadata-Driven:* No central database or config files; just tag your volumes with metadata and it works. (Helper command takes care of metadata updates) - *Atomic-ish VM Snapshots:* Automatically identifies all volumes attached to a specific VM and snapshots them in parallel to minimize consistency gaps. - *Smart Concurrency:* Uses a hybrid model (Parallel for attached volumes, Sequential for unattached) to prevent API throttling. - *Resilient:* Handles transient OpenStack errors (500s) and cleans up orphaned "zombie" snapshots automatically. - *Secure:* Supports restricted Application Credentials for least-privilege operation. Here is the repo: https://github.com/aravindh-murugesan/openstack-snapsentry-go Cheers, and feedback is welcome! Note: I do use AI assistant for docs, but the logic or the app itself is not vibe-coded. I'm not sure if this is the right place to post this, if it's not, I apologize. Just trying to contribute to the openstack ecosystem :)

1 0

[neutron] bug deputy report for week of 2025-12-15
by Bence Romsics 02 Jan '26

02 Jan '26

Hi Team, Here's the deputy report for the week of 2025-12-15. Sorry for being so slow! High: * https://bugs.launchpad.net/neutron/+bug/2136827 Functional tests using SqlFixture are extremely slow fix in progress: https://review.opendev.org/c/openstack/neutron/+/971472 * https://bugs.launchpad.net/neutron/+bug/2136733 The highest priority chassis gateway port fails to bind after chassis was restarted/crashed wip fix: https://review.opendev.org/c/openstack/neutron/+/971876 Medium: * https://bugs.launchpad.net/neutron/+bug/2136747 [FT] "ResourceWarning: unclosed file" errors in the FT execution unassigned, but Jakub investigated a bit * https://bugs.launchpad.net/neutron/+bug/2136147 test_two_vms_fips test has very specific expectations from availability zones fix in progress: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/970865 * https://bugs.launchpad.net/neutron/+bug/2136864 [ovn-db-sync] sync_hostname_and_physical_networks is expecting to find Chassis among agents fix in progress: https://review.opendev.org/c/openstack/neutron/+/971527 Incomplete: * https://bugs.launchpad.net/neutron/+bug/2136466 [rbac] Unexpected behaviour on floating ip port forwarding list Regards, Bence

1 0

need help in managing production openstack cluster
by Tarun 31 Dec '25

31 Dec '25

Hi, I am migrating from VMware to Openstack 2025.2 Please help me to validate my production cluster to avoid any future issue Thanks Tarun

3 3