October 2025 - openstack-discuss - lists.openstack.org

[nova] [nova-scheduler] Performance Issue with CPU Pinning Scheduling and Duplicate IPs
by İzzettin Erdem 22 Oct '25

22 Oct '25

Hi, In our environment (Kolla-Ansible Ussuri, 1200+ hypervisors), we are experiencing performance issues during bulk VM deployments. When launching around 100 VMs, the operation takes approximately *15 minutes*. During this time, the *nova-scheduler* spends over *15 minutes calculating available pinned CPUs*. However, in our Yoga production deployment, the same operation is about *10 times faster* — around *1–2 minutes* for 100 VMs. Additionally, after the 100 VMs are created, we have observed that some instances could not be created and some of the rest receive *multiple (2–3) IP addresses* from Neutron instead of one. Our nova services are running in a *NUMA-aware, CPU-pinning* configuration, and other settings are almost identical to the default configuration. Below is an excerpt from the nova-scheduler logs during VM creation (this message is repeated thousands of times within 15 minutes): Computed NUMA topology CPU pinning: usable pCPUs: [[26, 90], [89, 25], [24, 88], [27, 91], [29, 93], [10, 74], [86, 22], [28, 92], [73, 9], [23, 87], [8, 72], [18, 82], [94, 30], [81, 17], [31, 95]], vCPUs mapping: [(0, 26), (1, 90), (2, 89), (3, 25), (4, 24), (5, 88), (6, 27), (7, 91), (8, 29), (9, 93), (10, 10), (11, 74), (12, 86), (13, 22), (14, 28)] Do you have any suggestions on what might be causing: 1. The scheduler to spend so much time calculating pinned CPUs in Yoga, and 2. Some VMs to get multiple IPs from Neutron after creation? Any guidance would be greatly appreciated. Best regards, İzzettin

1 0

[kolla-ansible] [nova] Database connection points to nova cell db
by İzzettin Erdem 22 Oct '25

22 Oct '25

Hi, In Kolla-Ansible deployments, the connection parameter under the [database] section in the nova.conf template points to the *nova_cell* database [1] <https://opendev.org/openstack/kolla-ansible/src/commit/4e3208b16c452d1cf2cd…>. As far as I understand, the *nova_cell* database is primarily used to store failed operations. I was wondering: if we are not using a multi-cell architecture, should the connection instead point directly to the *nova* database rather than *nova_cell*? There was an open bug related to this issue almost a year ago, but it has not received any updates yet [2] <https://bugs.launchpad.net/kolla-ansible/+bug/2088264>. Could you please confirm whether the current configuration is correct? We have deployed OpenStack using the existing configuration and it is currently running in production. If we change the database connection in the running environment, would that cause any issues? Thank you for your time and assistance. Best regards, İzzettin

2 1

[publiccloud-sig] Public Cloud SIG meeting d.d. 22 October
by Kees Meijs | Nefos 21 Oct '25

21 Oct '25

Hello Stackers, For whom I've met at the summit: it was great to be there and meet you all! Tomorrow (that's 22 October) the bi-weekly Public Cloud SIG meeting will be held again. The publicly available video meeting will be conducted from 07:00 hrs UTC (09:00 hrs CEST) and on here: https://meetpad.opendev.org/publiccloud-sig Additional information to be found on https://wiki.openstack.org/wiki/PublicCloudSIG Cheers, Kees __ Kees Meijs BICT Nefos Cloud & IT <https://nefos.com/contact> Nefos IT bv Burgemeester Mollaan 34a 5582 CK Waalre - NL kvk 66494931 +31 (0)88 2088 188 <tel:+31882088188> nefos.com <https://nefos.com/contact> The information contained in this message is intended for the addressee only and may contain classified information. If you are not the addressee, please delete this message and notify the sender; you should not copy or distribute this message or disclose its contents to anyone. Any views or opinions expressed in this message are those of the individual(s) and not necessarily of the organization. No reliance may be placed on this message without written confirmation from an authorised representative of its contents. No guarantee is implied that this message or any attachment is virus free or has not been intercepted and amended. General terms and conditions ("The NLdigital Terms") apply to all our products and services.

1 1

[nova] Memory performance issue while using property "hw:numa_nodes='2'" in flavors
by brijesh.mishra＠five9.com 21 Oct '25

21 Oct '25

Hi Team We have use Kolla ansible for openstack Caracal 2024.1 deployment. Having 2 production sites, we have observer that the Nova scheduler is allocating virtual machines (VMs) only on a single NUMA socket of the compute host. Since the flavor does not include the property hw:numa_nodes='2', vCPUs spread on two NUMA nodes scheduling is not enabled. As a result, all vCPUs and memory for a VM are placed on a single NUMA node, while the other NUMA socket(s) remain underutilized. So we tested this property in our test flavor and found that now scheduler is able to spread vcpu/vm across two numa sockets but performance on memory side is degraded for about 20% to 30%. So we are looking for suggestions on this if there is better way to accommodate more VMs or spread vcpu on two numa socket without having performance issues on memory or cpu side, or something which have less impact? Regards Brijesh

2 1

[Caracal]Optimizing Nova NUMA Scheduling Behavior (Single vs Dual NUMA Socket Allocation)
by Vish Mudemela 21 Oct '25

21 Oct '25

Hi Team, We are using Kolla Ansible for our OpenStack Caracal (2024.1) deployment across two production sites. We have observed that the Nova scheduler is currently allocating virtual machines (VMs) only on a single NUMA socket of the compute host. Since our flavors do not include the property hw:numa_nodes='2', the scheduler places all vCPUs and memory for a VM on a single NUMA node, leaving the other NUMA socket(s) underutilized. In our testing, we added the hw:numa_nodes='2' property to the flavor and confirmed that the scheduler can now distribute vCPUs across both NUMA sockets. However, this configuration led to 20–30% memory performance degradation, likely due to cross-NUMA memory access. We also noticed that once this property is set, the scheduler always attempts to spread vCPUs across two NUMA sockets — even when there are sufficient resources available on a single NUMA node. We are looking for a way to achieve conditional NUMA placement — ideally: - If a single NUMA node has enough capacity, schedule the VM entirely on that node. - Otherwise, allow the scheduler to spread the VM across multiple NUMA nodes. We would appreciate any suggestions or best practices to: Improve NUMA-aware scheduling efficiency,Better utilize both NUMA sockets without significant memory performance penalties, andAvoid forcing cross-NUMA allocation unless necessary. Has anyone in the community implemented or tuned Nova in a way that balances these requirements effectively? Thanks in advance for your insights and recommendations. Thanks

2 1

[Ceph Tracing] Limited Jaeger traces for OSD, no MON/MGR/RGW
by Thamanna Farhath 21 Oct '25

21 Oct '25

Hi Ceph Team, We’ve enabled Jaeger tracing in our Ceph cluster using: ceph config set global jaeger_tracing_enable true Currently, only OSD traces (e.g., enqueue_op, op-request-created) are visible, while MON, MGR, and RGW traces are missing. I noticed PR https://github.com/ceph/ceph/pull/59365 adds more tracers but hasn’t been merged yet. Could you please confirm: If this is an ongoing bug or current limitation? Whether there’s an alternative branch, backport, or workaround to achieve full tracing coverage? Appreciate your guidance. Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

2 3

[octavia] Octavia weekly IRC meeting cancelled 10/22
by Michael Johnson 20 Oct '25

20 Oct '25

As discussed at last week's meeting, we are cancelling this week's Octavia IRC meeting. We will see you at the PTG! Michael

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2026.1/R-23)
by Goutham Pacha Ravi 20 Oct '25

20 Oct '25

Hello Stackers, Over the weekend, the OpenInfra Summit in Paris-Saclay wrapped up. From what I heard, it was a successful event with a lot to unpack for the entire community. We'll discuss the summit over trip reports and activity across and within project teams over the coming weeks. We're twenty-three weeks away from the coordinated release of OpenStack 2026.1 "Gazpacho" [1]. Next week, the OpenStack community will be participating in the OpenInfra Virtual Project Teams Gathering (PTG). Project teams are grooming discussion topics on Etherpads [2]. The OpenStack Technical Committee will be meeting alongside project teams [3], and we'll kick off discussions as we've usually done by meeting with project maintainers. All project team leads, DPL liaisons, and core maintainers are invited to attend, and brainstorm community-wide concerns during this time slot. This event will be held at 1600 UTC on Tuesday, October 28th, 2025. Please see the PTG website for the call information, and better yet, pop into the #openinfra-events IRC channel on OFTC for real-time updates from all project discussions throughout the week. If you haven't already registered for the event, please do so [4]. This past week was a busy week for the community and the OpenStack Technical Committee. The "ironic-inspector" repositories were removed, and all the functionality provided has long been integrated into Ironic. Michał Nasiadka (mnasiadka) volunteered to be the TC liaison for the Requirements and Release Management teams. The TC also marked the Vitrage and Venus project teams as "inactive" [5]. These project teams need help. If you're a maintainer, contributor, or user, please help us re-form the project teams by determining project leadership and fixing up any failing CI jobs. All deliverables associated with these project teams will be dropped from the 2026.1 "Gazpacho" release cycle. If activity does not resume, we'll retire the deliverables and the respective project teams in the 2026.2 release cycle. This week, the TC will begin our first monthly meeting at an APAC-friendly time! As a globally distributed community, our preferred mantra of communication is asynchronous; however, once in a while, sync meetings help us make progress, and we'll try to cater to as many different timezones as possible. For now, on every third Tuesday of the month, we'll meet at 0800 UTC (instead of 1700 UTC) in the #openstack-tc channel on OFTC. At the same time, we'll be stopping our Video + IRC meeting for the rest of this release development cycle. Please download the meeting's ICS file and update your calendars [6]. === Weekly Meeting === The last weekly meeting of the OpenStack Technical Committee occurred on October 14th, 2025 [7]. The TC discussed several outstanding governance changes and gathered the roll call votes needed to mark Vitrage and Venus project teams inactive. We reiterated that the proposal to retire the Monasca project is still pending, but a current maintainer needs assistance with the release process, and Elod Illes (elodilles) would walk them through it. Michał Nasiadka (mnasiadka) volunteered to be the TC liaison for the Requirements and Release Management project teams. The TC agreed to sunset scheduled video meetings, opting to bring them back only on an ad-hoc basis due to lack of interest. The remaining portion of the meeting focused on establishing a new, more EU/APAC-friendly meeting time. We decided on doing this monthly on the third Tuesday at 08:00–09:00 UTC, replacing the regular 17:00 UTC slot for that day. We also called for topics for the upcoming PTG (Project Team Gathering) [3] and briefly reviewed the runtime and job changes for the Gazpacho release, which includes making the Python 3.13 job voting, creating a non-voting Python 3.14 job, and fixing up the grenade templates [8]. The next meeting of the TC is on October 21st, 2025, at 0800 UTC. This meeting will be hosted in the #openstack-tc channel on OFTC, and mnasiadka will be chairing it. Please find the agenda on the meeting's wiki page [9]. Please chime in and say hi. I do hope this new meeting slot is feasible for contributors in the Eastern Hemisphere. Do remember, you can always edit the wiki and propose new meeting topics. === Governance Proposals === ==== Merged ==== - Update TC liaisons for Freezer and Oslo | https://review.opendev.org/c/openstack/governance/+/964257 - Retire ironic-inspector from bare metal program | https://review.opendev.org/c/openstack/governance/+/963470 - Mark Vitrage project team inactive | https://review.opendev.org/c/openstack/governance/+/963227 - Mark Venus project team inactive | https://review.opendev.org/c/openstack/governance/+/963228 - Propose mnasiadka as tc-liaison for Requirements and Release Management | https://review.opendev.org/c/openstack/governance/+/963989 ==== Open For Review ==== - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 - Retire Monasca project | https://review.opendev.org/c/openstack/governance/+/953671 === Upcoming Events === - 2025-10-22: OpenInfra Meetup, Jakarta Selatan: https://www.meetup.com/openinfraid/events/31140538 - 2025-10-27: OpenInfra PTG: https://openinfra.org/ptg/ - 2025-11-04: OpenInfra Board meeting: https://board.openinfra.org/ - 2025-11-15: Celebrating 15 years of OpenStack, Montréal - 2025-11-15: KCD + OpenInfra Days, Hangzhou: https://sessionize.com/kcd-hangzhou-and-oicd-2025/ === How to Contact the TC === You can reach the TC in several ways: 1. Email: send an email with the tag [tc] on this mailing list. 2. Ping us using the 'tc-members' keyword on the #openstack-tc IRC channel on OFTC. 3. Join us at our weekly meeting: The Technical Committee meets every Tuesday at 1700 UTC, except the third Tuesday of the month, when we meet at 0800 UTC [9]. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2026.1 "Gazpacho" Release Schedule: https://releases.openstack.org/gazpacho/schedule.html [2] 2026.1 PTG Etherpads: https://ptg.opendev.org/etherpads.html [3] Openstack TC 2026.1 PTG Etherpad: https://etherpad.opendev.org/p/oct2025-ptg-os-tc [4] Register to attend the OpenInfra PTG: https://ptg.openinfra.org/ [5] Inactive project teams: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [6] Download the TC meeting ICS: https://meetings.opendev.org/#Technical_Committee_Meeting [7] TC Meeting IRC Log, Oct 14th 2025: https://meetings.opendev.org/meetings/tc/2025/tc.2025-10-14-17.00.html [8] 2026.1 runtime test changes: https://review.opendev.org/q/hashtag:%222026-1-runtime-update%22 [9] TC Meeting Agenda, Oct 21st 2025: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[PTG] Glance cross-project sessions with Cinder/Nova
by Cyril Roelandt 20 Oct '25

20 Oct '25

Hello, We would like to host two cross-project sessions regarding the following topics: - Image encryption and disk format enforcement[1]: this has implicatoins for both Nova and Cinder. This should probably be a one hour session. - Make multiattach workflow compatible with multiple glance workers: this is related to both Cinder and Glance. This could be a half hour or full hour session, depending on what Rajat has in mind. Glance is very flexible with its scheduling: we will be busy during the eventlet session on Tuesday between 15:00 UTC and 16:00 UTC, otherwise we're pretty much free every day between 13:00 UTC and 17:00 UTC. Ideally we would host these sessions on Wednesday or Thursday. What would work for members of the Nova and Cinder teams? On behalf of the Glance team, Cyril Roelandt [1] https://etherpad.opendev.org/p/2026.1-ptg-glance-planning#L128 [2] https://etherpad.opendev.org/p/2026.1-ptg-glance-planning#L140

1 0

[Ironic] Outreachy - Starting work on Bug #2072349: Rewrite page headings
by abhi pc 20 Oct '25

20 Oct '25

Hello Ironic Team and Documentation folks, I am an Outreachy applicant and I would like to take on Bug #2072349, "Rewrite page headings," as my first contribution. I understand this task involves applying the documentation style rules (gerunds for tasks, labels for reference, etc.) and will be a site-wide change. *Could someone please confirm if this bug is still active and suitable for an Outreachy applicant to start on?* I am planning to start by reviewing the documentation structure in the doc/source/ directory and will submit the changes in a series of small, logical patches. Thank you for your guidance! Abhijith P C

1 0