- openstack-discuss - lists.openstack.org

Live-migration never completes memory copy
by Karl Kloppenborg 16 Sep '25

16 Sep '25

Hi Openstack Teams, We’re attempting to live-migrate instances off of a node, we continuously hit a timeout issue where memory copy doesn’t work: 2025-09-12 08:55:20.116 971204 INFO nova.virt.libvirt.driver [None req-250bc815-3323-4619-aef8-a11fdf92b27a 405874cabc5b4bf3912a5f89d54eb0d1 21eb701c2a1f48b38dab8f34c0a20902 - - default default] [instance: 2f496843-a2c4-48d4-bbdc-149a2ea76f1c] Migration running for 274 secs, memory 100% remaining (bytes processed=281505, remaining=8595255296, total=8604033024); disk 100% remaining (bytes processed=0, remaining=0, total=0). 2025-09-12 08:55:58.807 971204 INFO nova.virt.libvirt.driver [None req-250bc815-3323-4619-aef8-a11fdf92b27a 405874cabc5b4bf3912a5f89d54eb0d1 21eb701c2a1f48b38dab8f34c0a20902 - - default default] [instance: 2f496843-a2c4-48d4-bbdc-149a2ea76f1c] Migration running for 312 secs, memory 100% remaining (bytes processed=281505, remaining=8595255296, total=8604033024); disk 100% remaining (bytes processed=0, remaining=0, total=0). 2025-09-12 08:56:02.468 971204 INFO nova.compute.manager [None req-1f91a8d6-1fa7-47b2-8c3f-70925fb7a219 - - - - - -] [instance: 2f496843-a2c4-48d4-bbdc-149a2ea76f1c] During sync_power_state the instance has a pending task (migrating). Skip. 2025-09-12 08:56:37.985 971204 INFO nova.virt.libvirt.driver [None req-250bc815-3323-4619-aef8-a11fdf92b27a 405874cabc5b4bf3912a5f89d54eb0d1 21eb701c2a1f48b38dab8f34c0a20902 - - default default] [instance: 2f496843-a2c4-48d4-bbdc-149a2ea76f1c] Migration running for 352 secs, memory 100% remaining (bytes processed=281505, remaining=8595255296, total=8604033024); disk 100% remaining (bytes processed=0, remaining=0, total=0). 2025-09-12 08:57:17.377 971204 INFO nova.virt.libvirt.driver [None req-250bc815-3323-4619-aef8-a11fdf92b27a 405874cabc5b4bf3912a5f89d54eb0d1 21eb701c2a1f48b38dab8f34c0a20902 - - default default] [instance: 2f496843-a2c4-48d4-bbdc-149a2ea76f1c] Migration running for 391 secs, memory 100% remaining (bytes processed=281505, remaining=8595255296, total=8604033024); disk 100% remaining (bytes processed=0, remaining=0, total=0). 2025-09-12 08:57:56.877 971204 INFO nova.virt.libvirt.driver [None req-250bc815-3323-4619-aef8-a11fdf92b27a 405874cabc5b4bf3912a5f89d54eb0d1 21eb701c2a1f48b38dab8f34c0a20902 - - default default] [instance: 2f496843-a2c4-48d4-bbdc-149a2ea76f1c] Migration running for 431 secs, memory 100% remaining (bytes processed=281505, remaining=8595255296, total=8604033024); disk 100% remaining (bytes processed=0, remaining=0, total=0). Has anyone got insights for this issue? Your help is greatly appreciated. Thanks, Karl. Karl Kloppenborg Chief Technology Officer m: +61 437 239 565 resetdata.com<https://resetdata.com/> [cid:reset_69557fc2-1d63-4932-b5fd-93bd4f39ca7b.png] ResetData supports Mandatory Client Related Financial Disclosures – Scope 3 Emissions Reporting For more information on the phasing of these requirements for business please visit; https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…<https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…> This email transmission is intended only for the addressee / person responsible for delivery of the message to such person and may contain confidential or privileged information. Confidentiality and legal privilege are not waived or lost by reason of mistaken delivery to you, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. Whilst this email has been checked for viruses, the sender does not warrant that any attachments are free from viruses or other defects. You assume all liability for any loss, damage or other consequences which may arise from opening or using the attachments. If you received this e-mail in error please delete it and any attachments and kindly notify us by immediately sending an email to contact(a)resetdata.com.au<mailto:contact@resetdata.com.au>

4 8

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-2)
by Goutham Pacha Ravi 15 Sep '25

15 Sep '25

Hello Stackers, We're two weeks away from the date of the coordinated 2025.2 "Flamingo" release [1]. At this time, nearly all deliverables that follow the "cycle-with-rc" release model [2] have shipped their first release candidate. Over the next couple of weeks, OpenStack project teams will be expected to test, find and resolve any release blockers in these deliverables. The ongoing elections to fill four seats on the OpenStack Technical Committee and to elect a PTL for the OpenStack Dashboard ("Horizon") project team will conclude at 23:45 UTC on September 17, 2025 [3]. If you haven't already voted, please do so. There were no new governance proposals that were merged in the past week. === Weekly Meeting === The last weekly meeting of the OpenStack Technical Committee was held on September 9, 2025 [4]. We discussed project teams that had no PTL candidates in the ongoing renewal exercise: Vitrage, OpenStack Charms, Venus, and Monasca. You are all aware that we began the retirement process for the Monasca team [5] and are actively considering a maintainer request. We expect to make a decision past the 2025.2 "Flamingo" coordinated release. We have had a call for help [6][7][8] advertisement for the other projects and will discuss alternatives if those go unanswered. The TC discussed efforts to increase voter turnout for OpenStack elections by encouraging people to opt in to CIVS emails. A related topic was the reduced electorate, which was suspected to be due to OpenInfra Foundation (OIF) membership renewal requirements. The committee agreed to follow up on this after the elections conclude. We discussed default MySQL/MariaDB charset/collation changes that affect most OpenStack service projects. A few key points were discussed, including the need for a potential community goal to migrate from utf8 (which aliases to utf8mb3) to utf8mb4 due to MariaDB's plan to remove the utf8 alias. The meeting also addressed a known issue with Ironic, which has conflicts with the larger utf8mb4 type due to index size limitations. We decided to gather more data on what each project is currently using to better understand the scope of the problem. Dmitriy Robotyogev (noonedeadpunk) and Seunghun Lee (seunghunlee) volunteered to collect this information and add it to a new Etherpad [9] that will inform next steps. After the meeting, Jonathan Wright (JonathanWright[m]) from AlmaLinux joined us to discuss the potential for adding AlmaLinux to OpenStack's supported runtimes, noting its benefits like support for older CPUs (AlmaLinux 10 continues to support older CPUs like x86-64-v2) and more stable CI mirrors compared to CentOS Stream. This could be a significant benefit for OpenStack's CI (Continuous Integration) providers, as it would allow them to utilize their existing, aging hardware more effectively for testing. Community contributors from AlmaLinux want to improve its integration with OpenStack, including creating tailored cloud images for CI and contributing infrastructure resources. We agreed that a clear goal is to establish a working relationship with AlmaLinux volunteers to create and maintain CI images. This effort would likely involve a "pop-up team" to handle the initial implementation, with a long-term plan for AlmaLinux contributors to take ownership of the testing infrastructure. The primary objective is ensuring OpenStack services and deployments can be reliably tested and deployed on AlmaLinux, thereby expanding the project's supported user base. If you're interested to be a part of this effort, please chime in on the #openstack-tc channel on OFTC. The next meeting of the OpenStack Technical Committee is on Tuesday, September 16, 2025, at 1700 UTC. It will be hosted on the #openstack-tc IRC channel on OFTC. Please find the meeting's agenda on the OpenStack wiki [10]. I hope you'll be able to join us there. === Governance Proposals === ==== Open For Review ==== - Retire metalsmith project from bare metal program | https://review.opendev.org/c/openstack/governance/+/958297 - Retire Monasca project | https://review.opendev.org/c/openstack/governance/+/953671 - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 - Extend Monasca's inactive state | https://review.opendev.org/c/openstack/governance/+/959392 === Upcoming Events === - 2025-09-17: OpenStack TC and Horizon project elections conclude: https://governance.openstack.org/election/ - 2025-10-01: OpenStack 2026.1 "Flamingo" release day - 2025-10-03: OpenStack's Birthday Meetup, Austin: https://www.meetup.com/openinfra-austin/events/310508680 - 2025-10-17: OpenInfra Summit, Paris-Saclay: https://summit2025.openinfra.org/ - 2025-10-27: OpenInfra PTG: https://openinfra.org/ptg/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] "cycle-with-rc" release model: https://releases.openstack.org/reference/release_models.html#cycle-with-rc [3] OpenStack Elections: https://governance.openstack.org/election/ [4] TC Meeting IRC Log, Sep 9, 2025: https://meetings.opendev.org/meetings/tc/2025/tc.2025-09-09-17.00.html [5] Retire Monasca project: https://review.opendev.org/c/openstack/governance/+/953671 [6] PTL needed for OpenStack Vitrage: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [7] PTL needed for OpenStack Charms: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [8] PTL needed for OpenStack Venus: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [9] Dealing with Mariadb/Mysql changes: https://etherpad.opendev.org/p/gazpacho-collations-charsets [10] TC Meeting Agenda, Sep 16, 2025: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[neutron] Bug deputy report for week of September 8th
by Brian Haley 15 Sep '25

15 Sep '25

Hi, I was the Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * None High bugs --------- * https://bugs.launchpad.net/neutron/+bug/2122483 - [FT][OVN] ``TestOvn*Sync*`` test failing due to missing IDL lock - https://review.opendev.org/c/openstack/neutron/+/960384 * https://bugs.launchpad.net/neutron/+bug/2122429 - [logapi] ``_check_port_bound_sg`` called outside a READER context - https://review.opendev.org/c/openstack/neutron/+/960271 * https://bugs.launchpad.net/neutron/+bug/2122505 - [OVN][FT] The "Chassis_Private" register contains not expected values in the metadata tests - https://review.opendev.org/c/openstack/neutron/+/960369 * https://bugs.launchpad.net/neutron/+bug/2122524 - [FT] Random fails with ``test_ovs_bridge_name_not_set_if_integration_bridge_not_set`` - Needs owner * https://bugs.launchpad.net/neutron/+bug/2122525 - [FT] Test ``test_create_floatingip_port_forwarding_and_remove_subnets`` randomly failing - Needs owner * https://bugs.launchpad.net/neutron/+bug/2122651 - [FT] ``DBConnectionError`` exception during the ``OpportunisticSqlFixture`` cleanup process - https://review.opendev.org/c/openstack/neutron/+/960727 Medium ------ * https://bugs.launchpad.net/neutron/+bug/2122495 - [OVS Firewall] some flows, IPs from allowed_address_pairs, are dropped after neutron-openvswitch-agent service restart due to CT_MARK=0x1 - https://review.opendev.org/c/openstack/neutron/+/960602 * https://bugs.launchpad.net/neutron/+bug/2122606 - neutron-tempest-plugin-designate-scenario job fails to run due to over-restricted irrelevant files - https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/960659 * https://bugs.launchpad.net/neutron/+bug/2122706 - TestOVNNeutronAgentBase functional tests use system ovsdb - https://review.opendev.org/c/openstack/neutron/+/960892 Low --- * None Wishlist -------- * None

1 0

[neutron][taas][taap-as-a-service] moving client to seperate package
by Mohammed Naser 15 Sep '25

15 Sep '25

Hi there, It seems that at the moment, tap-as-a-service package contains both the server and the client components, which means that if you are trying to pull in the CLI components, you end up with Neutron as well, which is quite heavy. Is there a reason behind it that decision or is this something that someone needs to get around doing but no one has? Thanks, Mohammed

2 4

[all] No documentation for the API wsgi entry point of project APIs
by Thomas Goirand 13 Sep '25

13 Sep '25

Hi, As we're supposed to move away from uwsgi's config: wsgi-script=/usr/bin/<something> I'm switching all (-most every) Debian package to: module=<something-else> in the shipped <project>-api-uwsgi.ini file. In some case, that's fine, the release note tells where to find it. However, in a general way, it's documented nowhere but in the release note. In some cases, like for Magnum, the release note only tells that the wsgi-script is deprecated, without even telling to what I should switch. I've double-guessed that it should be: magnum.wsgi.api:application as this is what devstack/lib/magnum is doing. But I'm not sure, and I'll have to check for that fact after my package is built. Someone told me on IRC that I was right, and that this path is probably the "standard" for all projects. One shouldn't have to hunt for this kind of information. Could we agree on a *STANDARD* way to document this in every project? Or at least document somewhere OpenStack wide, that the standard path *must* be: magnum.wsgi.api:application and that distros and operators should expect it? Or is it that I missed the information and that's already documented? If that's the case, where is it? Cheers, Thomas Goirand (zigo)

4 4

[masakari]HACluster Containers Failing
by Thamanna Farhath 13 Sep '25

13 Sep '25

Hi OpenStack Community, I have installed *Masakari* using Kolla-Ansible, and after that I also installed *HACluster*. However, the HACluster containers are failing and keep exiting. Setup: - 3 controller nodes - Multiple compute nodes - Deployment: Kolla-Ansible My questions are: 1. Do I really need Masakari together with HACluster for instance HA? 2. Is there an alternative or recommended way to deploy HACluster with Kolla-Ansible? 3. Has anyone faced the issue where HACluster containers keep exiting after deployment? Any guidance, troubleshooting tips, or experiences would be very helpful Thanks, Thamanna Farhath N Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

2 1

[release] Release countdown for week R-2
by Thierry Carrez 12 Sep '25

12 Sep '25

Development Focus ----------------- At this point we should have release candidates (RC1 or recent intermediary release) for (almost) all the 2025.2 "Flamingo" deliverables. Teams should be working on any release-critical bugs that would require another RC or intermediary release before the final release. Actions ------- Early in the week, the release team will be proposing stable/2025.2 branch creation for all deliverables that have not branched yet, using the latest available 2025.2 "Flamingo" release as the branch point. If your team is ready to go for creating that branch, please let us know by leaving a +1 on these patches. If you would like to wait for another release before branching, you can -1 the patch and update it later in the week with the new release you would like to use. By the end of the week the release team will merge those patches though, unless an exception is granted. Once stable/2025.2 branches are created, if a release-critical bug is detected, you will need to fix the issue in the master branch first, then backport the fix to the stable/2025.2 branch before releasing out of the stable/2025.2 branch. After all of the cycle-with-rc projects have branched we will branch devstack, grenade, and the requirements repos. This will effectively open them up for 2026.1 "Gazpacho" development, though the focus should still be on finishing up 2025.2 "Flamingo" until the final release. Opening up the new development cycle is also a good opportunity to teams to revise their zuul jobs and used nodesets and clean up all unnecessary ones. For projects with translations, watch for any translation patches coming through and merge them quickly. A new release should be produced so that translations are included in the final 2025.2 "Flamingo" release. Finally, now is a good time to finalize release notes. In particular, consider adding any relevant "prelude" content. Release notes are targeted for the downstream consumers of your project, so it would be great to include any useful information for those that are going to pick up and use or deploy the 2025.2 "Flamingo" version of your project. Upcoming Deadlines & Dates -------------------------- Final RC deadline: September 26th, 2025 (R-1 week) Final 2025.2 "Flamingo" release: October 1st, 2025 OpenInfra Summit Europe: October 17th-19th, 2025 -- Thierry Carrez (ttx)

1 0

[nova] CPU extra flag invalid
by Kees Meijs | Nefos 12 Sep '25

12 Sep '25

Hi list, On a Compute Node we're seeing the following: > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service Traceback > (most recent call last): > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service File > "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line > 788, in _check_cpu_compatibility > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > self._compare_cpu(cpu, self._get_cpu_info(), None) > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service File > "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line > 9029, in _compare_cpu > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service raise > exception.InvalidCPUInfo(reason=m % {'ret': ret, 'u': u}) > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > nova.exception.InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have > compatibility. > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service 0 > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service Refer to > http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service During > handling of the above exception, another exception occurred: > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service Traceback > (most recent call last): > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service File > "/usr/lib/python3/dist-packages/oslo_service/service.py", line 807, in > run_service > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service service.start() > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service File > "/usr/lib/python3/dist-packages/nova/service.py", line 159, in start > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > self.manager.init_host() > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service File > "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 1421, > in init_host > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > self.driver.init_host(host=self.host) > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service File > "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line > 723, in init_host > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > self._check_cpu_compatibility() > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service File > "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line > 794, in _check_cpu_compatibility > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service raise > exception.InvalidCPUInfo(msg) > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > nova.exception.InvalidCPUInfo: Configured extra flag: ibpb it not > correct, or the host CPU does not support this flag. Please correct > the config and try again. Unacceptable CPU info: CPU doesn't have > compatibility. > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service 0 > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service Refer to > http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult > 2025-09-12 15:10:37.052 735327 ERROR oslo_service.service This is very interesting, since the CPU supports the ibpb flag for sure (i.e. /proc/cpuinfo cleary lists the feature). Manually adding: > <feature policy="require" name="ibpb"/> to the XML definition works and libvirt is happy. Is there some other name or alias for ibpb we should use? Cheers, Kees __ Kees Meijs BICT Nefos Cloud & IT <https://nefos.com/contact> Nefos IT bv Burgemeester Mollaan 34a 5582 CK Waalre - NL kvk 66494931 +31 (0)88 2088 188 <tel:+31882088188> nefos.com <https://nefos.com/contact> The information contained in this message is intended for the addressee only and may contain classified information. If you are not the addressee, please delete this message and notify the sender; you should not copy or distribute this message or disclose its contents to anyone. Any views or opinions expressed in this message are those of the individual(s) and not necessarily of the organization. No reliance may be placed on this message without written confirmation from an authorised representative of its contents. No guarantee is implied that this message or any attachment is virus free or has not been intercepted and amended. General terms and conditions ("The NLdigital Terms") apply to all our products and services.

1 0

[requirements]Flamingo pyroute2 regression
by Balazs Gibizer 12 Sep '25

12 Sep '25

Hi requirements folks, We identified a bug[1] in pyroute2 0.9.1 that causes intermittent failures during interface attach / detach operations. Originally we got the indication that the pyroute2 0.9.4 fixed the issue and our tests showed green results so we bumped to that version. But since then we observed that the problem still happens in 0.9.4 as well. Can we roll back our pyroute2 dependency to < 0.9 in Flamingo so close to the release? If not, can we later roll back the requirement on stable/2025.2? Or the only way is to roll forward on stable/2025.2 (and on master) when the fix appears in pyroute2? Thanks, gibi [1]https://bugs.launchpad.net/os-vif/+bug/2109396

6 10

[nova]Guidance on Instance Migration Issues in Production
by Thamanna Farhath 11 Sep '25

11 Sep '25

Hi All, I’m facing some issues with instance migration in production and need advice: Migrations are taking longer than expected—any tips to handle delays? When the host is fully utilized or overcommit ratio is high, migration sometimes fails with "valid host not found". Are there other common causes? Can we use a separate network interface for migration in OpenStack, and is it recommended? Any guidance or references would be appreciated. Thanks, Thamanna Farhath N https://www.facebook.com/Zybisys/ Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

2 1