- openstack-discuss - lists.openstack.org

[openstack-ansible][ussuri] moving openstack deployment node to other host
by Danish Khan 25 Sep '25

25 Sep '25

Dear Team, Our controller1 node which was also openstack deployment node somehow crashed and is not accessible now. I am trying to make controller3 as new deployment node. Below is the step which I am planning to follow. I have backup of (openstack-ansible, openstack_deploy and ansible directory of controller1 ): 1. Place /opt/openstack-ansible directory from controller1 to controller3 2. copy /etc/openstack_deploy directory from controller1 to controller3 3. copy /etc/ansible/ directory from controller1 to controller3 4. run bootstrap on controller3 5. test if openstack-ansible has picked same clusters and inventory. Can someone please suggest if this is correct approach or if I am missing something. Regard, Danish

2 1

[kolla] [swift] Seeking reviews for reinstating Swift support for 2025.2
by Amir Hossein Ahmadi 25 Sep '25

25 Sep '25

Hello Kolla and OpenStack community, A few months ago, Swift support was removed from the Kolla project due to several challenges, including the need for role modernization, the lack of dedicated maintainership, and persistent CI failures. Following that, I volunteered to take on this effort. Over the last few months, I have been working to address these issues and am happy to report significant progress: - *Modernized Roles:* The Swift roles have been fully updated to align with the latest Kolla and Kolla-Ansible development standards. - *Reliable CI:* The associated Zuul jobs are now consistently green, with all tests successfully validated. - *Maintainership:* I have stepped up to maintain the Swift service within Kolla, ensuring its long-term health and stability. A considerable amount of effort has been invested to ensure this implementation is robust and maintainable for the community. With the *2025.2 release deadline just four weeks away*, the final and most critical step is to get these patches reviewed and merged. Your feedback and reviews are crucial to landing this work in time for the release. Reinstating Swift support would be a valuable addition for many operators. Any time you can spare to look over the patches would be greatly appreciated. You can find the main patch series here: https://review.opendev.org/q/topic:%22dev-swift-modernization%22 Thank you for your consideration and for all your contributions to the OpenStack community. Best regards, Amir Hossein (IRC: amir58118)

5 5

[largescale-sig] Next meeting: September 23, 14utc
by Thierry Carrez 24 Sep '25

24 Sep '25

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC at 14UTC. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20250923T14 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

2 3

Upgrade Caracal --> Epoxy obstacles
by Eugen Block 24 Sep '25

24 Sep '25

Good morning, I just wanted to share some of our experience from this week when we upgraded our production cloud from Caracal to Epoxy (and also Ubuntu 22.04 --> 24.04). It wasn't as easy as previous upgrades, there were a couple of obstacles on the way. # RabbitMQ The main issue was rabbitmq, I'd say. Since the rabbitmq version was 3.9 on Jammy, we had to get to 3.12 for Noble. I had tried a lot of things in my test environment, in the end I decided to reduce the number of rabbitmq hosts to one, upgraded it with podman from 3.9 to 3.10 to 3.11 to enable the feature flags required for 3.12 (all still under Jammy). Then I could upgrade the OS to Noble so the package based rabbitmq-server would start successfully. # Database The recent thread about older password hashes (https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…) was helpful, we set the same passwords for those system users that came up in the MySQL query, and we didn't notice any issue. But there were two issues we had to deal with regarding the database: During nova-manage db sync we got these errors: 2025-09-22 14:08:19 2 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), fou nd type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB'). 2025-09-22 14:08:19 2 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255). This had not popped up during my tests, but our production database is around 10 years old when we started with our OpenStack journey, my test environment is quite new. What fixed it was: mysql_upgrade check-if-upgrade-is-needed Then there's another thing I have to fully grasp yet, but we now have a ".local" directory in /var/lib/mysql: find /var/lib/mysql/.local/share/containers/storage/ /var/lib/mysql/.local/share/containers/storage/ /var/lib/mysql/.local/share/containers/storage/libpod /var/lib/mysql/.local/share/containers/storage/db.sql Which mysql_upgrade didn't like. I assume that podman created that, but I'm not sure why (I only created a rabbitmq pod). Anyway, I removed that directory so I could proceed with the db sync commands to complete the upgrade process. In my test cloud I removed podman, then restarted galera and the directory is gone, so that's what I'm gonna do in the production as well. # opentack-client Then there are these messages during every openstack command: Could not load 'message_list': module 'zaqarclient.queues.v2.cli' has no attribute 'OldListMessages' Could not load 'message_post': module 'zaqarclient.queues.v2.cli' has no attribute 'OldPostMessages' I'll need to check if they impact any of our scripts, we don't use zaqar, but removing that package would also remove heat, so that won't be possible. # nova-compute: I waited for version 19.2.1 of librbd1 to be released, otherwise nova-compute would fail: qemu-block-extra : Depends: librbd1 (>= 19.2.1-0ubuntu0.24.04.1) but 19.2.0-0ubuntu0.24.04.2 is to be installed # neutron-metadata-agent There's a bug for functioning agents showing as dead, applying the mentioned patches fixes it: https://bugs.launchpad.net/neutron/+bug/2112492 # Pacemaker We use pacemaker for high availability and during the dist-upgrade I had to run 'apt --fix broken', then remove resource-agents and resource-agents-base to get through the dist-upgrade, then install pacemaker and crmsh again afterwards. Not sure if anyone will read this far, but I wanted to get that off my chest. ;-) So obviously, most of the issues weren't really OpenStack issues, but I wanted to share this experience anyway. If you read this far, thanks! ;-) Regards, Eugen

3 2

[kolla] Weekly meeting cancelled today (24th Sep 2025)
by Michał Nasiadka 24 Sep '25

24 Sep '25

Hello, Apologies for bad timing, but the weekly meeting is cancelled today due to lack of key participants. See you next week! Michal

1 0

[ops][largescale-sig][scientific-sig] Ops Radio Hour on September 26, 2025
by Ildiko Vancsa 23 Sep '25

23 Sep '25

Hi OpenStack community, I'm reaching out to you with a friendly reminder that the next OpenStack Ops Radio Hour[1] is scheduled to happen on __September 26th, 2025 at 1300UTC.__ The call series is a casual setting for operators, engineers, and contributors to come together, share knowledge, and collaborate on the complexities of managing and deploying OpenStack environments. Topics the next gathering will dive into: - (SWIFT) Compliance processes If you have further topics you would like to dive into during the call, please add them to this etherpad: https://etherpad.opendev.org/p/openstack_ops_radiohour A few reminders: - The group on the call agreed to try out Matrix as a chat platform for more synchronous communication outside calls + You can access the Matrix room here: https://matrix.to/#/#openstack-ops:opendev.org<https://matrix.to/#/%23openstack-ops:opendev.org> + Further details are here: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… - If you would like to read and catch up on notes from previous calls, you can find them in this etherpad: https://etherpad.opendev.org/p/openstack_ops_radiohour See you on the call on September 26th! Bets Regards, Ildikó [1] https://wiki.openstack.org/wiki/Operations/Meetups

1 1

[watcher] watcher and affinity policy
by Nguyễn Hữu Khôi 23 Sep '25

23 Sep '25

Hi, Would you like to correct me if I understand that watcher will not consider server groups when it does jobs? If I am right, do we have any solutions to this problem? Thank you. Regards Nguyen Huu Khoi

3 3

[tc][all] OpenInfra Summit 2025 Open Source Pavilion Opportunity
by Clark Boylan 23 Sep '25

23 Sep '25

The OpenInfra Summit 2025 will host an Open Source Pavilion booth space where open source projects can book an hour or two to showcase their work, engage with attendees, and share updates on their latest developments at the Summit. If you are interested in taking advantage of this opportunity for your project please fill out this form [0] and the event planners will get back to you with scheduling details. This seems like a good opportunity for both the broader OpenStack community to engage with attendees, and also for specific OpenStack projects, working groups, and SIGs. Feel free to followup with any questions you might have here on this thread, and I'll do my best to get answers. I suspect that you can also get the process started by filling out the form, and the organizers can followup with you to answer questions as well. [0] https://openinfrafoundation.formstack.com/forms/open_source_pavillion Clark

2 1

[designate] DNS records stuck in PENDING state but records created succesfully
by Jadon Naas 23 Sep '25

23 Sep '25

Hello! I am using Designate on 2023.2 with a PowerDNS backend. I have a recordset with three A records created through Designate. The A records are stuck in "PENDING" state with action "CREATE" or "UPDATE". The records were successfully created in PowerDNS, and changes are pushed to PowerDNS. However, the record status never finishes updating. Deleting and recreating the recordset results in the same situation. I do not see any notable errors in the logs for Designate or for PowerDNS. I have checked that the Desigate configuration for the nameservers and for the targets match the correct IP addresses for my PowerDNS server and for my Designate service hosts. Would folks have any suggestions for what I could do to determine what is preventing Designate from updating these records or to manually update the status of the records? Thank you! Jadon Naas

2 1

[publiccloud-sig] Public Cloud SIG meeting d.d. 23 September
by Kees Meijs | Nefos 23 Sep '25

23 Sep '25

Hello Stackers, Tomorrow (that's 23 September) the bi-weekly Public Cloud SIG meeting will be held again. The publicly available video meeting will be conducted from 07:00 hrs UTC (09:00 hrs CEST) and on here: https://meetpad.opendev.org/publiccloud-sig Additional information to be found on https://wiki.openstack.org/wiki/PublicCloudSIG See you tomorrow! Cheers, Kees __ Kees Meijs BICT Nefos Cloud & IT <https://nefos.com/contact> Nefos IT bv Burgemeester Mollaan 34a 5582 CK Waalre - NL kvk 66494931 +31 (0)88 2088 188 <tel:+31882088188> nefos.com <https://nefos.com/contact> The information contained in this message is intended for the addressee only and may contain classified information. If you are not the addressee, please delete this message and notify the sender; you should not copy or distribute this message or disclose its contents to anyone. Any views or opinions expressed in this message are those of the individual(s) and not necessarily of the organization. No reliance may be placed on this message without written confirmation from an authorised representative of its contents. No guarantee is implied that this message or any attachment is virus free or has not been intercepted and amended. General terms and conditions ("The NLdigital Terms") apply to all our products and services.

1 1