September 2024 - openstack-discuss

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-1)
by Goutham Pacha Ravi 22 Sep '24

22 Sep '24

Hello Stackers, We're in the home stretch of the 2024.2 ("Dalmatian") release cycle this week [1]. OpenStack services will be tagging their final release candidates in preparation for the final release next week. Since the release train never stops, I'd direct your attention to the 2025.1 ("Epoxy") release cycle's schedule [2], active feature development has already begun across the board. We concluded elections for four seats on the OpenStack Technical Committee last week [3]. I'm pleased to welcome first-time TC members Doug Goldstein (cardoe) and Sylvain Bauza (bauzas), and welcome back Ghanshyam Mann (gmann) and Dr. Jens Harbott (frickler). I'd also like to extend congratulations to the PTLs of 37 project teams who volunteered to lead their teams. Most of these leaders have served in the past cycle, and on behalf of the TC, I'd like to thank them for their dedication. I also welcome Guillaume Boutry (gboutry), who will be the PTL for the Sunbeam project for the first time. Last week, the TC merged a goal proposal to move all Ubuntu-based testing in the Zuul CI system to the latest LTS release: 24.04 LTS (Noble Numbat). Ghanshyam Mann (gmann) is expected to lead this goal and is asking project teams to help resolve test failures and incompatibilities when transitioning both unit and functional test jobs to this distribution. This goal may be selected for implementation during the 2025.1 ("Epoxy") release cycle. Several minor governance changes were also merged and proposed, as listed below. === Weekly Meeting === The OpenStack TC weekly meeting was held over IRC on 2024-09-17. The IRC meeting logs are available on eavesdrop [4]. We continued to populate topics for the upcoming virtual Project Teams Gathering [5] and began addressing leadership appointments for two projects with volunteers (Swift and Mistral). We're also checking the community's interest in maintaining the "Watcher" project [6], and waiting on the Tacker team to drop its dependency on "kuryr-kubernetes" so that project can be retired. Amy Marrich (spotz) and Sylvain Bauza (bauzas) recapped the OpenInfra Summit in Suwon, noting the packed rooms and commending the organizers for a great job. Feedback from hallway discussions highlighted the need for improved documentation translation, as the lack of translated materials makes it difficult for new users and contributors, especially in non-English-speaking regions. We’ll discuss this further at the PTG in October. bauzas also raised the need for better tracking and communication regarding the etherpads used in the community for various tasks and contributor work. The next OpenStack TC meeting will be held over IRC in OFTC's #openstack-tc channel on 2024-09-24 at 1800 UTC. I hope you'll be able to attend and participate in the discussions. The agenda for the meeting is on the meeting wiki [7]. Please remember that you can always edit the wiki and propose new topics. === Governance Proposals === ==== Merged ==== - Add goal to migrate CI/CD jobs to Ubuntu Noble | https://review.opendev.org/c/openstack/governance/+/928078 - PTI: Project-first wording | https://review.opendev.org/c/openstack/governance/+/929644 - Charms: Retire unmaintained charms | https://review.opendev.org/c/openstack/governance/+/926793 - Add TC/PTL results from the 2025.1 election | https://review.opendev.org/c/openstack/governance/+/929940 ==== Open for Review ==== - Appoint Tim Burke as PTL for Swift | https://review.opendev.org/c/openstack/governance/+/928881 - Add Axel Vanzaghi as PTL for Mistral | https://review.opendev.org/c/openstack/governance/+/927962 - Retire all single charm repositories | https://review.opendev.org/c/openstack/governance/+/903490 - Add gouthamr nomination for 2025.1 TC chair | https://review.opendev.org/c/openstack/governance/+/929971 - Add chair nomination folder for 2025.1 | https://review.opendev.org/c/openstack/governance/+/929970 - Select community-wide goal: Migrate CI/CD jobs to Ubuntu Noble | https://review.opendev.org/c/openstack/governance/+/929703 - Mark kuryr-kubernetes and kuryr-tempest-plugin inactive | https://review.opendev.org/c/openstack/governance/+/929698 === Upcoming Events === - 2024-10-08: OpenInfra Monthly Board Meeting: https://board.openinfra.dev/ - 2024-10-15: OpenInfra Days NA, Indianapolis: https://ittraining.iu.edu/explore-topics/titles/oid-iu/ - 2024-10-21: OpenInfra Project Teams Gathering: https://openinfra.dev/ptg/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2024.2 "Dalmatian" Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] 2025.1 "Epoxy" Release Schedule: https://releases.openstack.org/epoxy/schedule.html [3] 2025.1 TC and PTL Election Results: [4] TC Meeting IRC Log, 2024-09-17: https://meetings.opendev.org/meetings/tc/2024/tc.2024-09-17-18.00.log.html [5] 2025.1 OpenStack TC/PTG planning etherpad: https://etherpad.opendev.org/p/oct2024-ptg-os-tc [6] Retirement of openstack/watcher: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [7] TC Meeting Agenda, 2024-09-24: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee

1 0

[upgrade][openstack] OpenStack Upgrade Procedure
by Ammar Al-Sayegh 21 Sep '24

21 Sep '24

Hello, I need to upgrade an OpenStack installation for the first time. The system Admin who performed the original installation is no longer available. How can I determine the current OpenStack version, modules installed, type of installation, and networking paradigm from OpenStack Horizon? Once determined, where I can find the appropriate upgrade procedure? Thank you. -ammar

2 1

[networking-mlnx] Project Health
by William Szumski 20 Sep '24

20 Sep '24

Hi all, I've noticed that there hasn't been any stable releases of networking-mlnx since stable/zed and was just wondering if this project still has any active maintainers. I would be interested in helping to keep this project going, so please give me a shout if you require any assistance. Thanks for all your hard work, Will

3 3

[unmaintained][cinder] question about zuul jobs for unmaintained branches
by Brian Rosmaita 20 Sep '24

20 Sep '24

Hello Unmaintained maintenance team, I was reviewing a cinder-tempest-plugin patch (which is in an un-branched repository), and noticed that we have a bunch of Zuul cinder jobs defined for stable branches that no longer exist, for example, stable/wallaby [0]. Where this impacts the unmaintained branches is that, for example, unmaintained/wallaby's .zuul.yaml is configured to run (for example) cinder-tempest-plugin-lvm-lio-barbican in the check [1], but you can see on this review, it's not actually running: https://review.opendev.org/c/openstack/cinder/+/926279?tab=change-view-tab-… So, my question is what to do about these unused jobs: (a) delete them (b) only delete stable/x for x <= ussuri, and update the jobs for stable/x for victoria <= x <= zed to unmaintained/x (c) delete them all, but propose a patch to the appropriate unmaintained branches zuul.yaml files that contain the job definition so that the Unmaintained team can make any changes needed to get them running without having to go through the cinder-tempest-plugin-core team's review/approval process (d) something I haven't thought of Thanks! brian [0] https://opendev.org/openstack/cinder-tempest-plugin/src/branch/master/.zuul… [1] https://opendev.org/openstack/cinder/src/branch/unmaintained/wallaby/.zuul.…

2 1

New to OpenStack. Test environment with 1 control node and 2 compute nodes. Unable to migrate or resize instances
by collinl＠churchofjesuschrist.org 20 Sep '24

20 Sep '24

As the subject says, I am new to OpenStack, and have spun up a test cluster with one control node, and two compute nodes. It seems to work for several tasks (i.e. I can spin up a cirros instance on both of the compute nodes, which seems to work just fine) However, when I attempt to migrate an instance from one compute node to the other, the instance goes in to an error state. (In order to get it running again, I have to set the state to active, then reboot the instance) I am not sure which logs will have the required information to help debug this issue, but I have looked through nova and cinder and keystone logs. Nothing I am seeing is obvious to me as to the cause of the failures. Also taking a snapshot seems to fail as well, with this error: Error: Unable to create snapshot. Details Invalid input received: Invalid volume: Volume e12b3ff8-9870-45f9-b59a-c6fb194ed517 status must be available, in-use, but current status is: attaching. (HTTP 400) (Request-ID: req-499d3bf9-97a0-48b2-8565-254a87287cff) (HTTP 400) (Request-ID: req-d0cd5aed-894e-40b2-8a2f-4b22188ec372) Looking at the volumes in my environment, both volumes show a status of "attaching" (both are for a test cirros instance, both are type nfs, and both instances seem to work just fine (meaning I can connect to them either on the console or via ssh and run commands without errors) Is the reason that each of those tasks is failing because of the "attaching" status of the volumes? I spun up the cluster using packstack and all the nodes are installed on Rocky Linux 9.4. I added an nfs backend after packstack was all done, and that does seem to be working, as the ephemeral volumes used to spin up the test instances does seem to be on the nfs storage, rather than the default lvm storage backend. I have installed openstack using the bobcat release of openstack (from the centos-openstack-bobcat repository). I have been crawling through the standalone install documentation to see if I can find something that explains why the instance migration isn't working, but, again, nothing is jumping out at me. Any guidance on where I should be looking, or advice on what I may be missing would be greatly appreciated.

4 10

upper-constraints (< $latest) considered harmful
by Stephen Finucane 20 Sep '24

20 Sep '24

o/ I'll jump straight into it. For those of you not in the loop we're currently going through a situation where it looks like OSC will be held back due to a bug that the Nova gates are highlighted. The actual details aren't hugely important, but what is important is the fact that this bug was first included in the OSC 7.0.0 release, which was released on August 6th. For those of you who are better at mental calendar maths than I am (or already looked at your calendar), you'll know this was over 6 weeks ago, well before the various feature freeze deadlines . and you might ask why did we only spot this last week? That would be because the upper-constraint (henceforth referred to as u-c) bump failed [1] due a different issue, meaning u-c kept us pinned at <= 6.6.1 which in turn meant every CI job *except* the ones running in the OSC gate itself kept testing the old release instead of the thing we were supposed to be released as part of Dalmatian. That only changed last week when 7.1.0 was released and the u-c bump for *that* merged [2], at which point people started shouting and hollering. Now, there is one other important consideration to factor into this, namely that, due to a flaw in our tooling, we didn't end up cutting a release branch for OSC alongside the rest of the client-library deliverables. Had we done so, we'd have seen 7.1.0 much earlier than we actually did [3]. That 7.1.0 release fixed the initial issue that prevented us releasing 7.0.0. However, this doesn't change the fact that 7.0.0 came out 6 weeks ago and not once did we get to test it outside of OSC's gate, and what's worse, we (the SDK team) were totally oblivious to the fact). Had it been known, we'd have cut a new release as soon as the fix [4] for that issue had merged (August 12th, which for the record is over 5 weeks ago). This is not the first time this has happened to us with one of our own dependencies. We had a very similar issue in Bobcat (2023.2) where we cut an oslo.db 13.0.0 release very early in that cycle, followed by a number of subsequent releases, only to have none of them end up making their way into u-c due to issues with some services. We then got to the end of the cycle, realized this, and had to do a frenzied mega revert to get us back to a pre-13.x state and allow us to release an oslo.db deliverable for Bobcat. It also happened with Castellan, whose 4.2.0 release sat waiting to merge for 4 months [4] before undergoing a similar mass revert. Adding more fuel to this fire, it's not just a problem with our own dependencies either. In fact, the problem is probably worse there. Need I remind people that it took 5 releases or 2 years, 8 months and change to uncap SQLAlchemy 2.x (a contributing factor in the oslo.db issue, fwiw). We had to struggle to get Sphinx uncapped in Bobcat [5] and fight to not revert that cap. You currently can't build docs for many projects on Fedora hosts since we're capping, among many other things, the Pillow library. The list goes on. Which brings me to my main points, of which there are three. Firstly: * There needs to be a clear time limit for how long a u-c bump can remain unmerged and a plan for what happens if affected projects have not resolved their issues within that time. There will always be exceptions - no one could rightly have asked services to switch to SQLAlchemy 2.x in a fortnight, and libraries can have legit bugs that justify a block - but those exceptions should be well-understood and, as with the bumps themselves, time-limited. A project that needs 4 releases to adapt to a crucial new library version is not a healthy project. secondly: * Caps or u-c bump reverts should similarly be clearly time-limited. We should not be defaulting to e.g re-capping Sphinx because your docs build has started to fail and then leaving that cap in place for months. Cap, fix the issue, and remove the cap asap. and, finally: * A failing u-c bump for an openstack deliverable should be treated with the highest priority 🚨🚨🚨 and should be something that the corresponding team should be made aware of immediately so they can start coordinating a resolution. We should not have u-c patches languishing for weeks, let alone months, only for there to be a last minute panic like this. I have ideas on the above (which basically warrant to more stick/[7]) but, perhaps fortunately, this isn't something I can decide on my own. Nor, should I add, is it something I expect the understaffed and oversubscribed release team to be able to do themselves. Instead, I think it's something that the TC and community as a whole need to settle on. So as soon as we get Dalmatian out the door, let's do that. Cheers, Stephen [1] https://review.opendev.org/c/openstack/requirements/+/925763 [2] https://review.opendev.org/c/openstack/requirements/+/928948 [3] https://review.opendev.org/c/openstack/releases/+/928838 [4] https://review.opendev.org/c/openstack/requirements/+/883141 [5] https://review.opendev.org/c/openstack/requirements/+/891694 [6] https://review.opendev.org/c/openstack/requirements/+/927102 [7] https://y.yarn.co/dbdc0d5b-04e6-4c39-9cd3-44c3be3729ab.mp4

8 9

[release] Release countdown for week R-1, Sep 23-27
by Elõd Illés 20 Sep '24

20 Sep '24

Development Focus ----------------- We are on the final mile of the 2024.2 Dalmatian development cycle! Remember that the 2024.2 Dalmatian final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. September 26th, 2024 is the deadline for final 2024.2 Dalmatian release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on October 2nd, 2024. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the 2025.1 Epoxy development cycle, including discussing PTG sessions content, in preparation of Project Teams Gathering. Actions ------- Watch for any translation patches coming through on the stable/2024.2 branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/2024.2 branch before triggering a new release. Please drop by #openstack-release with any questions or concerns about the upcoming release ! Upcoming Deadlines & Dates -------------------------- Final 2024.2 Dalmatian release: October 2nd, 2024 Project Teams Gathering: October 21-25, 2024 Előd Illés irc: elodilles @ #openstack-release

1 0

[cinder] Retype encrypted cinder volume to non-encrypted volume
by Jan Wasilewski 20 Sep '24

20 Sep '24

Hi, I would like to ask about the procedure to retype a Cinder volume that is in-use (bootable volume, connected to a VM) from an encrypted (LUKS) type to a non-encrypted type (1000 IOPS). I tried to do this via the CLI[1], and while it seemed to work (at least according to the CLI output), the process didn’t complete successfully. Cinder logs[2] indicate that everything looks more or less fine: the volume is created, the volume migration is OK, and the retype is marked as completed. However, in the end, it doesn’t work (final output of [1]). I reviewed the blueprint at https://blueprints.launchpad.net/cinder/+spec/retype-encrypted-volume, which appears to be implemented. Am I doing something wrong, or does this functionality not work as expected? Thanks in advance for your help and guidance. /JW *[1] Retype from CLI: https://paste.openstack.org/show/bVpFT82oURnanXkixreT/ <https://paste.openstack.org/show/bVpFT82oURnanXkixreT/>[2] cinder-logs: https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/ <https://paste.openstack.org/show/b5rZpWNMqdi6qdpjaDfg/>*

4 6

[glance] Still not working over uwsgi
by Thomas Goirand 20 Sep '24

20 Sep '24

Hi, I've been told in the past in this very list, that Glance would now work over uwsgi. Though that's not the case. Normal image upload works, but doing a "openstack server backup" fails, as well as shelving (it's the same operation...). We get: "Failed to upload image data due to HTTP error: webob.exc.HTTPRequestEntityTooLarge: Image storage media is full: Object PUT failed" even though the Swift user Glance is configured for isn't over quota. Is there any configuration options that we missed? Or should we still consider that Glance over uwsgi is broken? Cheers, Thomas Goirand (zigo)

3 2

[neutron] Drivers meeting canceled today and next Friday, September 27th
by Brian Haley 20 Sep '24

20 Sep '24

Hello Neutrinos: I'm going to cancel today's and next Friday's drivers meeting as there is no agenda and I have conflicts both days/times. We can spend the time making sure 2024.2 RC is in a good place :) Have a nice weekend! -Brian

1 0