August 2024 - openstack-discuss - lists.openstack.org

[ops][Image Encryption] Proposed Upgrade Workflow for the Image Encryption
by Josephine Seifert 19 Aug '24

19 Aug '24

Hi operators, we want to integrate image encryption into openstack. When the patches will be merged, there will be a change in a metadata name. Patches are here: https://review.opendev.org/q/topic:%22LUKS-image-encryption%22 We want to propose the following upgrade workflow: 1. install 2024.2 or later, new resources get os_encrypt_* metadata, legacy cinder_encrypt_* metadata is still honored for a deprication period 2. run glance-manage to update all the legacy metadata names to the new os_encrypt* 3. at this point, no openstack service will create cinder_encrypt* metadata, so anything from here must be created by a user 4. Operators need to reach out to the user and notify them, that shortly the cinder_encrypt* will no longer work We don't think the last step will occur often, because it is not documented anywhere, that it is possible to add these metadata to an image right now. If not done correctly (only possible if you know how Cinder does its volume encryption) it would render the image unusable. If you object to this workflow please let me know, we still have time to discuss this. greetings Josephine (Luzi)

1 0

[publiccloud-sig] Reminder - next meeting August 21st - 0700 UTC
by Felix Kronlage-Dammers 19 Aug '24

19 Aug '24

Hi everyone, This Wednesday (August 21st) the next meeting of the public cloud sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig <https://meetpad.opendev.org/publiccloud-sig> See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG <https://wiki.openstack.org/wiki/PublicCloudSIG> See you on Wednesday! felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[cinder] Instance Volume Issue After Restarting tgtd Container During Upgrade
by Vahideh Alinouri 19 Aug '24

19 Aug '24

Hi Everyone, I'm currently upgrading an OpenStack deployment that was set up using Kolla-Ansible. I'm currently on Kolla version 14.8 (Yoga) and I am trying to upgrade to (Latest Yoga) 14.11, to eventually upgrade to the Zed release. During the upgrade process, I encountered an issue with the compute node that uses the Cinder LVM backend, which the instances rely on. Restarting the tgtd container caused the connection between the iscsid container and tgtd to be lost. In the Nova compute logs, I found the following error: Stderr: 'blockdev: cannot open /dev/sdb: No such device or address\n' Because of this, the instance transitions to an error state. I also observed this error in the tgtd container: login failed ISCSI_CONN_STATE_IN_LOGIN/R_STAGE_SESSION_REOPEN 500 10:18 Kernel reported iSCSI connection 6:0 error (1019 - ISCSI_ERR_XMIT_FAILED: Transmission of iSCSI packet failed) state (2). When I check the iSCSI session with the following command, it shows that the session is established: iscsiadm --mode session However, when I run the following command on the compute host (which handles the Cinder LVM backend via iSCSI), no output is returned: tgtadm --lld iscsi --op show --mode target I tried logging out and logging back into the session, but the logout operation failed! Here is my cinder.conf file on the compute host: [os-compXXXX-nvme] volume_group = vgnvme volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver volume_backend_name = os-compXXXX-nvme target_helper = tgtadm target_protocol = iscsi lvm_mirrors = 0 lvm_max_over_subscription_ratio = 1.0 The operating system on the compute node is Ubuntu 20.04. After further investigation, I found a temporary workaround to bring the instance back to an active running state: 1. Power off the compute node. 2. Evacuate the node (it gets stuck in rebuild, but that's okay). 3. Power on the compute node. 4. Reset the instance state to "active." 5. Check the volume status and update it if it's not in the "in-use" state. 6. Perform a hard reboot of the instance. 7. Restart the tgtd container. However, this is only a temporary solution. When I attempt to upgrade to the next version, I encounter the same issue again. Is there anything specific I should consider? Could this be a bug? I would greatly appreciate your help.

1 1

[MAGNUM] Fedora Core OS include NVIDIA GPU drivers
by Oliver Weinmann 19 Aug '24

19 Aug '24

Hi all, I'm running OpenStack 2023.2 (Bobcat) deployed with Kolla-Ansible. Currently I'm struggling to get the NVIDIA drivers to work in a FCOS 35 image. I know FCOS 35 is EOL and outdated, but for a K8s 1.21.11 cluster it is the only version that still works with Magnum. So far I managed to get the GPU Passthrough working for two NVIDIA Tesla T4 GPUs. I have attached the GPUs to an Ubuntu instance and ran geekbench without any issues. Then I tried to install the latest NVIDIA drivers in FCOS 35 by running: sudo rpm-ostree install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$ (rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$ (rpm -E %fedora).noarch.rpm systemctl reboot rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda It fails on: rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda error: Cannot download rpm-build-libs-4.17.0-4.fc35.x86_64.rpm: All mirrors were tried; Last error: Status code: 403 for https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/rp… I guess the binaries are no longer hosted? Running the same on FCOS 36 and latest FCOS 40 works fine. Next I have taken new images from the FCOS instances with the NVIDIA driver. openstack server image create --name fedora_coreos_36_nvidia_cuda fedora_coreos_35_latest_no_update --wait This takes ages but anyway I can successfully deploy new instances from the images with my gpu flavor and the GPU is working fine. Again tested with geekbench. But now when adding new K8s worker nodes in a fresh nodegroup on an existing K8s 1.21.11 cluster the nodes are not joining the cluster: openstack coe nodegroup create \ --node-count 1 \ --role worker-gpu \ --flavor gpuflavor \ --image fedora_coreos_36_nvidia_cuda \ $CLUSTER_ID worker-gpu I have tested the latest available images of FCOS36 and they work fine, so something must be wrong with how I created the new image. Is there some magic that needs to be done? I'm also not sure if this is the best way of doing this. Maybe there is already a ready FCOS image with nvidia drivers? I can find lots of outdated information on the internet. Generally I tend to use Vexxhost CAPI from now on, which works great. This is really just a one off attempt to have a working NVIDIA T4 GPU in a K8s 1.21.11 cluster. Best Regards, Oliver

5 16

[template][cinder]Dell powerflex: cinder-plugin-ceph-tempest cannot be deployed successfully
by cuiye.liu＠dell.com 19 Aug '24

19 Aug '24

Because the https://zuul.opendev.org/t/openstack/build/cc284844b56c41b3aa58669036f25a71 is failed. I deploy a local environmet according the guide https://github.com/openstack/devstack-plugin-ceph. But it is failed, and the error is as following: +inc/python:pip_install:151 set +o xtrace Using python 3.10 to install /opt/stack/keystone +inc/python:pip_install:219 env http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= /opt/stack/data/venv/bin/pip install -c /opt/stack/requirements/upper-constraints.txt -e /opt/stack/keystone Obtaining file:///opt/stack/keystone Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [1 lines of output] ERROR: Can not execute `setup.py` since setuptools is not available in the build environment. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip.

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-6)
by Goutham Pacha Ravi 18 Aug '24

18 Aug '24

Hello Stackers, Greetings from the OpenStack Technical Committee! We're two weeks away from freezing the 2024.2 release for feature changes across most service projects [1]. So the focus of these final few weeks has been on code reviews and testing. The community's election season is also underway. I hope you're able to carve out some time to consider submitting a nomination to be a Project Team Leader [2] or a member of the Technical Committee [3] this week [4]. Please feel free to ask questions on the #openstack-dev or #openstack-tc channels on OFTC. === Weekly Meeting === It was the usual order of business at the OpenStack TC meeting last week. This meeting was held over IRC and the log is available on eavesdrop [5]. The discussion was focussed on gate stability, CI and infrastructure enhancements as well as the ongoing elections. Our call for help with Rocky Linux in the CI is being heeded. Thank you very much to the volunteers that are stepping up and the #opendev-infra team for coordinating this. Ghanshyam Mann (gmann) has proposed updates to the runtimes for Python and Operating System platforms for 2025.1 ("Epoxy") [6]. I hope you'll be able to review this proposal. We tabled a discussion regarding testing/shipping binaries with non-OSI compatible licenses to the next meeting. The next IRC meeting of the OpenStack TC is on Tuesday, 2024-08-20. We hope you'll be able to join us on the #openstack-tc channel on OFTC. Please find the agenda in the meeting wiki [7]. Do note that you can always propose discussion items to this agenda, be sure to leave an IRC nick that we can call upon to lead/assist with the discussion. === Governance Proposals === ==== Open for review/discussion ==== Define testing runtime for 2025.1 release | https://review.opendev.org/c/openstack/governance/+/926150 Inactive state extensions: Monasca | https://review.opendev.org/c/openstack/governance/+/923919 Opt-in the oslo DPL model for 2025.1 cycle | https://review.opendev.org/c/openstack/governance/+/926154 Retire all single charm repositories | https://review.opendev.org/c/openstack/governance/+/903490 Update DPL model reset steps | https://review.opendev.org/c/openstack/governance/+/925926 === Upcoming Events === 2024-08-24: Open Infra Days, Vietnam: https://2024.vietopeninfra.org/ 2024-08-28: Nomination period closes for 2025.1 OpenStack PTL+TC elections 2024-08-29: Dalmatian-3 milestone 2024-09-03: Open Infra Summit Asia, Suwon 2024-09-06: Open Infra Days, Tokyo === How to contact the TC === You can reach out to the TC in multiple ways: 1. Email: you can send the email with tag [tc] on this email list. 2. Ping us using 'tc-members' keyword on #openstack-tc IRC channel 3. Join us at our weekly meeting: The Technical Committee meets every week on Tuesdays at 1800 UTC [7] Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] Becoming a PTL: https://docs.openstack.org/project-team-guide/ptl.html [3] OpenStack TC Charter: https://governance.openstack.org/tc/reference/charter.html [4] Combined PTL/TC 2025.1 cycle Election Nominations Kickoff https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [5] TC Meeting IRC Log, 2024-08-13: https://meetings.opendev.org/meetings/tc/2024/tc.2024-08-13-18.00.log.html [6] Proposal for testing runtimes for the 2025.1 release: https://review.opendev.org/c/openstack/governance/+/926150 [7] TC Meeting, 2024-08-20: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

Eventlet and debugging
by engineer2024 18 Aug '24

18 Aug '24

Hi All, How many more releases are you planning to include eventlet in them ? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives ? Also, can someone write a small blog on how to use backdoor to debug the nova processes. That would be a great help until then.... Thanks elinux

7 16

[cinder][tempest] service_token_roles=admin unexpected behaviour
by Sven Springer 16 Aug '24

16 Aug '24

Hi, I have a question in regards to the service_token_roles setting in combination with an old vulnerability: I have a failing tempest test case, which is there to specifically test this vulnerability: https://lists.openstack.org/pipermail/openstack-announce/2023-May/002070.ht… I checked the cinder in our setup, and it contains the fix Test-reference: https://review.opendev.org/c/openstack/tempest/+/882876 Now I wonder if the test fails because in our setup, we set "service_token_roles = admin" for cinder, and the detach calls therefore being accepted (when the tempest tests expect them to be denied). Can someone confirm or deny this hypothesis? Best regards, Sven.

1 0

[release] Release countdown for week R-6, Aug 19-23
by Elõd Illés 16 Aug '24

16 Aug '24

Development Focus ----------------- Work on libraries should be wrapping up, in preparation for the various72 library-related deadlines coming up. Now is a good time to make decisions on deferring feature work to the next development cycle in order to be able to focus on finishing already-started feature work. General Information ------------------- We are now getting close to the end of the cycle, and will be gradually freezing feature work on the various deliverables that make up the OpenStack release. This coming week is the deadline for general libraries (except client libraries): their last feature release needs to happen before "Non-client library freeze" on August 22nd, 2024. Only bugfixes releases will be allowed beyond this point. When requesting those library releases, you can also include the stable/2024.2 branching request with the review (as an example, see the "branches" section here: https://opendev.org/openstack/releases/src/branch/master/deliverables/pike/… In the next weeks we will have deadlines for: * Client libraries (think python-*client libraries), which need to have their last feature release before "Client library freeze" (August 29th, 2024) * Deliverables following a cycle-with-rc model (that would be most services), which observe a Feature freeze on that same date, August 29th, 2024. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. As we are getting to the point of creating stable/2024.2 branches, this would be a good point for teams to review membership in their $project-stable-maint groups. Once the stable/2024.2 branches are cut for a repo, the ability to approve any necessary backports into those branches for 2024.2 Dalmatian will be limited to the members of that stable team. If there are any questions about stable policy or stable team membership, please reach out in the #openstack-stable channel. Upcoming Deadlines & Dates -------------------------- Non-client library freeze: August 22nd, 2024 (R-6 week) Client library freeze: August 29th, 2024 (R-5 week) Dalmatian-3 milestone: August 29th, 2024 (R-5 week) Cycle Highlights Due: September 5th, 2024 (R-4 week) 2024.2 Dalmatian final release: October 2nd, 2024 Előd Illés irc: elodilles @ #openstack-release

1 0

[neutron] Drivers meeting canceled tomorrow, Friday, August 16th
by Brian Haley 16 Aug '24

16 Aug '24

Hello Neutrinos: I'm going to cancel tomorrow's drivers meeting as there is no agenda and two members cannot attend. There is one item for next week's meeting so far: * [RFE] Allow binding Security Groups to Network https://bugs.launchpad.net/neutron/+bug/2075955 Feel free to add any comments to the bug so the submitter can try and answer them before then. Thanks! -Brian

1 0