July 2024 - openstack-discuss - lists.openstack.org

Unrestricted AC Access Rules: "You must upgrade keystonemiddleware on all services"
by openstack＠tr.id.au 23 Jul '24

23 Jul '24

Hey there, I'm interested in creating an unrestricted Application Credential (AC) which is still constrained by Access Rules. A use could be allowing admins to use automation to generate read-only ACs for various purposes. The AC for automatically generating other ACs would need to be unrestricted, but should still only have the minimal privileges required to create its children. However, when attempting this with multiple versions of keystone, including 2024.1, I get the following error in the keystone service log: ``` Attempted to use application credential access rules with a middleware that does not understand them. You must upgrade keystonemiddleware on all services that accept application credentials as an authentication method.: NoneType: None ``` A minimal way to reproduce this: 1. Install keystone according to https://docs.openstack.org/keystone/latest/install/keystone-install-ubuntu.…. (I've also tested with keystone running in a virtualenv and in a kolla container.) 2. Generate a "parent" AC using the bootstrap admin: `openstack application credential create --max-width 120 --role admin --access-rules '[ { "path": "/v3/**", "method": "POST", "service": "identity" }, { "path": "/v3/**", "method": "GET", "service": "identity" } ]' --expiration '2024-07-31T23:59:59' --unrestricted test_unrestricted` (This gives expected output, including an AC id and secret.) 3. In another terminal, load the parent AC and then try to generate the child: `openstack application credential create --max-width 120 --role admin --access-rules '[ { "path": "/v3/**", "method": "GET", "service": "identity" } ]' --expiration '2024-07-31T23:59:59' test_read_only`. (This returns `Invalid application credential: Cannot create an application credential with unassigned role` in the client, as well as the earlier error about upgrading middleware in keystone service log.) I've tested that the parent/unrestricted AC *can* generate the child successfully when the parent does *not* have Access Rules. I didn't see anything about these errors or constraints when going through the application credential documentation (eg https://docs.openstack.org/keystone/latest/user/application_credentials.html), or in a web search, or in a search of the openstack-discuss archives. I may raise a bug report/feature request, but before I do that, I just wondered if I have likely overlooked some possible extra configuration or documentation which could explain what I've done wrong? Alternatively, if this were raised as a feature request, would it likely be supported, or would the functionality I'm imagining be too difficult or impossible to implement? Cheers, Tim

1 0

Scheduler and filters
by Gk Gk 23 Jul '24

23 Jul '24

Hi, Can anyone tell me the difference between the schedulers and filters in nova. As per what I grasped, it seems we have the same type of weighers like schedulers. Then what's the difference ? Please explain.. Thanks Y.G

3 2

[OSSA-2024-002] OpenStack Nova: Incomplete file access fix and regression for QCOW2 backing files and VMDK flat descriptors (CVE-2024-40767)
by Jeremy Stanley 23 Jul '24

23 Jul '24

================================================================== OSSA-2024-002: Incomplete file access fix and regression for QCOW2 backing files and VMDK flat descriptors ================================================================== :Date: July 23, 2024 :CVE: CVE-2024-40767 Affects ~~~~~~~ - Nova: <27.4.1, >=28.0.0 <28.2.1, >=29.0.0 <29.1.1 Description ~~~~~~~~~~~ Arnaud Morin (OVH) reported a vulnerability in Nova. By supplying a raw format image which is actually a specially crafted QCOW2 image with a backing file path or VMDK flat image with a descriptor file path, an authenticated user may convince systems to return a copy of the referenced file’s contents from the server resulting in unauthorized access to potentially sensitive data. All Nova deployments are affected. Patches ~~~~~~~ - https://review.opendev.org/924734 (2023.1/antelope) - https://review.opendev.org/924733 (2023.2/bobcat) - https://review.opendev.org/924732 (2024.1/caracal) - https://review.opendev.org/924731 (2024.2/dalmatian) Credits ~~~~~~~ - Arnaud Morin from OVH (CVE-2024-40767) References ~~~~~~~~~~ - https://launchpad.net/bugs/2071734 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-40767 Notes ~~~~~ - The patches linked above should apply cleanly to the public state of their respective branches at time of disclosure, and depend on some commits which merged after the `OSSA-2024-001 <https://security.openstack.org/ossa/OSSA-2024-001.html>`_ fixes as well as the final states of the Nova changes linked from that advisory (those did see some minor adjustments before they merged). - The QCOW2 issue is due to an incomplete fix in OSSA-2024-001 affecting systems where the ``use_cow_images`` configuration option is disabled, while the VMDK issue is a regression of the earlier `OSSA-2023-002 <https://security.openstack.org/ossa/OSSA-2023-002.html>`_ vulnerability reintroduced by the new implementation in OSSA-2024-001. Both problems were identified in the final hours before OSSA-2024-001 publication but, due to time constraints, were redacted from that bug and moved to a separate report. - Neither the methods introduced in these patches nor the fixes for OSSA-2024-001 are capable of blocking malicious images which are already resident in Nova's cache. At this time we do not have useful operator guidance for identifying and removing such existing images from the cache but strongly caution, if you do attempt to use the qemu-img tool to find them, to make sure you're using a version of it patched for `QEMU CVE-2024-4467 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-4467>`_. -- Jeremy Stanley OpenStack Vulnerability Management Team

1 0

[MAGNUM] Fedora Core OS include GPU drivers
by Oliver Weinmann 23 Jul '24

23 Jul '24

Hi all, I'm running OpenStack 2023.2 (Bobcat) deployed with Kolla-Ansible. Currently I'm struggling to get the NVIDIA drivers to work in a FCOS 35 image. I know FCOS 35 is EOL and outdated, but for a K8s 1.21.11 cluster it is the only version that still works with Magnum. So far I managed to get the GPU Passthrough working for two NVIDIA Tesla T4 GPUs. I have attached the GPUs to a Ubuntu instance and ran geekbench without any issues. Then I tried to install the latest NVIDIA drivers in FCOS 35 by running: sudo rpm-ostree install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$ (rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$ (rpm -E %fedora).noarch.rpm systemctl reboot rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda It fails on: rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda I guess the binaries are no longer hosted? Running the same on FCOS 36 and latest FCOS 40 works fine. Next I have taken new images from the FCOS instances with the NVIDIA driver. openstack server image create --name fedora_coreos_36_nvidia_cuda fedora_coreos_35_latest_no_update --wait This takes ages but anyway I can successfully deploy new instances from the images with my gpu flavor and the GPU is working fine. Again tested with geekbench. But now when adding new K8s worker nodes in a fresh nodegroup on an existing cluster 1.21.11 cluster the nodes are not joining the cluster: openstack coe nodegroup create \ --node-count 1 \ --role worker-gpu \ --flavor gpuflavor \ --image fedora_coreos_36_nvidia_cuda \ $CLUSTER_ID worker-gpu But I know from the past that FCOS 35 was the only version that was working fine with K8s 1.21.11. So I tried with FCOS 35 and this works fine, but unfortunately without the NVIDIA drivers. So maybe the problem is related to this and I need to get FCOS 35 working? Or a newer FCOS version? I'm also not sure if this is the best way of doing this. Maybe there is already a ready FCOS image with nvidia drivers? I can find lots of outdated information on the internet. Generally I tend to use Vexxhost CAPI from now on, which works great. This is really just a one off attempt to have a working NVIDIA T4 GPU in a K8s 1.21.11 cluster. Best Regards, Oliver

1 0

[Neutron] - Floating ip allocation in a specific subnet
by MIchele Delli Veneri 23 Jul '24

23 Jul '24

Dear all, My network setup includes a single external network (ext_vlan) with 2 subnets: 10.x.x.0/24 and 193.x.x.0/24. Is there a way to setup the allocation logic such as that given a project, all floating ips in the project are always allocated on one of the two subnets unless the subnet is explicited (as it may happen through the Openstack CLI or the admin horizon interface)? The ideal outcome would be, given the project test-project and test-project2 , that running openstack floating ip create —project test-project ext_vlan always allocate the floating ip on the 193.x.x subnet while openstack floating ip create —project test-project2 ext_vlan always allocates the floating ip on the 10.x.x. subnet. Best regards, Michele

4 4

Not able to find updated repos
by Ashok Thipe 22 Jul '24

22 Jul '24

Hi, We need to install latest community version of openstack but we are unable to get updated repositories. could anyone please help. With regards, Ashok Thipe

3 4

[cinder] How extend volume if cinder-volume host down
by Nguyễn Hữu Khôi 22 Jul '24

22 Jul '24

Hello guys. Every volume was managed by a specified host. When this host was down or removed. How Can I extend this volume which was managed by this host? I would like to know if we have a solution for this case? Nguyen Huu Khoi

2 4

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-10)
by Goutham Pacha Ravi 22 Jul '24

22 Jul '24

Hello Stackers, As we get closer to the code freeze dates for the upcoming 2024.2 ("Dalmatian") release cycle [1], OpenStack project teams are finalizing features and bug fixes. We're ten weeks away from the official release date, but the feature freeze is squarely at halfway to that milestone. I'd like to share the governance changes that the OpenStack TC has approved in the past week. TC members joined several reviewers to approve the proposal to drop Eventlet from OpenStack [2]. Hervé Beraud (hberaud) and others will update the proposal further based on the feedback that was received through the review process. You may also see updates regarding this across Oslo and OpenStack service projects; and this may become an official community goal if we are able to close out on the feedback and agree upon a structure/timeline. I'd like to congratulate the Skyline team for having graduated from its "Emerging Project" status under the OpenStack services umbrella. In a similar vein, the Watcher project has been bootstrapped with a few new core maintainers and they have decided to adopt a "Distributed Project Maintainers" leadership model. We approved a request from the Freezer team to remain "inactive" as it builds back the contributor momentum necessary to churn out releases of its deliverables again. With the next OpenStack TC and PTL elections around the corner (nominations open on 2024-08-14 [3]), we updated the TC charter to clarify that changes proposed to documentation and governance repositories also qualify towards becoming an "Active Contributor" [4]. === Weekly Meeting === The last meeting of the OpenStack TC was held on IRC on 2024-07-16 [5]. We discussed CI instability and pending patches in Masakari and Adjutant. The project PTLs and maintainers responded to TC requests and have now addressed many concerns. We brainstormed gate issues in the past weeks, and difficulties getting all the patches concerning CVE-2024-32498 merged [6]. A constant theme seems to be test VM SSH connections failing within tests. Although this has been addressed a lot over the years and many changes have made the situation better, intermittent failures still remain. If you're interested in helping the OpenStack QA team with this, please let us know! We also discussed the goal proposal to drop Eventlet [2]. The proposal will require further work to be targeted as a goal. The next meeting of the OpenStack Technical Committee is on 2024-07-23. The agenda is on the OpenStack TC's wiki [7]. Please attend and voice your opinions! === Governance Proposals === ==== Merged ==== Update to include docs and miscellaneous repos for AC status | https://review.opendev.org/c/openstack/governance/+/915021 Move Skyline out from the "Emerging Projects" list | https://review.opendev.org/c/openstack/governance/+/924109 Remove Eventlet from OpenStack | https://review.opendev.org/c/openstack/governance/+/902585 Transition Watcher project to DPL | https://review.opendev.org/c/openstack/governance/+/923583 Inactive state extensions: Freezer | https://review.opendev.org/c/openstack/governance/+/923441 ==== Open for review/discussion ==== Follow up on the affiliation diversity handling | https://review.opendev.org/c/openstack/governance/+/923876 Update criteria for the inactive projects to become active again | https://review.opendev.org/c/openstack/governance/+/921500 Inactive state extensions: Monasca | https://review.opendev.org/c/openstack/governance/+/923919 Retire kuryr-kubernetes and kuryr-tempest-plugin | https://review.opendev.org/c/openstack/governance/+/922507 === Upcoming Events === 2024-08-12: Call for Proposals due for the OpenInfra Days NA [8] 2024-08-14: Nominations open for 2025.1 OpenStack PTL+TC elections 2024-08-24: OpenInfra Days, Vietnam: https://2024.vietopeninfra.org/ 2024-08-29: Dalmatian-3 milestone 2024-09-03: OpenInfra Summit Asia, Suwon I'll leave you with a link to the upbeat OpenInfra Summit welcome video featuring some of our APAC contributors: https://www.youtube.com/watch?v=aGuB1cf3DbY. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] Goal to remove Eventlet from OpenStack: https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html [3] OpenStack Elections: https://governance.openstack.org/election/ [4] Active Contributors: https://governance.openstack.org/tc/reference/charter.html#voters-for-tc-se… [5] TC Meeting IRC Logs, 2024-07-16: https://meetings.opendev.org/meetings/tc/2024/tc.2024-07-16-18.00.log.html [6] Arbitrary file access through custom QCOW2 external data (CVE-2024-32498): https://www.openwall.com/lists/oss-security/2024/07/02/2 [7] TC Meeting, 2024-07-23: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [8] OID North America Call for Proposals: https://docs.google.com/forms/d/1U3TokM5QWTe9_MJvKbOiDc_cNNW_LlFpOL3t-Loib1…

1 0

[neutron] Bug Deputy Report July 15 - July 21
by Miro Tomaska 22 Jul '24

22 Jul '24

Hello All, Here is the bug deputy report for the week of July 15th *Critical:* - https://bugs.launchpad.net/neutron/+bug/2073251 - linuxbridge/ovs jobs fails randomly as Test vms not getting ip from dhcp agent Assigned: Yatin Karel *High:* - https://bugs.launchpad.net/neutron/+bug/2073464 - [DHCP] Race condition when creating a network Assigned: Rodolfo Alonso - https://bugs.launchpad.net/neutron/+bug/2073567 [master][ml2-ovn] Multiple Unexpected exception in notify_loop: neutron_lib.exceptions.PortNotFound Unassigned - https://bugs.launchpad.net/neutron/+bug/2073585 - ubuntu focal jobs fails at dbcounter installation with TypeError: canonicalize_version() Unassigned, but Rodolfo Alonso already proposed a patch to lock down setup tools to v70.2.0 *Medium:* - https://bugs.launchpad.net/neutron/+bug/2073490 - [DHCP] Improve the DHCP agent log Assigned: Rodolfo Alonso - https://bugs.launchpad.net/neutron/+bug/2073309 Neither fdb_removal_limit or mac_binding_removal_limit get set in NB_Global.options when they are configured Assigned: Terry Wilson *Undecided:* - https://bugs.launchpad.net/neutron/+bug/2073326 - ovn] OVSDB Error ' references nonexistent' occurs when updating one router. Most likely will get closed as an unsupported and modified branch is being used. Move to "won't fix"? - https://bugs.launchpad.net/neutron/+bug/2073509 - east and west traffic is not allowed/blocked with neutron fwaas v2 Involves Neutron fwaas, simple reproducing steps but need to confirm if this even a valid bug or is it by design. -- Miro Tomaska Senior Software Engineer

1 0

Kolla deploy
by William Muriithi 22 Jul '24

22 Jul '24

Hello, What could I be doing wrong here? When we attempt openstack deploy, this step fails with the error message below? kolla-ansible -i /etc/kolla/multinode deploy debug3: mux_client_request_alive: done pid = 91641 debug3: mux_client_request_session: session request sent debug1: mux_client_request_session: master session id: 2 debug3: mux_client_read_packet: read header failed: Broken pipe debug2: Received exit status from master 1 The full traceback is: File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/modules/kolla_container.py", line 425, in main File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py", line 515, in recreate_or_restart_container self.start_container() File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py", line 538, in start_container self.create_container() File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py", line 501, in create_container container = self.pc.containers.create(**args) File "/usr/local/lib/python3.9/site-packages/podman/domain/containers_create.py", line 354, in create response.raise_for_status(not_found=ImageNotFound) File "/usr/local/lib/python3.9/site-packages/podman/api/client.py", line 75, in raise_for_status raise APIError(cause, response=self._response, explanation=message) fatal: [ostack-40.openstack.perasotech.com]: FAILED! => { "changed": true, "invocation": { "module_args": { "action": "recreate_or_restart_container", "api_version": "auto", "auth_email": null, "auth_password": null, "auth_registry": "quay.io", "auth_username": null, "client_timeout": 120, "container_engine": "podman", "detach": true, "dimensions": { "ulimits": {} }, "environment": { "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS" }, "graceful_timeout": 60, "ignore_missing": false, "image": "quay.io/openstack.kolla/fluentd:master-rocky-9", "labels": {}, "name": "fluentd", "privileged": false, "remove_on_exit": true, "restart_policy": "unless-stopped", "restart_retries": 10, "security_opt": [], "state": "running", "tls_verify": false, "tty": false } }, "msg": "'Traceback (most recent call last):\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/modules/kolla_container.py\", line 425, in main\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py\", line 515, in recreate_or_restart_container\\n self.start_container()\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py\", line 538, in start_container\\n self.create_container()\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py\", line 501, in create_container\\n container = self.pc.containers.create(**args)\\n File \"/usr/local/lib/python3.9/site-packages/podman/domain/containers_create.py\", line 354, in create\\n response.raise_for_status(not_found=ImageNotFound)\\n File \"/usr/local/lib/python3.9/site-packages/podman/api/client.py\", line 75, in raise_for_status\\n raise APIError(cause, response=self._response, explanation=message)\\npodman.errors.exceptions.APIError: 500 Server Error: Internal Server Error (statfs /var/log/journal: no such file or directory)\\n'" } Have anyone see the issue before? Regards, William

2 3