October 2022 - OpenStack-announce

[OSSN-0091] BMC emulators developed in OpenStack community do not preserve passwords on VMs
by Jay Faulkner 01 Nov '22

01 Nov '22

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 ## Summary ## When deploying VirtualBMC or Sushy-Tools in an unsupported, production-like configuration, it can remove secret data, including VNC passwords, from a libvirt domain permanently. Operators impacted by this vulnerability must reconfigure any secret data, including VNC passwords, for the libvirt domain. These virtual machine emulators are tools to help emulate a physical machine's Baseboard Management Controller (BMC) to aid in development and testing of software that would otherwise require physical machines to perform integration testing activities. They are not intended or supported for production or long-term use of any kind. ## Affected Services / Software ## * Sushy-Tools, <=0.21.0 * VirtualBMC, <=2.2.2 There is no impact to any OpenStack software or services intended for production use. ## Patches ## * VirtualBMC: https://review.opendev.org/c/openstack/virtualbmc/+/862620 * Sushy-Tools: https://review.opendev.org/c/openstack/sushy-tools/+/862625 ## Discussion ## To perform some advanced operations on Libvirt virtual machines, the underlying XML document describing the virtual machine's domain must be extracted, modified, and then updated. These specific actions are for aspects such as "setting a boot device" (VirtualBMC, Sushy-Tools), Setting a boot mode (Sushy-Tools), and setting a virtual media device (Sushy-Tools). This issue is triggered when a VM has any kind of "secure" information defined in the XML domain definition. If an operator deploys VirtualBMC or Sushy-Tools to manage one of these libvirt VMs, the first time any action is performed that requires rewriting of the XML domain definition, all secure information -- including a VNC console password, if set -- is lost and removed from the domain definition, leaving the libvirt VM's exposed to a malicious console user. ## Recommended Actions ## Operators who may have been impacted by this vulnerability should immediately remove use of VirtualBMC and/or Sushy-Tools from their production environment. Then, validate and if necessary, reconfigure passwords for VNC access or any other impacted secrets. ## Notes ## The OpenStack team will ensure documentation is updated to clearly state these software packages are intended for development/CI use only, and are not safe to run in production. ## Credits ## Julia Kreger from Red Hat ## References ## Author: Jay Faulkner, G-Research Open Source Software This OSSN: https://wiki.openstack.org/wiki/OSSN/OSSN-0091 Original Storyboard bug: https://storyboard.openstack.org/#!/story/2010382 Mailing List : [Security] tag on openstack-discuss(a)lists.openstack.org OpenStack Security Project : https://launchpad.net/~openstack-ossg CVE: CVE-2022-44020 -----BEGIN PGP SIGNATURE----- Version: FlowCrypt Email Encryption 8.3.8 Comment: Seamlessly send and receive encrypted email wsFzBAEBCgAGBQJjYDqWACEJEGt12Tm0JMbUFiEEvF1YmsGLSYuWqE+ta3XZ ObQkxtSjBhAAgkeFOi39SEuatIMBLA+EEXyTzEyINJQnFmUqJlDLLckQv646 sotFsmYizNWn3GEVEYNjPWjMdQz11NTA4Ox8CijoPGHPNCoaHc461Vm3VH1Y 4rv4nzDI+JsU/r4HRNT6N5BveqiCpvpIvUmsxbdzAciBqqjSjpAk/wuY3v3p QRyqktMW3ep8evGw0xAaj3UWElLM+ooJM/Z4CayKQXeNTFDXdlQsIxGQAmrM s9U9sxf1I839oj5AbBQZ2mYV/oV6DdrTEvrDnGtwrowlugCdBXKVLETbylAn tvlPz+I6DrNWndCucZthwDesccpMvVqxCUup+lYs9o0RcfT8sK5PxNEzpi5U zR07iC9hJz3Pa9I7VWOQMtWIDD1XYgQZ7iCKu/kd4CXM74O1qpdoQQ0T4dCP J0Scv/L5TVkV3X7d6tyJ6i8bdVEt6XQvXLFIzo9iudheuxF8j4bGpgcMBw/a OUi7AGXE00P1qaCUON8hgu4F3b5iU3guoJD7EmW8CAyDVOqtZdgFQUs0UYnb 443nEe/MHrwed+UBXbrg9xKrXalRvkQP6wG1WvAV6U2yY6cMntYb4YWUl37K ggG+rsYIj/j8o5JncBuxyQyXGbjammZYJHOzDpJO42xwe5B8aXPamx8Z6V6I 6Fb07pYS1vlghBDnvU37KYF6bVpBBZ8QiiQ= =RmVg -----END PGP SIGNATURE-----

1 0

[0SSN-0090] Best practices when configuring Glance with COW backends
by Brian Rosmaita 17 Oct '22

17 Oct '22

Best practices when configuring Glance with COW backends --- ### Summary ### When deploying Glance in a popular configuration where Glance shares a common storage backend with Nova and/or Cinder, it is possible to open some known attack vectors by which malicious data modification can occur. This note reviews the known issues and suggests a Glance deployment configuration that can mitigate such attacks. ### Affected Services / Software ### Glance, all supported releases (Queens through Zed) ### Discussion ### This note applies to you if you are operating an end-user-facing glance-api service with the 'show_multiple_locations' option set to True (the default value is False) or if your end-user-facing glance-api has the 'show_image_direct_url' option set to True (default value is False). Our recommendation is that the image "locations" and "direct_url" fields [0] *never* be displayed to end users in a cloud. This can be accomplished by running two glance-api services: - A "user-facing" glance-api that is accessible to end users and which appears in users' service catalogs. - An "internal-only-facing" glance-api that is accessible only to those services that require access to the 'direct_url' or image location fields, and which is protected by firewalls from access by end users. (Nova, Cinder, and Ironic all have configuration options to specify the Glance API endpoint each service uses [1].) This dual glance-api deployment was suggested in "Known Issues" in Glance release notes in the Rocky [2] through Ussuri releases, but it seems that the idea has not received sufficient attention. Hence this security note. The attack vector that becomes available when image locations are exposed to end users was originally outlined in OSSN-0065 [3], though that note was not clear about the attack surface or mitigation, and contained some forward-looking statements that were not fulfilled. The attack vector is: [A] malicious user could create an image in Glance, set an additional location on that image pointing to an altered image, then delete the original location, so that consumers of the original image would unwittingly be using the malicious image. Note, however, that this attack vector cannot change the original image's checksum, and it is limited to images that are owned by the attacker. OSSN-0065 suggests that this is only an issue when users do not checksum their image data. It neglects the fact that in some popular deployment configurations in which Nova creates a root disk snapshot, data is never uploaded to Glance, but instead a snapshot is created directly in the backend and Nova creates a Glance image record with "size" 0 and an empty "os_hash_value" [4], making it impossible to compare the hash of downloaded image data to the value maintained by Glance. Further, when Nova is configured to use the same storage for ephemeral disks that is used as a Glance image store, Nova efficiently creates a server root disk directly in the backend without checksumming the image data. This is an intentional design choice to optimize storage space and host resources, but an implication is that even if the image record has a recorded hash, it is not being checked at the point of image consumption. Similarly, when using a shared backend, or a cinder glance_store, Cinder will efficiently clone a volume created from an image directly in the backend without checksumming the image data. Again, this is done intentionally in order to optimize resources, but it is important to be aware of the security tradeoff being made by this configuration. In other words, if the image data is not going to be checked at the point of image consumption, then extra care needs to be taken to ensure the integrity of the data path. OSSN-0065 suggested that the attack vector of substituting image data by modifying the image locations could be addressed by using policies, but that has turned out not to be the case. The only way currently to mitigate this vector is to deploy glance-api in a dual configuration as described above, namely with an internal-only-facing glance-api used by Nova and Cinder (that has show_multiple_locations enabled), and an end-user-facing glance-api (that has show_multiple_locations disabled). So far the focus has been on 'show_multiple_locations'. When that setting is disabled in Glance, it is not possible to manipulate the locations via the OpenStack Images API. Keep in mind, however, that in any Glance/Nova/Cinder configuration where Nova and/or Cinder do copy-on-write directly in the image store, image data transfer takes place outside Glance's image data download path, and hence the os_hash_value is *not* checked. Thus, if the backend store is itself compromised and image data is replaced directly in the backend, the substitution will *not* be detected. This brings us to the 'show_image_direct_url' option, which includes a "direct_url" field in the image-show response that can be used by various OpenStack services to consume images directly from the storage backend. Exposing the 'direct_url' to end users leaks information about the storage backend. What exactly that information consists of depends upon the backend in use and how it is configured, but in general, it is not a good idea to provide hints that could be useful to malicious actors in their attempts to compromise the backend storage by some type of independent exploit. The 'direct_url', being read-only, may appear innocuous, but its use by services is usually to perform some kind of optimized image data access that most likely does not include computing a hash of the image data. We therefore recommend that OpenStack services that require exposure of the 'direct_url' image property be similarly configured to use an internal-only-facing glance-api. It is worth nothing that end users who wish to download an image do not require access to the 'direct_url' image property because they can simply use the image-data-download API call [5]. ### Recommended Actions ### A glance-api service with 'show_multiple_locations' enabled should *never* be exposed directly to end users. This setting should only be enabled on an internal-only-facing glance-api that is used by OpenStack services that require access to image locations. This could be done, for example, by running two glance-api services with different configuration files and using the appropriate configuration options for each service to specify the Image API endpoint to access, and making sure the special internal endpoint is firewalled in such a way that only the appropriate OpenStack services can contact it. Similarly, enabling 'show_image_direct_url' exposes information about the storage backend that could be of use to malicious actors in as yet unknown exploits, so it should likewise only be enabled on an internal-only-facing glance-api. ### Notes / References ### [0] https://docs.openstack.org/api-ref/image/v2/index.html#show-image-schema [1] Nova and Ironic use 'endpoint_override' in the '[glance]' section of the configuration file; Cinder uses 'glance_api_servers' in the '[DEFAULT]' section. [2] OSSN-0065: https://wiki.openstack.org/wiki/OSSN/OSSN-0065 [3] The Glance "multihash" metadata pair of 'os_hash_algo' and 'os_hash_value' were introduced in Rocky to replace the legacy md5 'checksum' field. The python-glanceclient has used multihash checksumming for download verification since version 2.13.0. [4] https://docs.openstack.org/releasenotes/glance/rocky.html#known-issues [5] https://docs.openstack.org/api-ref/image/v2/index.html?#download-binary-ima… ### Contacts / References ### Author: Brian Rosmaita, Red Hat This OSSN : https://wiki.openstack.org/wiki/OSSN/OSSN-0090 Original LaunchPad Bug : https://bugs.launchpad.net/ossn/+bug/1990157 Mailing List : [Security] tag on openstack-discuss(a)lists.openstack.org OpenStack Security Project : https://launchpad.net/~openstack-ossg CVE: none

1 0

OpenStack Zed is officially released!
by Előd Illés 05 Oct '22

05 Oct '22

Hello OpenStack community, I'm excited to announce the final releases for the components of OpenStack Zed, which conclude the Zed development cycle. You will find a complete list of all components, their latest versions, and links to individual project release notes documents listed on the new release site. https://releases.openstack.org/zed/ Congratulations to all of the teams who have contributed to this release! Our next production cycle, 2023.1 Antelope, has already started. We will meet at the Virtual Project Team Gathering, October 17-21, 2022, to plan the work for the upcoming cycle. I hope to see you there! Thanks, Előd Illés and the Release Management team

1 0