- OpenStack-announce - lists.openstack.org

[0SSN-0090] Best practices when configuring Glance with COW backends
by Brian Rosmaita 17 Oct '22

17 Oct '22

Best practices when configuring Glance with COW backends --- ### Summary ### When deploying Glance in a popular configuration where Glance shares a common storage backend with Nova and/or Cinder, it is possible to open some known attack vectors by which malicious data modification can occur. This note reviews the known issues and suggests a Glance deployment configuration that can mitigate such attacks. ### Affected Services / Software ### Glance, all supported releases (Queens through Zed) ### Discussion ### This note applies to you if you are operating an end-user-facing glance-api service with the 'show_multiple_locations' option set to True (the default value is False) or if your end-user-facing glance-api has the 'show_image_direct_url' option set to True (default value is False). Our recommendation is that the image "locations" and "direct_url" fields [0] *never* be displayed to end users in a cloud. This can be accomplished by running two glance-api services: - A "user-facing" glance-api that is accessible to end users and which appears in users' service catalogs. - An "internal-only-facing" glance-api that is accessible only to those services that require access to the 'direct_url' or image location fields, and which is protected by firewalls from access by end users. (Nova, Cinder, and Ironic all have configuration options to specify the Glance API endpoint each service uses [1].) This dual glance-api deployment was suggested in "Known Issues" in Glance release notes in the Rocky [2] through Ussuri releases, but it seems that the idea has not received sufficient attention. Hence this security note. The attack vector that becomes available when image locations are exposed to end users was originally outlined in OSSN-0065 [3], though that note was not clear about the attack surface or mitigation, and contained some forward-looking statements that were not fulfilled. The attack vector is: [A] malicious user could create an image in Glance, set an additional location on that image pointing to an altered image, then delete the original location, so that consumers of the original image would unwittingly be using the malicious image. Note, however, that this attack vector cannot change the original image's checksum, and it is limited to images that are owned by the attacker. OSSN-0065 suggests that this is only an issue when users do not checksum their image data. It neglects the fact that in some popular deployment configurations in which Nova creates a root disk snapshot, data is never uploaded to Glance, but instead a snapshot is created directly in the backend and Nova creates a Glance image record with "size" 0 and an empty "os_hash_value" [4], making it impossible to compare the hash of downloaded image data to the value maintained by Glance. Further, when Nova is configured to use the same storage for ephemeral disks that is used as a Glance image store, Nova efficiently creates a server root disk directly in the backend without checksumming the image data. This is an intentional design choice to optimize storage space and host resources, but an implication is that even if the image record has a recorded hash, it is not being checked at the point of image consumption. Similarly, when using a shared backend, or a cinder glance_store, Cinder will efficiently clone a volume created from an image directly in the backend without checksumming the image data. Again, this is done intentionally in order to optimize resources, but it is important to be aware of the security tradeoff being made by this configuration. In other words, if the image data is not going to be checked at the point of image consumption, then extra care needs to be taken to ensure the integrity of the data path. OSSN-0065 suggested that the attack vector of substituting image data by modifying the image locations could be addressed by using policies, but that has turned out not to be the case. The only way currently to mitigate this vector is to deploy glance-api in a dual configuration as described above, namely with an internal-only-facing glance-api used by Nova and Cinder (that has show_multiple_locations enabled), and an end-user-facing glance-api (that has show_multiple_locations disabled). So far the focus has been on 'show_multiple_locations'. When that setting is disabled in Glance, it is not possible to manipulate the locations via the OpenStack Images API. Keep in mind, however, that in any Glance/Nova/Cinder configuration where Nova and/or Cinder do copy-on-write directly in the image store, image data transfer takes place outside Glance's image data download path, and hence the os_hash_value is *not* checked. Thus, if the backend store is itself compromised and image data is replaced directly in the backend, the substitution will *not* be detected. This brings us to the 'show_image_direct_url' option, which includes a "direct_url" field in the image-show response that can be used by various OpenStack services to consume images directly from the storage backend. Exposing the 'direct_url' to end users leaks information about the storage backend. What exactly that information consists of depends upon the backend in use and how it is configured, but in general, it is not a good idea to provide hints that could be useful to malicious actors in their attempts to compromise the backend storage by some type of independent exploit. The 'direct_url', being read-only, may appear innocuous, but its use by services is usually to perform some kind of optimized image data access that most likely does not include computing a hash of the image data. We therefore recommend that OpenStack services that require exposure of the 'direct_url' image property be similarly configured to use an internal-only-facing glance-api. It is worth nothing that end users who wish to download an image do not require access to the 'direct_url' image property because they can simply use the image-data-download API call [5]. ### Recommended Actions ### A glance-api service with 'show_multiple_locations' enabled should *never* be exposed directly to end users. This setting should only be enabled on an internal-only-facing glance-api that is used by OpenStack services that require access to image locations. This could be done, for example, by running two glance-api services with different configuration files and using the appropriate configuration options for each service to specify the Image API endpoint to access, and making sure the special internal endpoint is firewalled in such a way that only the appropriate OpenStack services can contact it. Similarly, enabling 'show_image_direct_url' exposes information about the storage backend that could be of use to malicious actors in as yet unknown exploits, so it should likewise only be enabled on an internal-only-facing glance-api. ### Notes / References ### [0] https://docs.openstack.org/api-ref/image/v2/index.html#show-image-schema [1] Nova and Ironic use 'endpoint_override' in the '[glance]' section of the configuration file; Cinder uses 'glance_api_servers' in the '[DEFAULT]' section. [2] OSSN-0065: https://wiki.openstack.org/wiki/OSSN/OSSN-0065 [3] The Glance "multihash" metadata pair of 'os_hash_algo' and 'os_hash_value' were introduced in Rocky to replace the legacy md5 'checksum' field. The python-glanceclient has used multihash checksumming for download verification since version 2.13.0. [4] https://docs.openstack.org/releasenotes/glance/rocky.html#known-issues [5] https://docs.openstack.org/api-ref/image/v2/index.html?#download-binary-ima… ### Contacts / References ### Author: Brian Rosmaita, Red Hat This OSSN : https://wiki.openstack.org/wiki/OSSN/OSSN-0090 Original LaunchPad Bug : https://bugs.launchpad.net/ossn/+bug/1990157 Mailing List : [Security] tag on openstack-discuss(a)lists.openstack.org OpenStack Security Project : https://launchpad.net/~openstack-ossg CVE: none

1 0

OpenStack Zed is officially released!
by Előd Illés 05 Oct '22

05 Oct '22

Hello OpenStack community, I'm excited to announce the final releases for the components of OpenStack Zed, which conclude the Zed development cycle. You will find a complete list of all components, their latest versions, and links to individual project release notes documents listed on the new release site. https://releases.openstack.org/zed/ Congratulations to all of the teams who have contributed to this release! Our next production cycle, 2023.1 Antelope, has already started. We will meet at the Virtual Project Team Gathering, October 17-21, 2022, to plan the work for the upcoming cycle. I hope to see you there! Thanks, Előd Illés and the Release Management team

1 0

Re: [openstack-announce] [all][elections][tc][ptl] Antelope: Technical Committee & PTL Election Conclusion & Results
by Ian Y. Choi 20 Sep '22

20 Sep '22

(Re-sending with better formatting) Thank you to the electorate, to all those who voted and to all candidates who put their name forward for Technical Committee (TC) & Project Team Lead (PTL) in this election. A healthy, open process breeds trust in our decision making capability thank you to all those who make this process possible. 1. Please join me in congratulating the 4 newly elected members of the TC: Dan Smith (dansmith) Ghanshyam Mann (gmann) Jay Faulkner (JayF) Dmitriy Rabotyagov (noonedeadpunk) Full results: https://civs1.civs.us/cgi-bin/results.pl?id=E_41d42603087bcf58 2. For the results of the PTL election process, please join me in extending congratulations to the following PTLs: * Barbican : Grzegorz Grasza (xek) * Blazar : Pierre Riteau (priteau) * Cinder : Rajat Dhasmana () * Cloudkitty : Rafael Weingartner () * Cyborg : alex song () * Designate : Michael Johnson (johnsom) * Ec2 Api : Andrey Pavlov (andrey-mp) * Freezer : ge cong () * Glance : Pranali Deore (pdeore) * Heat : Brendan Shephard () * Horizon : Vishal Manchanda (vishalmanchanda) * Ironic : Jay Faulkner (JayF) * Kolla : Michal Nasiadka (mnasiadka) * Kuryr : Roman Dobosz (gryf) * Magnum : Jake Yip (jakeyip) * Manila : Carlos Silva (carloss) * Monasca : Martin Chacon Piza (chaconpiza) * Murano : Rong Zhu (zhurong) * Neutron : Rodolfo Alonso Hernandez () * Nova : Sylvain Bauza (bauzas) * Octavia : Gregory Thiemonge (gthiemonge) * Openstack Chef : Lance Albertson (Ramereth) * OpenStack Helm : Gage Hugo (gagehugo) * OpenStackAnsible : Dmitriy Rabotyagov (noonedeadpunk) * OpenStackSDK : Artem Goncharov (gtema) * Puppet OpenStack : Takashi Kajinami (tkajinam) * Quality Assurance : Martin Kopec (kopecmartin) * Rally : Andrey Kurilin (andreykurilin) * Skyline : Wenxiang Wu (wu_wenxiang) * Solum : Rong Zhu (zhurong) * Storlets : Takashi Kajinami (tkajinam) * Tacker : Yasufumi Ogawa (yasufum) * Telemetry : Matthias Runge (mrunge) * Tripleo : Rabi Mishra (ramishra) * Trove : wu chunyang (wuchunyang) * Vitrage : Eyal Bar-Ilan () * Watcher : chen ke () * Winstackers : Lucian Petrut () * Zaqar : Hao Wang () PTL Election: * Ironic: https://civs1.civs.us/cgi-bin/results.pl?id=E_c7d81a19f30c67f0 Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all of the candidates, having a good group of candidates helps engage the community in our democratic process. Thank you to all who voted and who encouraged others to vote. We need to ensure your voice is heard. Thank you for another great round. Ian Y. Choi, on behalf of the OpenStack Technical Election Officials On Tue, Sep 20, 2022 at 11:57 PM Ian Y. Choi <ianyrchoi(a)gmail.com> wrote: > Thank you to the electorate, to all those who voted and to all candidates > who put their name forward for Technical Committee (TC) & Project Team Lead > (PTL) in this election. A healthy, open process breeds trust in our > decision making capability thank you to all those who make this process > possible. > > 1. Please join me in congratulating the 4 newly elected members of the TC: > Dan Smith (dansmith)Ghanshyam Mann (gmann)Jay Faulkner (JayF)Dmitriy > Rabotyagov (noonedeadpunk) > Full results: > https://civs1.civs.us/cgi-bin/results.pl?id=E_41d42603087bcf58 > 2. For the results of the PTL election process, please join me in > extending congratulations to the following PTLs: > * Barbican : Grzegorz Grasza (xek)* Blazar : Pierre > Riteau (priteau)* Cinder : Rajat Dhasmana ()* Cloudkitty > : Rafael Weingartner ()* Cyborg : alex song ()* Designate > : Michael Johnson (johnsom)* Ec2 Api : Andrey Pavlov > (andrey-mp)* Freezer : ge cong ()* Glance : Pranali > Deore (pdeore)* Heat : Brendan Shephard ()* Horizon > : Vishal Manchanda (vishalmanchanda)* Ironic : Jay Faulkner > (JayF)* Kolla : Michal Nasiadka (mnasiadka)* Kuryr > : Roman Dobosz (gryf)* Magnum : Jake Yip (jakeyip)* Manila > : Carlos Silva (carloss)* Monasca : Martin Chacon Piza > (chaconpiza)* Murano : Rong Zhu (zhurong)* Neutron : > Rodolfo Alonso Hernandez ()* Nova : Sylvain Bauza (bauzas)* > Octavia : Gregory Thiemonge (gthiemonge)* Openstack Chef : > Lance Albertson (Ramereth)* OpenStack Helm : Gage Hugo (gagehugo)* > OpenStackAnsible : Dmitriy Rabotyagov (noonedeadpunk)* OpenStackSDK : > Artem Goncharov (gtema)* Puppet OpenStack : Takashi Kajinami (tkajinam)* > Quality Assurance : Martin Kopec (kopecmartin)* Rally : Andrey > Kurilin (andreykurilin)* Skyline : Wenxiang Wu (wu_wenxiang)* > Solum : Rong Zhu (zhurong)* Storlets : Takashi > Kajinami (tkajinam)* Tacker : Yasufumi Ogawa (yasufum)* > Telemetry : Matthias Runge (mrunge)* Tripleo : Rabi > Mishra (ramishra)* Trove : wu chunyang (wuchunyang)* Vitrage > : Eyal Bar-Ilan ()* Watcher : chen ke ()* Winstackers > : Lucian Petrut ()* Zaqar : Hao Wang () > PTL Election:* Ironic: > https://civs1.civs.us/cgi-bin/results.pl?id=E_c7d81a19f30c67f0 > Election process details and results are also available here: > https://governance.openstack.org/election/ > Thank you to all of the candidates, having a good group of candidates > helps engage the community in our democratic process. > Thank you to all who voted and who encouraged others to vote. We need to > ensure your voice is heard. > Thank you for another great round. > > Ian Y. Choi, on behalf of the OpenStack Technical Election Officials >

1 0

OpenStack Yoga is officially released!
by Előd Illés 30 Mar '22

30 Mar '22

Hello OpenStack community, I'm excited to announce the final releases for the components of OpenStack Yoga, which conclude the Yoga development cycle. You will find a complete list of all components, their latest versions, and links to individual project release notes documents listed on the new release site. https://releases.openstack.org/yoga/ Congratulations to all of the teams who have contributed to this release! Our next production cycle, Zed, has already started. We will meet at the virtual Project Team Gathering next week (April 4 - 8, 2022) to plan the work for the upcoming cycle. We hope to see you there! Thanks, Előd Illés and the Release Management team

1 0

[all][elections][tc] Technical Committee Election Results
by Ghanshyam Mann 25 Feb '22

25 Feb '22

For 5 TC open seats, there were 5 candidates so no election was required. Please join me in congratulating the 5 newly elected members of the Technical Committee (TC). Amy Marrich (spotz) Arne Wiebalck (arnewiebalck) Kristi Nikolla (knikolla) Brian Rosmaita (rosmaita) Sławek Kapłoński (slaweq) Full results: no election required. Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all of the candidates, having a good group of candidates helps engage the community in our democratic process. Thank you for another great round. -gmann, on behalf of the OpenStack Technical Election Officials

1 0

[all][elections][ptl] Project Team Lead Zed Election Conclusion and Results
by Ghanshyam Mann 25 Feb '22

25 Feb '22

Thank you to the all candidates who put their names forward for Project Team Lead (PTL) in this election. A healthy, open process breeds trust in our decision making capability. Thank you to all those who make this process possible. Now for the results of the PTL election process, please join me in extending congratulations to the following PTLs: * Barbican : Douglas Mendizábal * Blazar : Pierre Riteau * Cinder : Rajat Dhasmana * Cloudkitty : Rafael Weingartner * Cyborg : Bailin Zhang * Designate : Michael Johnson * Ec2 Api : Andrey Pavlov * Glance : Abhishek Kekane * Horizon : Vishal Manchanda * Ironic : Iury Gregory Melo Ferreira * Keystone : Douglas Mendizábal * Kolla : Michal Nasiadka * Kuryr : Maysa de Macedo Souza * Manila : Carlos Silva * Neutron : Lajos Katona * Nova : Sylvain Bauza * Octavia : Gregory Thiemonge * OpenStack Helm : Gage Hugo * OpenStackAnsible : Dmitriy Rabotyagov * OpenStackSDK : Artem Goncharov * Puppet OpenStack : Takashi Kajinami * Quality Assurance : Martin Kopec * Rally : Andrey Kurilin * Release Management : Elod Illes * Requirements : Matthew Thode * Sahara : Fossen Qiu * Storlets : Takashi Kajinami * Swift : Tim Burke * Tacker : Yasufumi Ogawa * Telemetry : Matthias Runge * Tripleo : James Slagle * Vitrage : Eyal Bar-Ilan * Winstackers : Lucian Petrut For remaining projects (without PTL candaidates), TC will work on the appointment. Elections: no election required. Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all involved in the PTL election process -gmann, on behalf of the OpenStack Technical Election Officials

1 0

OpenStack Xena is officially released!
by Herve Beraud 06 Oct '21

06 Oct '21

Hello OpenStack community, I'm excited to announce the final releases for the components of OpenStack Xena, which conclude the Xena development cycle. You will find a complete list of all components, their latest versions, and links to individual project release notes documents listed on the new release site. https://releases.openstack.org/xena/ Congratulations to all of the teams who have contributed to this release! Our next production cycle, Yoga, has already started. We will virtually meet on October 18-22 at the Project Team Gathering to plan the work for the upcoming cycle. I hope to see you there! Thanks, Hervé Beraud and the Release Management team. -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

1 0

[OSSA-2021-002] Nova: Open Redirect in noVNC proxy (CVE-2021-3654)
by Jeremy Stanley 27 Sep '21

27 Sep '21

=========================================== OSSA-2021-002: Open Redirect in noVNC proxy =========================================== :Date: July 29, 2021 :CVE: CVE-2021-3654 Affects ~~~~~~~ - Nova: <21.2.3, >=22.0.0 <22.2.3, >=23.0.0 <23.0.3 Description ~~~~~~~~~~~ Swe Aung, Shahaan Ayyub, and Salman Khan with the Monash University Cyber Security team reported a vulnerability affecting Nova's noVNC proxying implementation which exposed access to a well-known redirect behavior in the Python standard library's http.server.SimpleHTTPRequestHandler and thus noVNC's WebSockifyRequestHandler which uses it. By convincing a user to follow a specially-crafted novncproxy URL, the user could be redirected to an unrelated site under control of the attacker in an attempt to convince them to divulge credentials or other sensitive data. All Nova deployments with novncproxy enabled are affected. Errata ~~~~~~ The initial fix did not take into account the possibility of bypass using exactly three slashes. This update provides a more thorough revised fix for the issue. The affected versions list has been updated to indicate versions expected to include the newer solution. Patches ~~~~~~~ - https://review.opendev.org/791807 (Train) - https://review.opendev.org/806629 (errata 1) (Train) - https://review.opendev.org/791806 (Ussuri) - https://review.opendev.org/806628 (errata 1) (Ussuri) - https://review.opendev.org/791805 (Victoria) - https://review.opendev.org/806626 (errata 1) (Victoria) - https://review.opendev.org/791577 (Wallaby) - https://review.opendev.org/805818 (errata 1) (Wallaby) - https://review.opendev.org/791297 (Xena) - https://review.opendev.org/805654 (errata 1) (Xena) Credits ~~~~~~~ - Swe Aung from Monash University Cyber Security team (CVE-2021-3654) - Shahaan Ayyub from Monash University Cyber Security team (CVE-2021-3654) - Salman Khan from Monash University Cyber Security team (CVE-2021-3654) References ~~~~~~~~~~ - https://launchpad.net/bugs/1927677 - https://bugs.python.org/issue32084 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-3654 Notes ~~~~~ - The stable/train branch is under extended maintenance and will receive no new point releases, but a patch for it is provided as a courtesy. OSSA History ~~~~~~~~~~~~ - 2021-09-27 - Errata 1 - 2021-07-29 - Original Version -- Jeremy Stanley

1 0

[OSSA-2021-006] Neutron: Routes middleware memory leak for nonexistent controllers (CVE-2021-40797)
by Jeremy Stanley 09 Sep '21

09 Sep '21

======================================================================== OSSA-2021-006: Routes middleware memory leak for nonexistent controllers ======================================================================== :Date: September 09, 2021 :CVE: CVE-2021-40797 Affects ~~~~~~~ - Neutron: <16.4.1, >=17.0.0 <17.2.1, >=18.0.0 <18.1.1 Description ~~~~~~~~~~~ Slawek Kaplonski with Red Hat reported a vulnerability in Neutron's routes middleware. By making API requests involving nonexistent controllers, an authenticated user may cause the API worker to consume increasing amounts of memory, resulting in API performance degradation or denial of service. All Neutron deployments are affected. Patches ~~~~~~~ - https://review.opendev.org/807638 (Queens) - https://review.opendev.org/807637 (Rocky) - https://review.opendev.org/807636 (Stein) - https://review.opendev.org/807635 (Train) - https://review.opendev.org/807634 (Ussuri) - https://review.opendev.org/807633 (Victoria) - https://review.opendev.org/807632 (Wallaby) - https://review.opendev.org/807335 (Xena) Credits ~~~~~~~ - Slawek Kaplonski from Red Hat (CVE-2021-40797) References ~~~~~~~~~~ - https://launchpad.net/bugs/1942179 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-40797 Notes ~~~~~ - The stable/train, stable/stein, stable/rocky, and stable/queens branches are under extended maintenance and will receive no new point releases, but patches for them are provided as a courtesy. -- Jeremy Stanley

1 0

[all][elections][ptl] Project Team Lead Election Conclusion and Results
by Ian Y. Choi 08 Sep '21

08 Sep '21

Thank you to the electorate, to all those who voted and to all candidates who put their name forward for Project Team Lead (PTL) in this election. A healthy, open process breeds trust in our decision making capability thank you to all those who make this process possible. Now for the results of the PTL election process, please join me in extending congratulations to the following PTLs: * Barbican : Douglas Mendizábal * Blazar : Pierre Riteau * Cinder : Brian Rosmaita * Cloudkitty : Rafael Weingartner * Cyborg : Bailin Zhang * Designate : Michael Johnson * Ec2 Api : Andrey Pavlov * Freezer : ge cong * Glance : Abhishek Kekane * Horizon : Vishal Manchanda * Ironic : Iury Gregory Melo Ferreira * Kolla : Michal Nasiadka * Kuryr : Maysa de Macedo Souza * Magnum : Feilong Wang * Manila : Goutham Pacha Ravi * Masakari : Radosław Piliszek * Murano : Rong Zhu * Neutron : Lajos Katona * Nova : Sylvain Bauza * Octavia : Gregory Thiemonge * OpenStack Charms : Alex Kavanagh * Openstack Chef : Lance Albertson * OpenStack Helm : Gage Hugo * OpenStackAnsible : Dmitriy Rabotyagov * OpenStackSDK : Artem Goncharov * Quality Assurance : Martin Kopec * Rally : Andrey Kurilin * Release Management : Elod Illes * Requirements : Matthew Thode * Senlin : XueFeng Liu * Solum : Rong Zhu * Storlets : Takashi Kajinami * Swift : Tim Burke * Tacker : Yasufumi Ogawa * Telemetry : Matthias Runge * Tripleo : James Slagle * Trove : Lingxian Kong * Vitrage : Eyal Bar-Ilan * Watcher : chen ke * Winstackers : Lucian Petrut * Zaqar : wang hao Elections: * Cyborg: https://civs1.civs.us/cgi-bin/results.pl?id=E_807146df7d6a1d3b Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all involved in the PTL election process, Ian Y. Choi, on behalf of the OpenStack Technical Election Officials

1 0