- openstack-discuss - lists.openstack.org

[TripleO] Removal of TripleO Zed Integration and Component Lines
by Ronelle Landy 17 May '23

17 May '23

Hello All, Removal of TripleO Zed Integration and Component Lines Per the decision to not maintain TripleO after the Zed release [1], the Zed integration and component lines are being removed in the following patches: https://review.rdoproject.org/r/c/config/+/48073 https://review.rdoproject.org/r/c/config/+/48074 https://review.rdoproject.org/r/c/rdo-jobs/+/48075 To be clear, please note that following these changes, the gate for stable/zed TripleO repos will no longer be updated or maintained. Per the earlier communications, there should be no more patches submitted for stable/zed TripleO repos, and any backports will go to stable/wallaby or stable/train. The last promoted release of Zed through TripleO is: https://trunk.rdoproject.org/centos9-zed/current-tripleo/delorean.repo (hash:61828177e94d5f179ee0885cf3eee102), which was promoted on 05/15/2023. The Ceph promotion lines related to Zed are also removed in the above patches. Check/gate testing for the master branch is in process of being removed as well (https://review.opendev.org/c/openstack/tripleo-ci/+/882759) [1] https://review.opendev.org/c/openstack/governance/+/878799

5 8

[Glance] Weekly Meeting Cancelled
by Pranali Deore 16 May '23

16 May '23

Hello, As per the discussion [1], glance weekly meeting is cancelled for this week as most of the team members will not be around. For next week - 25th May, we will have the meeting if anything comes urgent, otherwise we'll directly meet on Thursday 1st June. [1]: https://meetings.opendev.org/meetings/glance/2023/glance.2023-05-11-13.59.l… Thanks & Regards, Pranali Deore

1 0

[keystone][horizon][kolla-ansible] user access specific domain
by James Leong 16 May '23

16 May '23

Hi all, I am playing around with the domain in the yoga version of OpenStack using kolla-ansible as the deployment tool. I have set up Globus as my authentication tool. However, I am curious if it is possible to log in to an existing OpenStack user account via federated login (based on Gmail) In my case, first, I created a user named "James" in one of the domains called federated_login. When I attempt to log in, a new user is created in the default domain instead of the federated_login domain. Below is a sample of my globus.json. [{"local": [ { "user": { "name":"{0}, "email":"{2} }, "group":{ "name": "federated_user", "domain: {"name":"{1} } } ], "remote": [ { "type":"OIDC-name"}, { "type":"OIDC-organization"},{"type":"OIDC-email"} ] }] Apart from the above question, is there another easier way of restricting users from login in via federated? For example, allow only existing users on OpenStack with a specific email to access the OpenStack dashboard via federated login. Best Regards, James

2 3

[Storyboard] How to reach the Storyboard operators / maintainers?
by Christian Rohmann 16 May '23

16 May '23

Hello, a while back I opened a story / bug on storyboard (https://storyboard.openstack.org/#!/story/2010689) about improving email deliver-ability. I am uncertain if that was the right place to raise this issue or if there is another channel. In any case, could someone kindly point me or this suggestion into the right direction? Regards Christian

2 1

RMQ + Juno
by Ken D'Ambrosio 16 May '23

16 May '23

Yeah, I know. 173 versions out of date. But we've got an old cloud kicking around and it's having some weird problems, and we're trying to eliminate things. One thing is RMQ -- we were wondering if anyone knows what would happen if we flushed all the queues; we'd kinda like to start that from a clean slate. Is that a Bad Idea(tm), or would stuff simply re-populate organically? Thanks! -Ken

3 2

[Port Creation failed] - openstack Wallaby
by Lokendra Rathour 16 May '23

16 May '23

Hi All, Was trying to create OpenStack VM in OpenStack wallaby release, not able to create VM, it is failing because of Port not getting created. The error that we are getting: nova-compute.log: 2023-05-16 18:15:35.495 7 INFO nova.compute.provider_config [req-faaf38e7-b5ee-43d1-9303-d508285f5ab7 - - - - -] No provider configs found in /etc/nova/provider_config. If files are present, ensure the Nova process has access. 2023-05-16 18:15:35.549 7 ERROR nova.cmd.common [req-8842f11c-fe5a-4ad3-92ea-a6898f482bf0 - - - - -] No db access allowed in nova-compute: File "/usr/bin/nova-compute", line 10, in <module> sys.exit(main()) File "/usr/lib/python3.6/site-packages/nova/cmd/compute.py", line 59, in main topic=compute_rpcapi.RPC_TOPIC) File "/usr/lib/python3.6/site-packages/nova/service.py", line 264, in create utils.raise_if_old_compute() File "/usr/lib/python3.6/site-packages/nova/utils.py", line 1068, in raise_if_old_compute ctxt, ['nova-compute']) File "/usr/lib/python3.6/site-packages/nova/objects/service.py", line 563, in get_minimum_version_all_cells binaries) File "/usr/lib/python3.6/site-packages/nova/context.py", line 544, in scatter_gather_all_cells fn, *args, **kwargs) File "/usr/lib/python3.6/site-packages/nova/context.py", line 432, in scatter_gather_cells with target_cell(context, cell_mapping) as cctxt: File "/usr/lib64/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) neutron/ovn-metadata-agent.log 2023-05-16 22:33:41.871 45204 ERROR ovsdbapp.backend.ovs_idl.command ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Chassis_Private with name=b6e6f0d3-40c6-4d4e-8ef6-c935fa027bd2 2023-05-16 22:33:41.871 45204 ERROR ovsdbapp.backend.ovs_idl.command 2023-05-16 22:36:41.876 45204 ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run txn.results.put(txn.do_commit()) File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit command.run_idl(txn) File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 172, in run_idl record = self.api.lookup(self.table, self.record) File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup return self._lookup(table, record) File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup row = idlutils.row_by_value(self, rl.table, rl.column, record) File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value raise RowNotFound(table=table, col=column, match=match) ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Chassis_Private with name=b6e6f0d3-40c6-4d4e-8ef6-c935fa027bd2 any input to help get this issue fixed would be of great help. thanks -- ~ Lokendra skype: lokendrarathour

1 0

[Kolla] how to build kolla containers with kolla-build for specific branch
by wodel youchi 16 May '23

16 May '23

Hi, Could you help me understand the process of building kolla containers from source? I am not a developer but I want to be to build containers rapidly especially when there is an urgent patch. I am using Yoga branch which is the 14th version, do I need to use the same version of kolla-build to build containers or it matters not? Regards.

3 5

[neutron] CI meeting cancelled this week
by Slawek Kaplonski 16 May '23

16 May '23

Hi, I can't attend and chair today's Neutron CI meeting so we decided that we will cancel it if there will be no anything urgent. I think that CI in overall is running fine this week. Here's short summary of what I found: * Stadium projects - it seems that many projects may be affected by [1] - @Lajos, are You aware of those? * Grafana - it looks good this week * Rechecks - all good this week * Periodic jobs - looks good, only neutron-functional-with-sqlalchemy-master is still failing See You on the neutron team meeting today and on CI meeting next week. [1] https://review.opendev.org/c/openstack/neutron/+/879827 -- Slawek Kaplonski Principal Software Engineer Red Hat

2 1

[tc] Technical Committee next weekly meeting on May 16, 2023
by Nikolla, Kristi 16 May '23

16 May '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held on Tuesday, May 16, 2023 at 1800 UTC on #openstack-tc on OFTC IRC. I’m out of office this week, therefore this week’s meeting will be chaired by Jay Faulkner. Thank you Jay! The meeting’s agenda can be found on the wiki page at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting Thank you, Kristi Nikolla

1 0

[neutron] Bug Deputy Report May 8 - May 14
by Nurmatov Mamatisa 15 May '23

15 May '23

Hi, Below is the week summary of bug deputy report for last week. 2 bugs are not assigned. Details: Critical -------- - https://bugs.launchpad.net/neutron/+bug/2019097 - CI tests neutron-dynamic-routing broken - In progress: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/882815 - Assigned to Felix Huettner High ---- - https://bugs.launchpad.net/neutron/+bug/2018727 - [SRBAC] API policies for get_policy_*_rule are wrong - Fix released: https://review.opendev.org/c/openstack/neutron/+/882688 - Assigned to Slawek Kaplonski - https://bugs.launchpad.net/neutron/+bug/2019449 - Neutron server log file full of tracebacks in neutron-tempest-plugin-ovn job - Confirmed - Miguel Lavalle - https://bugs.launchpad.net/neutron/+bug/2019314 - On shutdown, linuxbridge agent queue on rabbitmq is not cleared - Confirmed - Unassigned Medium ------ - https://bugs.launchpad.net/neutron/+bug/2018967 - [fwaas] test_update_firewall_group fails randomly - Confirmed - Unassigned - https://bugs.launchpad.net/neutron/+bug/2018737 - neutron-dynamic-routing announces routes for disabled routers - In progress: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/882560 - Assigned to Felix Huettner - https://bugs.launchpad.net/neutron/+bug/2018989 - [SRBAC] FIP Port Forwarding policies should be available for PARENT_OWNER with proper role - Fix released: https://review.opendev.org/c/openstack/neutron/+/882691 - Assigned to Slawek Kaplonski - https://bugs.launchpad.net/neutron/+bug/2019119 - [sqlalchemy-20] Add the corresponding context to the upgrade checks methods - In progress: https://review.opendev.org/c/openstack/neutron/+/882865 - Assigned to Rodolfo Alonso - https://bugs.launchpad.net/neutron/+bug/2019132 - [OVN] ``OVNClient._add_router_ext_gw`` should use the context passed by the API call - In progress: https://review.opendev.org/c/openstack/neutron/+/882857 - Assigned to Rodolfo Alonso - https://bugs.launchpad.net/neutron/+bug/2019186 - [PostgreSQL] Issue with "get_scoped_floating_ips" query - In progress: https://review.opendev.org/c/openstack/neutron/+/882935 - Assigned to Rodolfo Alonso Low --- - https://bugs.launchpad.net/neutron/+bug/2019217 - [OVN] L3 port scheduler, defaults to use all chassis; stop this behavior - New - Assigned to Rodolfo Alonso Undecided --------- - https://bugs.launchpad.net/neutron/+bug/2019012 - When num of port mappings is substantial, the response time of List API so slow Best regards, Mamatisa Nurmatov Advanced Software Technology Lab / Cloud Technologies Research

1 0

(Openstack-glance) Image service Failing
by Adivya Singh 14 May '23

14 May '23

Hi Team, I have a scenario, where my glance nfs mount point are no longer require as the nfs external.server is down, i have remove all NFS variables from open stack user config file and user variable and delete the glance containet and try to bulid it again. But still it is trying to mount that path. Is therr any user variables to ignore this Regards Adivya Singh

1 1

Destroy an Openstack deployment and Keep Cinder Volumes
by A Monster 14 May '23

14 May '23

I have a deployment of openstack using kolla-ansible and LVM as a cinder backend, I want to make a kolla-ansible destroy to remove all openstack components and re deploy it again from scratch, however I want to keep the lvm volumes which is the backend for cinder, to re-use them for the fresh openstack deployment, and I was wondering if kolla-ansible destroys the lvm volumes used by cinder or no, and if you can suggest me a better alternative for what I'm trying to do. Thank you.

1 0

[kolla-ansible][yoga] fluentd problem with elasticsearch after update
by wodel youchi 13 May '23

13 May '23

Hi, I have finished the update of my openstack platform with newer containers. While verifying I noticed that fluentd container keeps restarting. In the log file I am having this : > 2023-05-03 16:07:59 +0100 [error]: #0 config error > file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError > error="Using Elasticsearch client 8.7.1 is not compatible for your > Elasticsearch server. Please check your using elasticsearch gem version and > Elasticsearch server." > 2023-05-03 16:07:59 +0100 [error]: Worker 0 finished unexpectedly with > status 2 > 2023-05-03 16:07:59 +0100 [info]: Received graceful stop > Those are the images I am using : (yogavenv) [deployer@rscdeployer ~]$ sudo docker images | grep -i elas 192.168.1.16:4000/openstack.kolla/centos-source-prometheus-elasticsearch-exporter yoga030523 b48f63ed0072 12 hours ago 539MB 192.168.1.16:4000/openstack.kolla/centos-source-elasticsearch yoga030523 3558611b0cf4 12 hours ago 1.2GB 192.168.1.16:4000/openstack.kolla/centos-source-elasticsearch-curator yoga030523 83a6b48339ea 12 hours ago 637MB (yogavenv) [deployer@rscdeployer ~]$ sudo docker images | grep -i fluen 192.168.1.16:4000/openstack.kolla/centos-source-fluentd yoga030523 bf6596e139e2 12 hours ago 847MB Any ideas? Regards. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free.www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2 6

[neutron] Gate broken, please don't recheck
by Brian Haley 12 May '23

12 May '23

Hi, With the latest release of neutronclient version 10.0.0 the neutron gate pep8 job is now failing. Please hold your rechecks until a fix [0] merges. There's also some old unused devstack code that uses this neutron-debug code, will remove that in a follow-up. Thanks, -Brian [0] https://review.opendev.org/c/openstack/neutron/+/883081

1 0

[release] Release countdown for week R-20, May 15-19
by Elõd Illés 12 May '23

12 May '23

Development Focus ----------------- We are now past the Bobcat-1 milestone. Teams should now be focused on feature development. General Information ------------------- Our next milestone in this development cycle will be Bobcat-2, on July 6th, 2023. This milestone is when we freeze the list of deliverables that will be included in the 2023.2 Bobcat final release, so if you plan to introduce new deliverables in this release, please propose a change to add an empty deliverable file in the deliverables/bobcat directory of the openstack/releases repository. Now is also generally a good time to look at bugfixes that were introduced in the master branch that might make sense to be backported and released in a stable release. If you have any question around the OpenStack release process, feel free to ask on this mailing-list or on the #openstack-release channel on IRC. Upcoming Deadlines & Dates -------------------------- OpenInfra Summit & PTG in Vancouver: June 13-15, 2023 Bobcat-2 Milestone: July 6th, 2023 Előd Illés irc: elodilles @ #openstack-release

1 0

Ussuri - Volume delete status not updated in horizon dashboard
by Vineet Thakur 12 May '23

12 May '23

Hi Team, The issue is about quota update in the horizon dashboard where volume deletion status is not updated. For example: We deleted the volume, cinder logs showing the deletion status as success and even in the dashboard's "volume" section as well. But in the dashboard quota view, the entry for volume existence is still available, and this problem is only for one volume . (not for all volumes) We have tried almost all the possible ways suggested in online blogs to remove this volume id. But still that volume is persisting in the dashboard quota. Could you please guide us what is the best practice to remove this volume id from the quota without affecting the production environment. Appreciate your help. Thank you. Kind Regards, Vineet

1 0

Did StarlingX Break?
by John van Ommen 12 May '23

12 May '23

I've installed StarlingX 10+ times using the iso "bootimage.iso" from their website. I opted to do a reinstall yesterday, and it keeps hitting the same error at the same spot in the install. I've been able to replicate the error three times using two different HP servers. The installer basically reaches the point where it's installing package number 866 out of 1,217 packages, and fails with this error: process [/usr/libexec/anaconda/anaconda-yum --config /tmp/anaconda-yum.conf --tsfile /mnt/sysimage/anaconda-yum.yumtx --rpmlog /tmp/rpm-script.log --installroot /mnt/sysimage --release 7 --arch x86_64 --macro __dbi_htconfig hash nofsync /etc/selinux/targeted/contexts/files/file_contexts exited with status 1 I did a Google search on this error, and there's a slightly similar error reported on a RedHat forum from someone who had a bad ISO. But I know my ISO is working; it's the exact same ISO I've already installed from 10+ times. In the event that I have a bad DIMM or a bad NVME, I tried an identical server, and was able to reproduce the same error. In an earlier message on the list today, someone reported that missing repos may be impacting StarlingX, which definitely makes me wonder if I'm being impacted by that. I'm on the CentOS based version of StarlingX, version 7.0 I also tried to install from the Debian based version (8.0) and that one wouldn't even boot properly. It basically asks you what type of install you want to do, then crashes.

2 1

[kolla-ansible] TLS and internal VIP
by Nicolas Froger 12 May '23

12 May '23

Hello, I was debugging the monitoring stack of our deployment and I noticed that our Prometheus could not reach the OpenStack Exporter. The error is about a certificate name mismatch because Prometheus is scraping the exporter with the internal IP address instead of the internal FQDN while the certificate we have is only valid for the internal FQDN. Indeed, the Prometheus config specifies kolla_internal_vip_address as a target and uses HTTPS when kolla_enable_tls_internal is true. Replacing the target with kolla_internal_fqdn which is a DNS name for which the certificate is valid fixed my issue. My question is the following: should the internal certificate also be valid for the internal VIP when kolla_enable_tls_internal is set to true or is it okay if it's only valid for the FQDN? In the later case, does it make sense if I open an issue to use the FQDN instead of the IP address in the Prometheus config? Regards, -- Nicolas Froger

1 0

[neutron][ironic] Postgres support?
by Julia Kreger 11 May '23

11 May '23

Greetings folks, I have spent a substantial amount of time investigating CI issues as of recent, and I noticed that at some point the legacy "ironic-tempest-pxe_ipmitool-postgres" CI job which has been kept around as a non-voting postgres support canary, is now failing on the master branch. Specifically, looking through the logs[0], it appears that structurally the queries are no longer compatible. np0033933847 neutron-server[90677]: WARNING oslo_db.sqlalchemy.exc_filters [None req-155d0299-07ef-433a-8576-467127c82be4 None None] DBAPIError exception wrapped.: psycopg2.errors.GroupingError: column "subnetpools.address_scope_id" must appear in the GROUP BY clause or be used in an aggregate function May 03 18:41:57.893335 np0033933847 neutron-server[90677]: LINE 2: ...standard_attr_id AS floatingips_standard_attr_id, subnetpool... May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ^ May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last): May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters self.dialect.do_execute( May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 736, in do_execute May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters) May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters psycopg2.errors.GroupingError: column "subnetpools.address_scope_id" must appear in the GROUP BY clause or be used in an aggregate function May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters LINE 2: ...standard_attr_id AS floatingips_standard_attr_id, subnetpool... May 03 18:41:57.893335 np0033933847 neutron-server[90677]: ERROR oslo_db.sqlalchemy.exc_filters ^ >From Ironic's point of view, we're wondering if this is expected? Is it likely to be fixed? If there are no plans to fix the DB queries, Ironic will have no choice but to drop the CI job. Thanks in advance! -Julia [0]: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/…

5 6

[ironic] Meeting cancelled; Metal3 PTG next Monday
by Jay Faulkner 11 May '23

11 May '23

Hi all, The Metal3 team informed us they are having a one day PTG, which will take place on May 15th, 2023 from 14:00 to 17:00 UTC. I strongly encourage any interested Ironic cores to review the invitation[1] and attend if interested. Because this overlaps our regularly scheduled Ironic meeting, and there is significant overlap between the Metal3 and Ironic communities, the Ironic meeting for Monday, May 15h is cancelled to allow our communities to collaborate without distraction. Thanks, Jay Faulkner Ironic PTL 1: https://groups.google.com/g/metal3-dev/c/62PCloXk-zQ

1 0

[kolla] [train] [neutron]
by Albert Braden 11 May '23

11 May '23

1 0

Extending the Forum
by Kendall Nelson 11 May '23

11 May '23

Hello Everyone, We received feedback that critical Forum conversations were not able to make the schedule due to the intentional work on our part to avoid PTG / Forum overlap which ended up restricting the number of Forum discussion proposals we were able to accept. To mitigate this, we have opened up the Forum room for a third day. We have created a form to collect critical Forum proposals. We will leave the form open for one week to give all timezones fair representation. The deadline to submit a proposal for a forum discussion is Thursday May, 18th at 7:00 UTC. Shortly after, the OpenInfra Foundation will make final decisions and program the Forum on Thursday. -Kendall Nelson [1] https://openinfrafoundation.formstack.com/forms/forum_expansion

1 0

[publiccloud-sig] Bi-weekly meeting times adjusted to 0700 UTC
by Felix Kronlage-Dammers 11 May '23

11 May '23

Hi, since some of us (to be honest, probably mostly me ;) have a conflicting meeting during summer time with the slot of the publiccloud-sig we discussed moving it to 0700 UTC during summer time. Since nobody in the past two meetings spoke against that, this was concluded yesterday. During summer time the publiccloud SIG will meet at 0700 UTC, not at 0800 UTC. The change is also reflected in the wiki page: https://wiki.openstack.org/wiki/PublicCloudSIG See you next time (and I'll set myself reminder to send out the reminder ;) felix -- GPG: 824CE0F0 / 2082 651E 5104 F989 4D18 BB2E 0B26 6738 824C E0F0 https://hazardous.org/ - fkr(a)hazardous.org - fkr@irc - @felixkronlage

1 0

[OSSA-2023-003] cinder, glance_store, nova, os-brick: Unauthorized volume access through deleted volume attachments (CVE-2023-2088)
by Jeremy Stanley 10 May '23

10 May '23

============================================================================ OSSA-2023-003: Unauthorized volume access through deleted volume attachments ============================================================================ :Date: May 10, 2023 :CVE: CVE-2023-2088 Affects ~~~~~~~ - Cinder: <20.2.1, >=21.0.0 <21.2.1, ==22.0.0 - Glance_store: <3.0.1, >=4.0.0 <4.1.1, >=4.2.0 <4.3.1 - Nova: <25.1.2, >=26.0.0 <26.1.2, ==27.0.0 - Os-brick: <5.2.3, >=6.0.0 <6.1.1, >=6.2.0 <6.2.2 Description ~~~~~~~~~~~ An unauthorized access to a volume could occur when an iSCSI or FC connection from a host is severed due to a volume being unmapped on the storage system and the device is later reused for another volume on the same host. **Scope:** Only deployments with iSCSI or FC volumes are affected. However, the fix for this issue includes a configuration change in Nova and Cinder that may impact you on your next upgrade regardless of what backend storage technology you use. See the *Configuration change* section below, and item 4(B) in the *Patches and Associated Deployment Changes* for details. This data leak can be triggered by two different situations. **Accidental case:** If there is a problem with network connectivity during a normal detach operation, OpenStack may fail to clean the situation up properly. Instead of force-detaching the compute node device, Nova ignores the error, assuming the instance has already been deleted. Due to this incomplete operation OpenStack may end up selecting the wrong multipath device when connecting another volume to an instance. **Intentional case:** A regular user can create an instance with a volume, and then delete the volume attachment directly in Cinder, which neglects to notify Nova. The compute node SCSI plumbing (over iSCSI/FC) will continue trying to connect to the original host/port/LUN, not knowing the attachment has been deleted. If a subsequent volume attachment re-uses the host/port/LUN for a different instance and volume, the original instance will gain access to it once the SCSI plumbing reconnects. Configuration Change -------------------- To prevent the intentional case, the Block Storage API provided by Cinder must only accept attachment delete requests from Nova for instance-attached volumes. A complicating factor is that Nova deletes an attachment by making a call to the Block Storage API on behalf of the user (that is, by passing the user's token), which makes the request indistinguishable from the user making this request directly. The solution is to have Nova include a service token along with the user's token so that Cinder can determine that the detach request is coming from Nova. The ability for Nova to pass a service token has been supported since Ocata, but has not been required until now. Thus, deployments that are not currently sending service user credentials from Nova will need to apply the relevant code changes and also make configuration changes to solve the problem. Patches and Associated Deployment Changes ----------------------------------------- Given the above analysis, a thorough fix must include the following elements: 1. The os-brick library must implement the ``force`` option for fibre channel, which which has only been available for iSCSI until now (covered by the linked patches). 2. Nova must call os-brick with the ``force`` option when disconnecting volumes from deleted instances (covered by the linked patches). 3. In deployments where Glance uses the cinder glance_store driver, glance must call os-brick with the ``force`` option when disconnecting volumes (covered by the linked patches). 4. Cinder must distinguish between safe and unsafe attachment delete requests and reject the unsafe ones. This part of the fix has two components: a. The Block Storage API will return a 409 (Conflict) for a request to delete an attachment if there is an instance currently using the attachment, **unless** the request is being made by a service (for example, Nova) on behalf of a user (covered by the linked patches). b. In order to recognize that a request is being made by a service on behalf of a user, Nova must be configured to send a service token along with the user token. If this configuration change is not made, the cinder change will reject **any** request to delete an attachment associated with a volume that is attached to an instance. Nova must be configured to send a service token to Cinder, and Cinder must be configured to accept service tokens. This is described in the following document and **IS NOT AUTOMATICALLY APPLIED BY THE LINKED PATCHES:** (Using service tokens to prevent long-running job failures) https://docs.openstack.org/cinder/latest/configuration/block-storage/servic… The Nova patch mentioned in step 2 includes a similar document more focused on Nova: doc/source/admin/configuration/service-user-token.rst 5. The cinder glance_store driver does not attach volumes to instances; instead, it attaches volumes directly to the Glance node. Thus, the Cinder change in step 4 will recognize an attachment-delete request coming from Glance as safe and allow it. (Of course, we expect that you will have applied the patches in steps 1 and 3 to your Glance nodes.) Errata ~~~~~~ An additional nova patch is required to fix a minor regression in periodic tasks and some nova-manage actions (errata 1). Also a patch to tempest is needed to account for behavior changes with fixes in place (errata 2). Patches ~~~~~~~ - https://review.opendev.org/882836 (2023.1/antelope cinder) - https://review.opendev.org/882851 (2023.1/antelope glance_store) - https://review.opendev.org/882858 (2023.1/antelope nova) - https://review.opendev.org/882859 (2023.1/antelope nova errata 1) - https://review.opendev.org/882843 (2023.1/antelope os-brick) - https://review.opendev.org/882835 (2023.2/bobcat cinder) - https://review.opendev.org/882834 (2023.2/bobcat glance_store) - https://review.opendev.org/882847 (2023.2/bobcat nova) - https://review.opendev.org/882852 (2023.2/bobcat nova errata 1) - https://review.opendev.org/882840 (2023.2/bobcat os-brick) - https://review.opendev.org/882876 (2023.2/bobcat tempest errata 2) - https://review.opendev.org/882869 (Wallaby nova) - https://review.opendev.org/882870 (Wallaby nova errata 1) - https://review.opendev.org/882839 (Xena cinder) - https://review.opendev.org/882855 (Xena glance_store) - https://review.opendev.org/882867 (Xena nova) - https://review.opendev.org/882868 (Xena nova errata 1) - https://review.opendev.org/882848 (Xena os-brick) - https://review.opendev.org/882838 (Yoga cinder) - https://review.opendev.org/882854 (Yoga glance_store) - https://review.opendev.org/882863 (Yoga nova) - https://review.opendev.org/882864 (Yoga nova errata 1) - https://review.opendev.org/882846 (Yoga os-brick) - https://review.opendev.org/882837 (Zed cinder) - https://review.opendev.org/882853 (Zed glance_store) - https://review.opendev.org/882860 (Zed nova) - https://review.opendev.org/882861 (Zed nova errata 1) - https://review.opendev.org/882844 (Zed os-brick) Credits ~~~~~~~ - Jan Wasilewski from Atman (CVE-2023-2088) - Gorka Eguileor from Red Hat (CVE-2023-2088) References ~~~~~~~~~~ - https://launchpad.net/bugs/2004555 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-2088 Notes ~~~~~ - Limited Protection Against Accidents... If you are only concerned with protecting against the accidental case described earlier in this document, steps 1-3 above should be sufficient. Note, however, that only applying steps 1-3 leaves your cloud wide open to the intentional exploitation of this vulnerability. Therefore, we recommend that the full fix be applied to all deployments. - Using Configuration as a Short-Term Mitigation... An alternative approach to mitigation can be found in OSSN-0092 https://wiki.openstack.org/wiki/OSSN/OSSN-0092 - The stable/xena and stable/wallaby branches are under extended maintenance and will receive no new point releases, but patches for them are provided as a courtesy where available. OSSA History ~~~~~~~~~~~~ - 2023-05-10 - Errata 2 - 2023-05-10 - Errata 1 - 2023-05-10 - Original Version -- Jeremy Stanley OpenStack Vulnerability Management Team

1 0

Cinder Bug Report 2023-05-10
by Sofia Enriquez 10 May '23

10 May '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> *Medium* - [rbac] Volume type's extra_specs dict is empty. <https://bugs.launchpad.net/cinder/+bug/2018467> - *Status:* Unassigned. - cinder services report_state hangs for a longer period than report_interval when losing database connection. <https://bugs.launchpad.net/cinder/+bug/2018595> - *Status:* Unassigned. *Low* - HPE 3par - Unable to use small QoS Latency value (less than 1) <https://bugs.launchpad.net/cinder/+bug/2018994> - *Status:* Fix proposed to master <https://review.opendev.org/c/openstack/cinder/+/882266>. Cheers -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0