July 2023 - openstack-discuss - lists.openstack.org

[tc] Technical Committee next weekly meeting today on July 18, 2023
by Nikolla, Kristi 18 Jul '23

18 Jul '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held Today on Tuesday, July 18, 2023, at 1800 UTC on #openstack-tc on OFTC IRC. Please find below the agenda for today's meeting: * Roll call * Follow up on past action items * knikolla will amend proposal for Unmaintained branches. * Extended Maintenance * https://etherpad.opendev.org/p/openstack-exteneded-maintenance * Release notes guidelines for SLURP/NON-SLURP cadence * Gate health check * Open Discussion and Reviews * https://review.opendev.org/q/projects:openstack/governance+is:open Thank you, Kristi Nikolla

1 0

[openstack][largescale-sig] Openstack multi region deployment
by Nguyễn Hữu Khôi 18 Jul '23

18 Jul '23

Hello guys, I am going to deploy openstack multi regions and I know that keystone replication is the most challenging. I plan to set up 2 regions which use centralize galera cluster(3 nodes). and one standby edge galera cluster(3 nodes) When region 1 is node available, I will map region 2 to use standby edge galera cluster. I hope you give me some experience and advice with multi regions. Thank you very much.

2 2

[ops] Built-in rolling-update/instance refresh capabilities
by Matthew Giassa 18 Jul '23

18 Jul '23

Good day, I was originally discussing this on the IRC channel, #openstack, and was advised to reach out to this mailing list. In short, does OpenStack (i.e. via Heat, Senlin, Masakari, or some other feature/plugin), have the ability to perform a rolling upgrade operation on a group of VMs (i.e. similar to the "rolling update" feature in K8S, or the "instance refresh" feature in AWS auto-scaling groups / ASGs). I've reviewed the available documentation, but all documents regarding "rolling upgrades" are with respect to a rolling upgrade of the OpenStack/IaaS stack itself (i.e. operator-level functionality) as opposed to groups of VMs managed by OpenStack (i.e. user-level functionality). I'd ideally like to use built-in functionality to handle this operation, rather than writing a service from scratch. I've also looked into tools such as Spinnaker.io and Harness.io, but could not find any FOSS or COTS tools for implementing rolling upgrades on OpenStack out-of-the-box (though if there are any options I've overlooked, I'm all ears). Thank you. -- ============================================================ Matthew Giassa e-mail: matthew(a)giassa.net website: www.giassa.net

1 0

CVE-2023-2088 for Charmed Openstack
by Corey Bryant 17 Jul '23

17 Jul '23

Bug LP:#2004555 (CVE-2023-2088) identified that “an unauthorised access to a volume could occur when an iSCSI or FC connection from a host is severed due to a volume being unmapped on the storage system and the device is later reused for another volume on the same host”. While the bug affects iSCSI and FC, the fix introduces a breaking change for configurations upgrading to the fixed packages if they do not have service token support enabled. The new code introduces mandatory use of service tokens to identify requests coming from a trusted control plane service. Service tokens are issued by Keystone for service users. For example, a token can be issued to the Nova service user. They have a particular trust level, defined by the service, that allows them to perform trusted actions. In the case of Nova, the action is detaching volumes from instances. Versions of Ubuntu OpenStack for Yoga and later will have new packages versions released with CVE patches applied that require the use of service tokens to enable the detaching and deletion of volumes on instances. These packages will be released on July 24, 2023. Versions of Ubuntu OpenStack for Xena and earlier will not have package updates released and will still be vulnerable to the “unauthorised detach of volumes” issue until a mitigation policy is applied. The mitigation policy provided requires the use of service tokens. For Charmed OpenStack releases, this means that some charms (identified below) must be upgraded to enable service tokens prior to the packages being upgraded and before the mitigation policy is applied. If not, users will lose the ability to detach volumes from their instances. = Patched packages = The following package versions include patches for CVE-2023-2088. These package versions deliver services that require the use of service tokens. Package versions prior to those listed will not be patched for CVE-2023-2088 and will therefore require policy updates for mitigation. Ubuntu 23.04 (Lunar) * nova >= 3:27.0.0-0ubuntu1.3 * cinder >= 2:22.0.0-0ubuntu1.3 * python-os-brick >= 6.2.0-0ubuntu2.3 * ironic >= 1:21.4.0-0ubuntu1.1 * python-glance-store >= 4.3.0-0ubuntu1.3 Ubuntu Cloud Archive Antelope * nova >= 3:27.0.0-0ubuntu1.3~cloud0 * cinder >= 2:22.0.0-0ubuntu1.3~cloud0 * python-os-brick >= 6.2.0-0ubuntu2.3~cloud0 * ironic >= 1:21.4.0-0ubuntu1.1~cloud0 * python-glance-store >= 4.3.0-0ubuntu1.3~cloud0 Ubuntu 22.10 (Kinetic) * nova >= 3:26.1.1-0ubuntu1.1 * cinder >= 2:21.2.0-0ubuntu1.1 * python-os-brick >= os-brick - 6.1.0-0ubuntu1.3 * ironic >= 1:21.1.0-0ubuntu1.1 * python-glance-store >= 4.1.0-0ubuntu1.3 Ubuntu Cloud Archive Zed * nova >= 3:26.1.1-0ubuntu1.1~cloud0 * cinder >= 2:21.2.0-0ubuntu1.1~cloud0 * python-os-brick >= os-brick - 6.1.0-0ubuntu1.3~cloud0 * ironic >= 1:21.1.0-0ubuntu1.1~cloud0 * python-glance-store >= 4.1.0-0ubuntu1.3~cloud0 Ubuntu 22.04 (Jammy) * nova >= 3:25.1.1-0ubuntu1.1 * cinder >= 2:20.2.0-0ubuntu1.1 * python-os-brick >= 5.2.2-0ubuntu1.2 * ironic >= 1:20.1.0-0ubuntu1.1 * python-glance-store >= 3.0.0-0ubuntu1.3 Ubuntu Cloud Archive Yoga * nova >= 3:25.1.1-0ubuntu1.1~cloud0 * cinder >= 2:20.2.0-0ubuntu1.1~cloud0 * python-os-brick >= 5.2.2-0ubuntu1.2~cloud0 * ironic >= 1:20.1.0-0ubuntu1.1~cloud0 * python-glance-store >= 3.0.0-0ubuntu1.3~cloud0 = Charms with Service Tokens Enabled by Default = Following is the list of charms and earliest revision that supports service tokens: Channel: 2023.1/stable * Keystone revision >= 645 * Cinder revision >= 639 * Nova-compute revision >= 678 * Nova-cloud-controller revision >= 674 * Ironic-api revision >= 54 * Ironic-conductor - s390x revision >= 151 - arm64 revision >= 152 - ppc64el revision >= 153 - amd64 revision >= 149 Channel: zed/stable * Keystone revision >= 646 * Cinder revision >= 647 * Nova-compute revision >= 683 * Nova-cloud-controller revision >= 676 * Ironic-api revision >= 53 * Ironic-conductor - s390x revision >= 148 - arm64 revision >= 154 - ppc64el revision >= 150 - amd64 revision >= 147 Channel: yoga/stable * Keystone revision >= 647 * Cinder revision >= 638 * Nova-compute revision >= 681 * Nova-cloud-controller revision >= 679 * Ironic-api revision >= 48 * Ironic-conductor revision >= 145 Channel: xena/stable * Keystone revision >= 651 * Cinder revision >= 640 * Nova-compute revision >= 680 * Nova-cloud-controller revision >= 677 * Ironic-api revision >= 50 * Ironic-conductor revision >= 143 Channel: wallaby/stable * Keystone revision >= 653 * Cinder revision >= 648 * Nova-compute revision >= 684 * Nova-cloud-controller revision >= 675 * Ironic-api revision >= 52 * Ironic-conductor revision >= 144 Channel: victoria/stable * Keystone revision >= 650 * Cinder revision >= 646 * Nova-compute revision >= 679 * Nova-cloud-controller revision >= 678 * Ironic-api revision >= 49 * Ironic-conductor revision >= 142 Channel: ussuri/stable * Keystone revision >= 652 * Cinder revision >= 645 * Nova-compute revision >= 682 * Nova-cloud-controller revision >= 680 * Ironic-api revision >= 51 * Ironic-conductor revision >= 155 = Using a custom Cinder policy as a mitigation = It’s possible to restrict the access to the operations affected by this CVE by providing a custom Cinder policy file that will prevent users from deleting attachments, detach and force detach. In order to use this mitigation, service token support must be enabled. The exact method for customising this depends on the deployment tooling being used. For Charmed OpenStack users it is the following: 1) Identify the nova service user id running the following command openstack user show --domain service_domain -f value -c id nova The command output looks like this ae034831d6bf41099327bf9d34988fbc 2) Create a policy file named cve-2023-2088-cinder.yaml with the following content: "is_service": "role:service or service_user_id:<nova_service_uuid>" "legacy_system_admin_or_project_member": "(role:admin) or (role:member and project_id:%(project_id)s)" "volume:attachment_delete": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_actions:terminate_connection": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_actions:detach": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_admin_actions:force_detach": "!" Replacing “<nova_service_uuid>” would result in a snippet that looks like this: "is_service": "role:service or service_user_id:ae034831d6bf41099327bf9d34988fbc" "legacy_system_admin_or_project_member": "(role:admin) or (role:member and project_id:%(project_id)s)" "volume:attachment_delete": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_actions:terminate_connection": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_actions:detach": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_admin_actions:force_detach": "!" 3) Create zip files from the yaml file: zip cve-2023-2088-cinder.zip cve-2023-2088-cinder.yaml 4) Attach the zip file as a resource to the cinder applications: juju attach-resource cinder policyd-override=cve-2023-2088-cinder.zip 5) Enable the overrides via the use-policyd-override charm option: juju config cinder use-policyd-override=true For more details on using policy overrides see the OpenStack Charm Guide. = Frequently Asked Questions = Q: What can I do if my deployed cloud doesn’t have service tokens enabled and my deployment tooling doesn’t have the option to enable them? A: Pin the version of the affected packages to block the upgrades to the versions listed in “Affected packages” section or newer, until service tokens are enabled. Q: For Charmed OpenStack, what’s the upgrade order for charms and deb packages? A: Upgrade the charms to the latest versions available in the charm channel tracked by your environment or at the least to the revisions listed in section “Charms with Service Token Enabled By Default”. After that, upgrade the deb packages with your preferred method (e.g. Landscape, apt-get, etc). = References = * CVE-2023-2088: https://ubuntu.com/security/CVE-2023-2088 * OSSA-2023-003 Unauthorized volume access through deleted volume attachments: https://security.openstack.org/ossa/OSSA-2023-003.html * OSSN-0092 Using Configuration as a Short-Term Mitigation for OSSA-2023-003: https://wiki.openstack.org/wiki/OSSN/OSSN-0092 * Bug 2004555: https://bugs.launchpad.net/nova/+bug/2004555 * OpenStack Charm Guide Policy overrides: https://docs.openstack.org/charm-guide/latest/admin/policy-overrides.html

1 0

CVE-2023-2088 for Ubuntu Openstack
by Corey Bryant 17 Jul '23

17 Jul '23

Bug LP:#2004555 (CVE-2023-2088) identified that “an unauthorised access to a volume could occur when an iSCSI or FC connection from a host is severed due to a volume being unmapped on the storage system and the device is later reused for another volume on the same host”. While the bug affects iSCSI and FC, the fix introduces a breaking change for configurations upgrading to the fixed packages if they do not have service token support enabled. The new code introduces mandatory use of service tokens to identify requests coming from a trusted control plane service. Service tokens are issued by Keystone for service users. For example, a token can be issued to the Nova service user. They have a particular trust level, defined by the service, that allows them to perform trusted actions. In the case of Nova, the action is detaching volumes from instances. Versions of Ubuntu OpenStack for Yoga and later will have new packages versions released with CVE patches applied that require the use of service tokens to enable the detaching and deletion of volumes on instances. These packages will be released on July 24, 2023. Versions of Ubuntu OpenStack for Xena and earlier will not have package updates released and will still be vulnerable to the “unauthorised detach of volumes” issue until a mitigation policy is applied. The mitigation policy provided requires the use of service tokens. = Patched packages = The following package versions include patches for CVE-2023-2088. These package versions deliver services that require the use of service tokens. Package versions prior to those listed will not be patched for CVE-2023-2088 and will therefore require policy updates for mitigation. Ubuntu 23.04 (Lunar) * nova >= 3:27.0.0-0ubuntu1.3 * cinder >= 2:22.0.0-0ubuntu1.3 * python-os-brick >= 6.2.0-0ubuntu2.3 * ironic >= 1:21.4.0-0ubuntu1.1 * python-glance-store >= 4.3.0-0ubuntu1.3 Ubuntu Cloud Archive Antelope * nova >= 3:27.0.0-0ubuntu1.3~cloud0 * cinder >= 2:22.0.0-0ubuntu1.3~cloud0 * python-os-brick >= 6.2.0-0ubuntu2.3~cloud0 * ironic >= 1:21.4.0-0ubuntu1.1~cloud0 * python-glance-store >= 4.3.0-0ubuntu1.3~cloud0 Ubuntu 22.10 (Kinetic) * nova >= 3:26.1.1-0ubuntu1.1 * cinder >= 2:21.2.0-0ubuntu1.1 * python-os-brick >= os-brick - 6.1.0-0ubuntu1.3 * ironic >= 1:21.1.0-0ubuntu1.1 * python-glance-store >= 4.1.0-0ubuntu1.3 Ubuntu Cloud Archive Zed * nova >= 3:26.1.1-0ubuntu1.1~cloud0 * cinder >= 2:21.2.0-0ubuntu1.1~cloud0 * python-os-brick >= os-brick - 6.1.0-0ubuntu1.3~cloud0 * ironic >= 1:21.1.0-0ubuntu1.1~cloud0 * python-glance-store >= 4.1.0-0ubuntu1.3~cloud0 Ubuntu 22.04 (Jammy) * nova >= 3:25.1.1-0ubuntu1.1 * cinder >= 2:20.2.0-0ubuntu1.1 * python-os-brick >= 5.2.2-0ubuntu1.2 * ironic >= 1:20.1.0-0ubuntu1.1 * python-glance-store >= 3.0.0-0ubuntu1.3 Ubuntu Cloud Archive Yoga * nova >= 3:25.1.1-0ubuntu1.1~cloud0 * cinder >= 2:20.2.0-0ubuntu1.1~cloud0 * python-os-brick >= 5.2.2-0ubuntu1.2~cloud0 * ironic >= 1:20.1.0-0ubuntu1.1~cloud0 * python-glance-store >= 3.0.0-0ubuntu1.3~cloud0 = Using a custom Cinder policy as a mitigation = It’s possible to restrict the access to the operations affected by this CVE by providing a custom Cinder policy file that will prevent users from deleting attachments, detach and force detach. In order to use this mitigation, service token support must be enabled. The exact method for customising this depends on the deployment tooling being used. The following steps can be used as a guildeline: 1) Identify the nova service user id running the following command openstack user show --domain service_domain -f value -c id nova The command output looks like this ae034831d6bf41099327bf9d34988fbc 2) Create and apply the following policy to cinder, using the output of the previous command: "is_service": "role:service or service_user_id:ae034831d6bf41099327bf9d34988fbc" "legacy_system_admin_or_project_member": "(role:admin) or (role:member and project_id:%(project_id)s)" "volume:attachment_delete": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_actions:terminate_connection": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_actions:detach": "rule:legacy_system_admin_or_project_member and rule:is_service" "volume_extension:volume_admin_actions:force_detach": "!" = Frequently Asked Questions = Q: What can I do if my deployed cloud doesn’t have service tokens enabled and my deployment tooling doesn’t have the option to enable them? A: Pin the version of the affected packages to block the upgrades to the versions listed in “Affected packages” section or newer, until service tokens are enabled. = References = * CVE-2023-2088: https://ubuntu.com/security/CVE-2023-2088 * OSSA-2023-003 Unauthorized volume access through deleted volume attachments: https://security.openstack.org/ossa/OSSA-2023-003.html * OSSN-0092 Using Configuration as a Short-Term Mitigation for OSSA-2023-003: https://wiki.openstack.org/wiki/OSSN/OSSN-0092 * Bug 2004555: https://bugs.launchpad.net/nova/+bug/2004555

1 0

[designate] About delegating PTR using shared zones on a shared internal network
by Thomas Goirand 17 Jul '23

17 Jul '23

Hi there! I have just watched Michael Johnson's video of the presentation at the last Vancouver summit: https://www.youtube.com/watch?v=zdhvNezU1w4 This was very interesting, and thanks Michael, for that awesome feature. In our use case, we created an OpenStack internal network, that is shared for all of our customers, so that they can allocate public IPs assigned directly to their instances, without the need to create a router and use NAT. For the moment (since we don't have the shared zones feature yet), we manage the PTR record through support tickets. We'd like to move forward, and delegate the PTR handling to our customer, whenever they allocate a port on that specific shared network. How can this be done? Should we create a new sink of our own, so that we would automate the the shared zone creation, so that it would delegate the in-addr.arpa. range, when a port on that network is created, and the same way, remove the delegation (and records) whenever the port is deleted? Would this be something similar, probably, we the sink we implemented to automatically delete PTR records in floating IPs, available here [1]? [1] https://salsa.debian.org/openstack-team/services/designate/-/blob/debian/an… I'd love to have some pointers / examples, so we could implement this directly in Designate, so that the feature could be shared in the OpenStack community. For sure, we aren't the only public cloud provider with the use case... Cheers, Thomas Goirand (zigo)

2 3

UX
by A.R. Rumman 17 Jul '23

17 Jul '23

Hi, I am a UX Designer. I would love to contribute to OpenStack. So what's the next step or process? Thank you.

2 1

OpenStack "C" Release Voting Reminder
by Kristin Barrientos 17 Jul '23

17 Jul '23

Hi everyone, Just a reminder to get your votes in for the OpenStack “C” Release. Voting closes Tuesday, July 18! Vote here: https://civs1.civs.us/cgi-bin/vote.pl?id=E_852cf3e53467d72e&akey=1504cb6ad9… Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

[publiccloud-sig] Reminder - next meeting July 19th - 0700 UTC
by Felix Kronlage-Dammers 17 Jul '23

17 Jul '23

Hi everyone, this Wednesday the next meeting of the public cloud sig is going to happen. We meet on IRC in #openstack-operators at 0700 UTC. This time, we will switch to a jitsi-based video-call directly after start. The url will be posted on #openstack-operators at the time of the meeting. A preliminary agenda can be found in the pad: https://etherpad.opendev.org/p/publiccloud-sig-meeting See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG read^see you on Wednesday! felix -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[neutron] Bugs deputy report July 10th - 16th
by Miguel Lavalle 17 Jul '23

17 Jul '23

Hi, Here's last week's bugs deputy report: Critical ====== https://bugs.launchpad.net/neutron/+bug/2027817 neutron slow jobs before xena broken with ERROR: unknown environment 'slow'. Assigned to Ghanshyam Mann. Proposed fix: https://review.opendev.org/c/openstack/tempest/+/888590. In progress High ==== https://bugs.launchpad.net/neutron/+bug/2027605 [OVN][Trunk] Sometimes subport doesn't reach status ACTIVE after live-migration. Requires owner Medium ====== https://bugs.launchpad.net/neutron/+bug/2027595 [sqlalchemy-20] Check represent "network" model regardless of the elements order. Assigned to Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/888286. Merged. https://bugs.launchpad.net/neutron/+bug/2027602 [sqlalchemy-20] "FromClause.join" accepts only one model and an optional expression to define the join. Assigned to Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/888295. Merged. https://bugs.launchpad.net/neutron/+bug/2027604 [sqlalchemy-20] The Query.get() method is considered legacy as of the 1.x series of SQLAlchemy and becomes a legacy construct in 2.0. Assigned to Rodolfo Alonso https://bugs.launchpad.net/neutron/+bug/2027610 [sqlalchemy-20] Error in "test_notify_port_status_all_values" using an wrong OVO parameter Assigned to Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/888305. Merged RFEs ==== https://bugs.launchpad.net/neutron/+bug/2027742 [RFE] unmanaged dynamic router resources - OVN Heads up from the Ironic team: ======================== https://bugs.launchpad.net/neutron/+bug/2026757. dnsmasq on Ubuntu Jammy/Lunar crashes on neutron-dhcp-agent updates. Julia Kreger indicates that most likely this is not a neutron bug. However, she believes we might encounter it at some point. Yatin Karel is watching it. Bringing it up just for visibility to the Neutron team

1 0