- openstack-discuss - lists.openstack.org

[cinder][dev][ptl] NVMe/FC
by yian.zong＠dell.com 25 Apr '24

25 Apr '24

Hi cinder team, I'd like to know if cinder supports NVMe/FC. From nvmeof connector [1], it seems only RoCE(RDMA) and TCP. [1] https://opendev.org/openstack/os-brick/src/branch/master/os_brick/initiator… Thank you! Yian

5 5

[cinder][PTG] 2024.2 Dalmatian PTG summary
by Jon Bernard 25 Apr '24

25 Apr '24

Hello everyone, thanks for a very productive PTG for the Dalmatian cycle. Below is a brief summary of the highlights. The raw etherpad notes are here[1] and both the summary and transcribed etherpad notes are captured in our wiki here[2]. # Improve driver documentation When proposing new or updating existing drivers, our interface can be confusing. We agreed on an effort to migrate class documentation to the interface definitions, make clear the mandatory and optional methods for drivers, and maintain this documentation going forward. # User visible information in volume types Volume type metadata and extra specs are not visible to users, making it difficult to ascertain whether a volume type will lead to encryption or replication. Agreement was reached on metadata fields similar to Glance's metadefs, allowing drivers to report capabilities in a standard way and allowing that metadata to become visible to admins and users. # Optional backup driver dependencies Although volume driver dependencies are optional, those of our backup drivers are listed in requirements and are therefore always installed irrespective of deployment configuration. Agreement was reached to move these dependencies to driver-requirements. This should simplify efforts of both deployers and packagers. # Simplify deployments A few ideas were proposed to improve our deployment: * When cinder-volume is deployed in an active-active configuration, our `backend_host` parameter must be updated on every backend to have the same value, we would like to remove this additional step. * Use of predictable names to define tenants, instead of unique IDs. * Have dedicated quote section in our configuration for quote-related options. # Response schema validation A spec to add response schema validation in addition to our existing request validation was proposed. With adequate coverage, clients in various languages could be autogenerated (in theory). No significant objects were raised, agreement to review related patches this cycle. # Documentation of Ceph auth requirements We do not provide comprehensive and easy-to-find documentation on exactly what authentication/permission expectations there are between different services, leading deployers troubleshoot themselves and not be able to rely on upstream best practices. Agreement to improve this situation in the current cycle. # Cinder backup improvements An accumulation of bug fixes and performance improvements have stalled in the review queue. We went through the major ones as a team to attempt to bring awareness and unblock the remaining review requirements. See WIKI notes for specific details. # Migrating backups between multiple backends There is desire to support multiple backup backends where new backups go to a new backend while backups in an old backend remain readable. We want to avoid needing to create a full backup in the new backend to support incremental snapshots (and the associated charge). A spec will be proposed for review and work towards this goal will proceed during this cycle. # Cross-project with Glance In a cross-project collaboration with the Glance team, an improved method for image migration was proposed. Agreement to introduce a new migration operation in Glance. Cinder context was provided and a consensus on path forward was reached. # Cross-project with Nova In a cross-project collaboration with both Nova and Glance, the topic of image encryption was discussed. Dan Smith from the Nova team provided input from the Nova side. Cinder and Nova expect LUKS formatted images, but Glance currently supports GPG encrypted images - requiring re-encryption prior to use. It was noted that LUKS encrypted images can be created without root permission and the Glance team is now looking to drop GPG support and consolidate around LUKS as our unified format. # Active-Active support for NetApp A NetApp engineer raised questions about adding active-active support to the driver. Questions were answered and that work should proceed in this cycle. # Performance of parallel clone operations For the clone operation in cinder we're using a single distributed lock to prevent the source volume from being deleted mid-operation. This causes multiple concurrent clone operations to block. Under the right conditions, this can cause a significant performance degradation. Multiple possible solutions were discussed (read-write locks) and a consensus to use the DB was reached. Some details are unclear, awaiting a spec before moving forward. # Volume encryption with user defined keys Cinder does not currently support encryption with key provided by the user. Users could both manage their own keys and data could be recovered even if keys are lost at the deployment. There are several technical challenges to support this. Several of these hurdles were raised, more thought and research is needed before we have a spec that could be reviewed. [1] https://etherpad.opendev.org/p/dalmatian-ptg-cinder [2] https://wiki.openstack.org/wiki/CinderDalmatianPTGSummary -- Jon

2 1

[ironic][neutron] Another networking path for Ironic?
by Julia Kreger 25 Apr '24

25 Apr '24

Greetings folks, During the Ironic PTG last week, Ironic had our busiest PTG session on the topic of networking, and what to do in order to move things forward. I think the best way to frame how we reached the discussion is to keep in mind that projects in our community have been built upon boundaries of delineation. For example, Nova for is virtualization. Ironic’s area has been bare metal machine deployment and lifecycle management. Neutron’s area has been networking. Ironic has always fit a little more awkwardly into this model due to the nature of bare metal – as evidenced by the networking related (networking-generic-switch, networking-baremetal) projects living under Ironic’s governance driven by the need to be able to toggle physical switch ports to ensure a secure provisioning process. Now it's getting even more awkward as the lines between a server and a switch start to blur. Ironic is one of the few OpenStack projects that has significant use cases outside of a fully integrated OpenStack cluster. Additionally, even in the fully integrated case, bare metal is often not the primary use case considered -- quite understandable since virtualized networking is not a trivial problem. This combination of two factors: physical networking being a secondary use case for Neutron, and operators wanting to get physical network automation even without a fully integrated OpenStack, leads to difficulties both on the development side and the operational side. This creates a situation where Ironic’s answer to just basic network automation of “use Neutron and an ML2 plugin” is really not tenable for many infrastructure operators. The underlying challenge is that infrastructure operators need something for just basic management actions like attaching or detaching hosts from physical networks which also works in a model of respecting separation of duties. This is further complicated with DPUs and the changing landscape DPUs are creating by forcing a much more complex interaction when it comes to just basic networking needs for supporting the deployment of a bare metal host. To attach a network to a host, it is no longer just toggle a switchport configuration, it is potentially toggle a switchport *and* toggle configuration on a DPU device. And in this weeks’ Ironic PTG session, “Future of networking”[0], discussion of DPUs helped us reach a tipping point for the discussion. The idea of a separate network interface (for Ironic to use) which was not neutron, reached a point of being a chorus of contributors. So to put it in different words, Ironic is considering implementing functionality for Switch and DPU device network configuration management in terms of device attachment to networks which would likely leverage the existing ML2 plugin model such that Ironic can trigger port attachment and detachment directly. We’re not talking about doing anything beyond attachment and detachment. This means aspects like IPAM, Security Groups, Routing, and so on and so forth are entirely off the table for us. Our answer for such will continue to be “use neutron.” The discussion thus far has focused on not replacing existing plugins, but to enable their use. To use the plugins today, Neutron and the involvement of other OpenStack services are required – which is not viable for approximately half of Ironic’s usage base. For example, a Metal3 user is only using Ironic as an embedded tool, and if their environment requires VXLAN networks to be terminated on a DPU then they would need to create/invent tooling to facilitate that, or we can solve that with Ironic and account for the overall physical machine lifecycle at the same time. We also see a path where we can better integrate with Neutron and help operator’s meet more restrictive security requirements by delineating part of the functionality into Ironic’s scope when leveraging upstream solutions. We realize there are many unknowns, and welcome Neutron contributors to collaborate[1] with us, as this could ultimately impact project scope. Thoughts? Thanks, Julia [0]: https://etherpad.opendev.org/p/ironic-ptg-april-2024#L609 [1]: https://review.opendev.org/c/openstack/ironic-specs/+/916126/

2 2

Instance stuck in BUILD status
by 西岭 25 Apr '24

25 Apr '24

Dear openstack trove module developers, I was having trouble creating a trove instance and kept showing build status. My create command is: trove create mysql_zh-11 1 --size 5 --nic net-id=5f161d13-0cfb-4707-9bd6-f23850d86fbf --databases dbtest --users user:password --datastore mysql --datastore_version mysql-5.6 After entering the trove list command, the final state is: | bdcb298c-86d3-42c5-b7af-1cc0c7f93904 | mysql_zh-11 | mysql | mysql-5.6 | BUILD | 1 | 5 | RegionOne | The image I used was downloaded from the openstack website: trove-train-mysql-ubuntu-xenial Enter the openstack server list command and it will display: 2bf580eb-3c93-4689-9b53-e76307beb241 | mysql_zh-11 | ACTIVE | external-net=192.168.0.56 | trove-train-mysql-ubuntu-xenial | mysql-mini | I don't know the username and password of the image, so I can't see the specific information inside the cloud host. In addition, when I used the tool to change the login username and password of the above image, and created the trove instance with the image, after waiting 3600 seconds, it showed: trove.taskmanager.models [-] Service status: ERROR, service error description: guestagent error: TroveError: Failed to get server 0908ee4e-9f1d-444a-9c69-c8b8cc9d8938 for instance 03c2a10c-87f1-4169-9cf7-13735ff8756c, error: Failed to validate token (HTTP 404) (Request-ID: req-9604fb4c-4aae-4351-8996-5ee4dfbd9ad3) This time, I was able to log in to the cloud host generated during the creation of trove. After entering, I found that the guest-agent service failed to start. The reasons were: sudo chown -R ubuntu:root /etc/trove/conf.d cannot be executed. I wonder if it is because the password of the image has been changed? Then, I used the same image to create the trove instance again, and during the instance creation process, I went to the cloud host and started the guest-agent service manually. The result was the same. What else did I miss when creating the instance? How to solve this problem? See attachment for details of trove log. Thank you very much for your help. ----------------------------------------------------- Xi Ling, from Qingdao, China April 25, 2024

1 0

Secure RBAC manager role
by Sławek Kapłoński 25 Apr '24

25 Apr '24

Hi, I started looking at the S-RBAC today and for the phase 3 [1] especially. My question is - do we have agreement how this MANAGER will look like? In the linked document there is only info that keystone's spec [2] will have to be changed but I'm not sure if this is final now and if we can/should start thinking and implementing policies for the MANAGER role or not yet. [1] https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rb… [2] https://review.opendev.org/c/openstack/keystone-specs/+/818603 -- Slawek Kaplonski Principal Software Engineer Red Hat

3 5

[Glance] Deprecate metadata_encryption_key
by Pranali Deore 25 Apr '24

25 Apr '24

Hello Everyone, Glance core team have discussed and decided during Dalmatian PTG[1] about depreciation of `metadata_encryption_key` config option. We decided to deprecate it in this cycle and remove it in `E` (2025.1) cycle[2]. Problems with `metadata_encryption_key` config option and its related functionality: 1. It was added quite a long time ago and even though as per the description it should encrypt the location metadata but it actually encrypts the location url. 2. It encrypts the location url only for image upload/import/download/show APIs and doesn't encrypt url on location APIs. 3. If it's enabled during upgrade, it will break the existing deployment since the existing image url is not encrypted. 4. It doesn't even work for location url encryption as expected since it does not encrypt the legacy images url on start up, download of that image fails with InvalidLocation error. If there are any objections/suggestions on the above plan kindly please let us know before the spec freeze date of glance which is July 05,2024. If there will be no response till 5th July, we will assume that it's a go forward signal and can merge the spec lite[2] for the same. [1]: https://etherpad.opendev.org/p/apr2024-ptg-glance#L339 [2]: https://review.opendev.org/c/openstack/glance-specs/+/916178 Thanks & Regards, Pranali Deore

2 1

[magnum] Policy doesn't allow certificate:get
by Satish Patel 24 Apr '24

24 Apr '24

Folks, I have recently deployed openstack 2023.1 using kolla-ansible and after setting up magnum I noticed the following policy error when obtaining cards. After reading [1] the policy file looks like it needs a reader role to obtain a certificate. I have assigned the "reader" role to the user but still getting the same error message and no luck. $ openstack role add --user user1 --user-domain mydomain1 --project myproject1 reader # Reload User Creds RC file. $ openstack coe cluster config dev2 Policy doesn't allow certificate:get to be performed (HTTP 403) (Request-ID: req-7445ef3c-52a3-4911-97f6-1fb25d9fac1f) What else could be wrong here? 1. https://docs.openstack.org/magnum/latest/configuration/sample-policy.html

2 5

[tc][ptl][dpl] Policy to reset the DPL model every cycle
by Ghanshyam Mann 24 Apr '24

24 Apr '24

Hello Everyone. In vPTG, TC discussed improving to monitor the DPL model & its liaison's activeness. As per current policy[1], we rely on the liaisons to make sure they update the governance if they are not active in assigned roles or TC members to regularly check the DPL projects. Due to various reasons, this did not help and we saw many projects or liaisons going inactive and are detected very late. To improve the DPL model monitoring, I proposed resetting the DPL model to the PTL model every cycle and project teams need to opt-in DPL model every cycle. I would like to highlight this change to the projects currently on the DPL model or planning to adopt the same in future. Feel free to review and provide your feedback on the gerrit change mentioned below: - https://review.opendev.org/c/openstack/governance/+/916833 [1] https://governance.openstack.org/tc/resolutions/20200803-distributed-projec… -gmann

1 0

[largescale-sig] Next meeting: March 27, 9utc
by Thierry Carrez 24 Apr '24

24 Apr '24

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 9UTC. Arnaud Morin will be chairing. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20240327T09 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

2 3

openstack python3-keystone package
by Sari Zidan 24 Apr '24

24 Apr '24

Dear OpenStack Support Team, I'm a DEVOPS engineer who works for Zadara LTD. I hope this email finds you well. I am reaching out to inquire about the python-keystone package provided by OpenStack. Currently, our company utilizes Ubuntu Bionic (18.04.6) on our machines, and we have the python-keystone package installed and in use. However, we are now looking to upgrade to Python 3 and, consequently, need to transition from python-keystone to python3-keystone. In our codebase, we've integrated interfaces and APIs that rely on functions from the python-keystone Python library. However, as we transition to python3-keystone, I've identified several deprecated functions and modules which are no longer supported within the new package. To address this, I'm reaching out for assistance in finding suitable replacements within the new package. The deprecated functions/modules that require replacement (no longer supported by the new package) are: 1. keystone.common.dependency.resolve_future_dependencies() 2. keystone.auth.controllers.Auth.validate_token() 3. keystone.contrib.ec2.controllers.Ec2ControllerV3 - specifically: create_credential(), delete_credential() and get_credential() 4. keystone.resource.controllers.ProjectV3 - specifically: get_project, update_project, delete_project, create_project. 5. keystone.assignment.controllers 6. keystone.token.persistence.PersistenceManager Any guidance or assistance you can provide on this matter would be greatly appreciated. Thank you for your attention to this inquiry, and I look forward to hearing from you soon.

1 0

RDO Caracal 2024.1 Released
by Amy Marrich 24 Apr '24

24 Apr '24

RDO Caracal 2024.1 Released The RDO community is pleased to announce the general availability of the RDO build for OpenStack 2024.1 Caracal for RPM-based distributions, CentOS Stream and Red Hat Enterprise Linux. RDO is suitable for building private, public, and hybrid clouds. Caracal is the 29th release from the OpenStack project, which is the work of more than 1,000 contributors from around the world. The release is already available for CentOS Stream 9 on the CentOS mirror network in: http://mirror.stream.centos.org/SIGs/9-stream/cloud/x86_64/openstack-caraca… The RDO community project curates, packages, builds, tests and maintains a complete OpenStack component set for RHEL and CentOS Stream and is a member of the CentOS Cloud SIG. The Cloud SIG focuses on delivering a great user experience for CentOS users looking to build and maintain their own on-premise, public or hybrid clouds. All work on RDO and on the downstream release, Red Hat OpenStack Platform, is 100% open source, with all code changes going upstream first. The highlights of the broader upstream OpenStack project may be read via https://releases.openstack.org/caracal/highlights.html but here are some highlights: - Drivers with inactive CI were marked unsupported including Windows iSCSI Driver, Windows SMB Driver, Dell SC Series Storage Driver (iSCSI, FC), Dell VNX Storage Driver (FC, iSCSI) and Dell XtremeIO Storage Driver (iSCSI, FC). - New driver features were added, notably, Fujitsu ETERNUS DX extend volume on RAID group, Pure Storage synchronous replication, NetApp iSCSI LUN space allocation, Dell PowerFlex Active-Active support, Dell PowerMax configurable SRDF snapshots. - Designate now supports Catalog Zones (RFC 9432). This can improve the scalability of Designate pools managing a large number of zones and significantly reduce the provisioning time when adding additional DNS servers to a Designate pool. - Horizon now uses Django 4.2 as default and dropped Django 3.2 support. - Ironic has enabled RBAC support by default by changing the default values of [oslo_policy]enforce_scope and [oslo_policy]enforce_new_defaults to True. Additionally, we added [DEFAULT]rbac_service_project_name to define a project where users in that project are treated as having a service role. Please see Ironic release notes for full details. - Ironic has added the ability to drain active tasks from a conductor before shutdown. Sending a SIGUSR2 signal to an ironic-conductor will now attempt to complete running tasks with a timeout of [DEFAULT]drain_shutdown_timeout. No new tasks will be started on the conductor while it’s draining. - Support was added for the external-gateway-multihoming API extension. The L3 service plugins supporting it can now create multiple gateway ports per router. It is currently limited to the L3 OVN plugin. OpenStack Caracal is marked as Skip Level Upgrade Release Process or SLURP. According to this model ( https://governance.openstack.org/tc/resolutions/20220210-release-cadence-ad…) this means that upgrades will only be supported from the Antelope 2023.1 release. RDO Caracal 2024.1 has been built and tested with the recently released Ceph 18.2.0 Reef version (https://docs.ceph.com/en/latest/releases/reef/) which has been published by the CentOS Storage SIG in the official CentOS repositories. *Note:* Follow the instructions in [RDO documentation]( https://www.rdoproject.org/install/install-with-ceph/) to install OpenStack and Ceph services in the same host. During Caracal cycle, some projects have been retired or declared inactive upstream. As such, the following packages for some projects are not present in the RDO Caracal 2024.1 release: - cinderlib - dib-utils - ec2api-tempest-plugin - ec2api - murano - murano-agent - muranoclient - murano-dashboard - murano-tempest-plugin - sahara - sahara-dashboard - sahara-image-elements - sahara-plugin-ambari - sahara-plugin-cdh - sahara-plugin-mapr - sahara-plugin-spark - sahara-plugin-storm - sahara-plugin-vanilla - sahara-tests - senlin - senlinclient - puppet-ec2api - puppet-etcd - puppet-haproxy - puppet-module-data - puppet-murano - puppet-qdr - puppet-rsyslog - puppet-sahara During the next release we will continue working on retiring inactive packages in order to ensure RDO content quality and security. Contributors: - During the Caracal cycle, we saw the following new RDO contributors: - Marihan Girgis - Balazs Gibizer - Fiorella Yanac Welcome to all of you and Thank You So Much for participating! But we wouldn’t want to overlook anyone. A super massive Thank You to all 47 contributors who participated in producing this release. This list includes commits to rdo-packages, rdo-infra, and rdo-website repositories: - Ade Lee - Alfredo Moralejo Alonso - Amy Marrich (spotz) - Arx Cruz - Balazs Gibizer - Bernard Cafarelli - Bohdan Dobrelia - Chandan Kumar - Cyril Roelandt - Daniel Pawlik - Douglas Viroel - Emma Foley - Fabricio Aguiar - Fiorella Yanac - Gregory Thiemonge - Grzegorz Grasza - Harald Jensås - Jiří Podivín - Joan Francesc Gilabert - Joel Capitao - Karolina Kula - Lewis Denny - Lon Hohberger - Luigi Toscano - Luis Tomas Bolivar - Lukáš Piwowarski - Maor Blaustein - Marihan Girgis - Marios Andreou - Martin Kopec - Martin Magr - Mathieu Bultel - Michael Johnson - Nate Johnston - Pablo Rodríguez Nava - Priscila Gutierres - Rabi Mishra - Roberto Alfieri - Ronelle Landy - Shreshtha Joshi - Soniya Vyas - Szymon Datko - Takashi Kajinami - Tobias Urdin - Tristan De Cacqueray - Yatin Karel - Zane Bitter The Next Release Cycle At the end of one release, focus shifts immediately to the next release i.e Dalmatian. Get Started To spin up a proof of concept cloud, quickly, and on limited hardware, try an All-In-One Packstack installation. You can run RDO on a single node to get a feel for how it works. For those that do not have any hardware or physical resources, there is the OpenStack Global Passport Program. This is a collaborative effort between OpenStack public cloud providers to let you experience the freedom, performance and interoperability of open source infrastructure. You can quickly and easily gain access to OpenStack infrastructure via trial programs from participating OpenStack public cloud providers around the world. Get Help The RDO Project has our users(a)lists.rdoproject.org for RDO-specific users and operators. For more developer-oriented content we recommend joining the dev(a)lists.rdoproject.org mailing list. Remember to post a brief introduction about yourself and your RDO story. The mailing lists archives are all available at https://mail.rdoproject.org. You can also find extensive documentation on RDOproject.org. The #rdo channel on OFTC IRC is also an excellent place to find and give help. We also welcome comments and requests on the CentOS devel mailing list and the CentOS IRC channels (#centos, #centos-cloud, #centos-devel in Libera.Chat network), however we have a more focused audience within the RDO venues. Get Involved To get involved in the OpenStack RPM packaging effort, check out the RDO contribute pages, peruse the CentOS Cloud SIG page, and inhale the RDO packaging documentation. Join us in #rdo and on the OFTC IRC network and follow us on Twitter @RDOCommunity. You can also find us on Facebook and YouTube.

1 0

[horizon] Weekly meeting cancellation
by Tatiana Ovchinnikova 24 Apr '24

24 Apr '24

Hello team! Apologies for the short notice. I am not feeling well, so I have to cancel today's meeting. See you next week! Tatiana

1 0

[kolla-ansible] redeploy service to another host
by Andrej Moravčík 24 Apr '24

24 Apr '24

Hi. What is the correct procedure for redeploying (generic) service to another host group in Kolla-ansible multinode environment? I did not find such information in documentation. I have mistakenly deployed Opensearch to [control] host group instead of dedicated monitoring host. First, I tried to “rollback” deployment by disabling Opensearch in globals.yml (enable_central_logging: “no”) then performed "kolla-ansible -i multinode.yml reconfigure” but Docker containers “opensearch” and "opensearch-dashboards” are still running on all control nodes. Any advice would be greatly appreciated. Thank you. Regards — Andrej Moravčík ________________________________ Táto správa a jej prílohy sú dôverné. Prosím, narábajte s týmito informáciami ako s dôvernými a nezverejňujte, nekopírujte ani neposielajte tieto informácie bez predchádzajúceho súhlasu spoločnosti Aliter Technologies, a.s. Ak nie ste adresátom indikovaným v záhlaví tejto správy alebo želaný príjemca, upovedomte prosím spoločnosť Aliter Technologies, a.s. odpoveďou na tento e-mail a pôvodnú správu vymažte zo svojho systému. Aliter Technologies - registrovaná ochranná známka pre EU, USA This message and any attachments are confidential. Please treat the information as confidential, and do not disclose, copy or deliver this message to anyone without Aliter Technologies' approval. If you are not the addressee indicated in this message or an intended recipient please notify Aliter Technologies a.s. by return e-mail and delete the message from your system. Aliter Technologies - Registered in WIPO & U.S. Patent and Trademark Office

4 4

[Kolla-ansible] is there a way to swicth from docker to podman
by wodel youchi 24 Apr '24

24 Apr '24

Hi, I am preparing to upgrade a platform from yoga to the latest version of kolla-ansible 2023.2 I am wondering if there is a way to switch from docker to podman while upgrading? Regards. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free.www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2 1

[ptg][openstack-ansible] OpenStack-Ansible PTG results for Dalmatian
by Dmitriy Rabotyagov 24 Apr '24

24 Apr '24

Hi everyone! Sorry for the late PTG report. But better late than never I guess :) This time we did not have a lot of discussion points, and mainly re-scoped not delivered topics from previous PTG, including: * Improvements to PKI role for integration with Hashi Vault and ability to handle encrypted keys with ansible-vault * ProxySQL implementation - it's long overdue, actually, but still seems beneficial * Incus support * Usage of deb822_repository module instead of apt_repository. This is going to be breaking change as does not assume backwards compatibility in a way of defining repositories when they're overridden (especially for deployments with limited connectivity) * Improving our documentation state. We are still lagging behind adopting all our docs to OVN driver being a default. We also need to document a way of defining pretty endpoint names (serving all endpoints from port 80/443 rather then unique ports per service). With that being said, we also reviewed what is feasible to finish for Caracal release. We still need to get some progress on following topics in upcoming weeks: * ovn-bgp-agent support: https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/909780 ** We have acquired the FRRouting role from Vexxhost (thanks for that!), but some changes to it are needed, ie EL9 support: https://review.opendev.org/q/project:openstack/ansible-role-frrouting+statu… * OVN driver support for Octavia: https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/868462 ** I need to look at testing of that driver and add corresponsive job to the role * Dealing with inactive projects and prevent new deployments during Caracal release: sahara, murano, senlin * Finalize Skyline support: https://review.opendev.org/q/topic:%22osa/skyline%22 * Enable Quorum Queues by default along with other oslo.messaging improvements: https://review.opendev.org/q/topic:%22osa/messaging_improvements%22 So thanks everyone for participation and let's keep the ball rolling!

2 1

[magnum] k8s cluster deploy stuck timeout
by Jaime Ibar 24 Apr '24

24 Apr '24

Hi all, I have installed magnum with no issues. After the installation, I tried to deploy a kubernetes cluster but after some time, the creation process times out and this always happens during the kube_cluster_deploy stage. I'm running 17.0.0 (Bobcat) so I tried with core os fedora-coreos-38.20230806.3.0 and kube_tag v1.26.8-rancher1 as stated in the documentation but no joy. I also tried different combinations of fedora core and kube_tag but same, no joy. I can ssh into the vms, podman exec into the containers but if I run kubectl version I get the following error The connection to the server localhost:8080 was refused - did you specify the right host or port? and tailing /var/log/heat-config/heat-config-script/ I get the following message ++ kubectl get --raw=/healthz The connection to the server localhost:8080 was refused - did you specify the right host or port? Any idea what might be the problem? TIA Jaime -- salu2 Jaime

4 5

[magnum] Weekly meeting cancelled
by Jake Yip 24 Apr '24

24 Apr '24

Hi all, I'll need to cancel this week's meeting as I am unwell. - Jake

1 0

[nova][ptg] 2024.2 Dalmatian PTG summary
by Sylvain Bauza 23 Apr '24

23 Apr '24

Hey folks, Sorry for being a bit late to provide this summary, but let me do it :-) So, we had in between 10 to 20 contributors during our 4 days. Wow, it was nice ! Thanks all of you that were there, and I was happy to see you even if it was the first time you were in. As a reminder, please don't open the main etherpad link [1] if you use a Web browser with an automatic translation modification feature. If you do it, then it would internally translate all the phrases directly in the etherpad as if you were directly modifying the etherpad by yourself. Please also make sure you don't accidentally remove or modify lines. In order to prevent any misusage, please rather look at this read-only etherpad link https://etherpad.opendev.org/p/r.3d37f484b24bb0415983f345582508f7 Now, let me explain the summary now (first, take a coffee if you want). First, we only had one cross-project meeting, with the Neutron team. I won't provide the summary for this as it was only a neutron discussion (this was about multiple port bindings support with only some neutron backends). Please rather look at the neutron etherpad or wait for its summary. ### Caracal retrospective and Dalmatian planning ### 19 blueprints were approved, 10 of them landed. We also had more contributors (38) than last cycle. We now unfortunately have >60 untriaged bugs so we discussed that. We agreed then that rather than triaging the bugs every week, we should propose a bug triage day on a periodic time and welcome any person who would want to help about it. We will also ask reporters for each bug report that is older than 5 years if they can try to reproduce the issue with a newer/supported release. We also discussed how to make sure we could review the feature implementations. Eventually, we agreed on the fact we should ask spec owners to discuss with us for having feature liaisons for their specs so we could have something like 'subteams' per feature implementation. For that, I'll explain it separately by a new email in the next few weeks. About the proposed Dalmatian planning, we'll have two spec review days (R-20 and R-13) but also a spec approval freeze day (a deadline, if your prefer) by R-11 (mid-July). We'll also have an implementation review day by R-10. ### Enhancing support for running nova services in k8s ### Gibi agreed on providing a backlog spec for explaining what we should modify in Nova. For graceful shutdown support, gibi will also create another spec to make oslo.messaging only supporting some RPC topics usage. ### Unified limits default behaviour with no registered limits ### We discussed some specific case you have today with unified limits : you need to register all the limits for all the resource classes you use for flavors, because if not, you wouldn't have quotas for them. We then agreed on the fact that all the 'core' resource classes (meaning the non-custom ones) should be registered. melwitt will provide a document for explaining it. For custom resource classes, we agreed on providing a relax mode (in the nova configuration) for accepting unregistered limits only for custom resource classes. That option would also have a strict mode for asking to register even the custom RCs. ### Missing instance action events ### We have a bug report about missing instance action events in our API [2]. We agreed on fixing this without requiring a new API microversion. For each of the missing events, ratailor will provide individual bugfixes and will also try to assert the events in Tempest scenarios. ### Ephemeral encryption feature ### As we started to merge some changes for this feature on Caracal, we also found some concerns that needed to be discussed during this PTG. Melwitt will provide a new spec for adding a new API microversion that will add new request parameters for snapshotting an instance. If an instance that has an encrypted disk needs to be rescued, either the rescue disk will also be encrypted or it will use the same key as the rescue image. Resize won't yet be accepted but melwitt will try to look how it could be unencrypted. Eventually, we also discussed the fact that compute services should check when moving an instance whether the target supports the encryption options. ### Nova Metadata optimisation ### In case the metadata API is called quite frequently, there is some read amplification. In order to fix this, fwiesel will create a blueprint for lazy loading some returned parameters and telling which metadata paths are the most called. For the moment, we won't do any cache invalidation feature as we first need to understand whether the lazy-loading helps. ### Extending memory encryption support ### For the moment, Nova only supports AMD-SEV for encrypting the instance memory. We discussed about two other hardware supports : AMD-SEV-ES and SGX. For AMD SEV-ES (new AMD SEV generation), we agreed on documenting some not configuration support by BIOS, modifying the inventories by using nested resource providers with the same resource class and maybe needing some reshape. Takashi will work on it for this cycle (live-migration wouldn't be done by this cycle) For SGX, we said we need a new blueprint and a spec. We are OK to create a new nested resource provider for the resource class, but we would like to know the current move operation limitations. ### Stateless firmware support ### Even if the libvirt driver recreates a new firmware when we're moving/resizing an instance, we discussed whether we should support a specific stateless firmware. We agreed on accepting it, so takashi will work on it, but will also fix the related bug reports [3]. ### Automatically detecting vTPM support ### Long story short, we accepted [4] as a specless blueprint. takashi will work on the implementation this cycle. ### vTPM migrations ### Even if Nova supports vTPM related instances, we don't support yet rebuild, rescue, shelve, live migration and evacuation. We agreed on fixing this cycle live-migration/evacuate as a priority by persisting the vTPM memory somewhere. We need to find a contributor that said, hopefully those weeks. A bit related, we discussed some live-migration other topic that was related to our SSH usage later, and we said in that topic that some kind of small object-store in nova (or elsewhere) could be useful for transferring instance-related local datablocks (like console logs, NVRAM or config drives), including vTPM memory state. ### Live migration options difference ### So, you probably know that we have a lot of live migration configuration options now, as least the fact tht the live_migration_uri option was deprecated in favor of live_migration_scheme and live_migration_inbound_addr. Unfortunately, those new options miss some way to provide some elements like user, port, path and query string. Eventually, we agreed on deprecating this cycle live_migration_scheme while *un*deprecating live_migration_uri. That won't mean that we'll remove live_migration_scheme quickly, at least until 2025.2 F (based on the SLURP policy) so distros shouldn't be afraid. ### Allow Userdata to be updated ### Well, that one is a long overdue feature request. crohmann kindly updated the spec [5] for this cycle based on our feedback from the last reviews, we agreed on reviewing that spec but a quick glance showed up it was going into the right direction. ### Clean up old compute service versions ### That one has been a long discussed topic, even if just tech debt related. The case is that we have old cruft of upgrade checks in our code that we could remove, but we were afraid of removing some unsupported yet useful 'upgrade enveloppe' between very old computes and new controllers. After lots of back and forths, we eventually agreed on having some kind of upcall check from compute to conductor that would signal this compute that it's too old and needs to self-disable, yet having some 'escape valve' (straighly quoting) in case it incorrectly disables things. Dansmith agreed to start scheming it. ### OpenAPI-compatible API reference documentation ### Stephenfin instructed us on the cross-project effort aiming to have project's api-refs automatically generated thru some tooling. In order to achieve that, there are a few gaps in our current code that need to be fixed, mostly about adding some response schema validation. stephenfin already proposed a spec [6]. We agreed on reviewing it. We also wanted the implementation to be iterative per API resource, to leave to the deprecated APIs untouched and to keep the existing api-ref until the new documentation is on par. Stephefin agreed to update his spec based on that feedback. ### SPICE-direct console support ### A new nova spec [7] was created this cycle by mikal in order to provide some hypervisor details for some external tool that could then pass a SPICE-direct console from a specific instance to some user. We discussed this feature and we said that the usecase was reasonable. That said, we had some concerns about the technical modifications but we'll discuss them in the spec review. ### Remote consoles API ### When an user asks Nova to create a new remote console for an instance, the user doesn't tell which kind of console they want to get, only the operator can set which console kind to use per compute. We agreed on providing a new API microversion for helping users which kind of console they want (html5 or tcp-direct for example) by a new request parameter. Fwiesel accepted to create a new spec for it this cycle. ### Resurrected computes should doublecheck the RPC messages they get ### Sorry for the terrible section name above, I tried to make it short. Basically, when a compute service disappears for some time and then returns, it can process any RPC request which was in queue. If some instance was evacuated in between, then this compute will dumbly try to process what was called earlier (like a move operation). We agreed on the fact this is a bug that needs to be fixed by adding some conditional against any move operation RPC targeted method, we don't know yet who could work on it, maybe tobias-urbin since he reported it. ### How can we get rid of eventlet in Nova ? #### I won't go into the details on what eventlet is but please consider that all nova services are hardly dependent on that python library for concurrency reasons, mostly the compute service (besides the API and other services for the workers usage). Sean Mooney proposed us a plan for using direct python threading (instead of asyncio) in order to replace our workers model and to create specific distinct threadpools for compute-related operations. We agreed on that approach so we'll let sean to create a blueprint (which will be approved as specless) and rebase his current series so we could review it. ### EOF ### That's a wrap now, my fingers are now quite bloody (they're not used to write that much). Sorry for the long summary, I just hope your coffee (or tea) was good. -Sylvain (who could have written his last PTG summary email if planets align) [1] https://etherpad.opendev.org/p/nova-dalmatian-pt g (please remove the empty char between 'pt' and 'g') [2] https://bugs.launchpad.net/nova/+bug/2058928 [3] https://bugs.launchpad.net/nova/+bug/1785123 and https://bugs.launchpad.net/nova/+bug/1633447 [4] https://blueprints.launchpad.net/nova/+spec/libvirt-detect-vtpm-support [5] https://review.opendev.org/c/openstack/nova-specs/+/863884 [6] https://review.opendev.org/c/openstack/nova-specs/+/909448 [7] https://review.opendev.org/c/openstack/nova-specs/+/915190

5 5

[kolla] opensearch build issue
by Andrej Moravčík 23 Apr '24

23 Apr '24

Hello, I am having trouble building opensearch image based on Ubuntu distro using Kolla 16.3.1 / OpenStack 2023.1. Has anyone run into the same problem? Any help appreciated. INFO:kolla.common.utils.opensearch:Preparing to unpack .../opensearch_2.13.0_amd64.deb ... INFO:kolla.common.utils.opensearch:Running OpenSearch Pre-Installation Script INFO:kolla.common.utils.opensearch:Unpacking opensearch (2.13.0) ... INFO:kolla.common.utils.opensearch:Setting up opensearch (2.13.0) ... INFO:kolla.common.utils.opensearch:Running OpenSearch Post-Installation Script INFO:kolla.common.utils.opensearch:ERROR: Something went wrong during demo configuration installation. Please see the logs in /var/log/opensearch/install_demo_configuration.log INFO:kolla.common.utils.opensearch:dpkg: error processing package opensearch (--configure): INFO:kolla.common.utils.opensearch: installed opensearch package post-installation script subprocess returned error exit status 1 INFO:kolla.common.utils.opensearch:Processing triggers for libc-bin (2.35-0ubuntu3.6) ... INFO:kolla.common.utils.opensearch:Errors were encountered while processing: INFO:kolla.common.utils.opensearch: opensearch INFO:kolla.common.utils.opensearch:E INFO:kolla.common.utils.opensearch:: INFO:kolla.common.utils.opensearch:Sub-process /usr/bin/dpkg returned an error code (1) INFO:kolla.common.utils.opensearch: INFO:kolla.common.utils.opensearch: INFO:kolla.common.utils.opensearch:Removing intermediate container 72435c692eff ERROR:kolla.common.utils.opensearch:Error'd with the following message ERROR:kolla.common.utils.opensearch:The command '/bin/sh -c apt-get --error-on=any update && apt-get -y install --no-install-recommends opensearch && apt-get clean && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100 Regards — Andrej Moravčík ________________________________ Táto správa a jej prílohy sú dôverné. Prosím, narábajte s týmito informáciami ako s dôvernými a nezverejňujte, nekopírujte ani neposielajte tieto informácie bez predchádzajúceho súhlasu spoločnosti Aliter Technologies, a.s. Ak nie ste adresátom indikovaným v záhlaví tejto správy alebo želaný príjemca, upovedomte prosím spoločnosť Aliter Technologies, a.s. odpoveďou na tento e-mail a pôvodnú správu vymažte zo svojho systému. Aliter Technologies - registrovaná ochranná známka pre EU, USA This message and any attachments are confidential. Please treat the information as confidential, and do not disclose, copy or deliver this message to anyone without Aliter Technologies' approval. If you are not the addressee indicated in this message or an intended recipient please notify Aliter Technologies a.s. by return e-mail and delete the message from your system. Aliter Technologies - Registered in WIPO & U.S. Patent and Trademark Office

2 4

[tc] How to indicate that a project will transition straight to EOL
by Stephen Finucane 23 Apr '24

23 Apr '24

o/ I've noticed that a variety of SDK-related projects have had 'Update .gitreview for unmaintained/wallaby' patches proposed against them, e.g [1]. I'm not sure this makes much sense. Both SDK and OSC are designed to work against any version of OpenStack so there's hardly a reason to keep stable branches around, let alone the unmaintained branches. I took a look through the Project Team Guide docs [2] but while I can find info on opting-in to keep older unmaintained branches around, I can't find any info on opting-out of their creation altogether. That brings me to my question: does such a thing exist, and if not, can it? How can we ensure we go straight from stable to EOL for these projects? As a separate point, we may wish to explore removing stable branches entirely for these projects. Open to input on whether this is even possible, or if their removal would have serious issues for folks. Cheers, Stephen [1] https://review.opendev.org/c/openstack/openstacksdk/+/911660 [2] https://docs.openstack.org/project-team-guide/stable-branches.html#unmainta…

5 7

[Neutron] [VPNaaS] VMs unable to hit tunnel remote cidr
by Karl Kloppenborg 23 Apr '24

23 Apr '24

Hi Neutron/VPNaaS teams. We run an openstack which neutron runs on the following VPNaaS setup: service_provider: VPN:strongswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default L3-agent vpn_device_driver: neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver With Neutron running the following service plugins: service_plugins: neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin,neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,router,vpnaas,trunk,segments,bgp And Ml2 config Openvswitch_agent -> Security Group -> firewall driver: openvswitch We have an IPSec tunnel up between a remote site (Fortinet device) and us. Computers within the remote site can reach the VM’s internal IP’s on Openstack across the tunnel, however VM’s from openstack cannot reach the internal network on the remote side. A traceroute from the VM to a known IP on the remote side shows the IP instead transiting out the router gateway. Has anyone seen this before? It looks like something isn’t being picked up/a route not being set. Any assistance would be greatly appreciated. Thanks, Karl.

3 4

[nova][kolla-ansible] having issue running nova API bootstrap container
by James Leong 23 Apr '24

23 Apr '24

Hi all, I am currently trying to deploy a yoga version of OpenStack, However, I am having the following issues when doing a deployment using kolla-ansible. The error occurs on task "[nova : Running Nova API bootstrap container]." "os_traits.COMPUTE_ADDRESS_SPACE_PASSTHROUGH,", "AttributeError: module 'os_traits' has no attribute 'COMPUTE_ADDRESS_SPACE_PASSTHROUGH' Best, James

2 2

[tc] Weekly TC meeting agenda for Apr 23rd 2024
by Goutham Pacha Ravi 22 Apr '24

22 Apr '24

Hello Stackers, We're four weeks [1] into the 2024.2 release of OpenStack and there's a lot of exciting governance proposals at the table already. Please join us to discuss these at the weekly meeting of the OpenStack Technical Committee tomorrow, Tuesday April 23rd, at 1800 UTC in the #openstack-tc channel on OFTC. If you haven't already done so, JayF wrote up a detailed summary of our recently concluded PTG for this list [2]. We'll be using this to drive some of the discussions tomorrow. Thank you, Jay! The agenda for tomorrow's meeting is: * Roll call * AIs from the TC vPTG 2024.2 #link https://etherpad.opendev.org/p/apr2024-ptg-os-tc (Technical Committee vPTG notes) * 2024.2 TC Tracker #link https://etherpad.opendev.org/p/tc-2024.2-tracker (Technical Committee activity tracker) * Ongoing business ** Clarify the timeline for project to move to Inactive state (https://review.opendev.org/c/openstack/governance/+/908880) ** Appoint Wenxiang Wu as PTL of Skyline (https://review.opendev.org/c/openstack/governance/+/915108) ** Transition Freezer project to DPL (https://review.opendev.org/c/openstack/governance/+/914911) ** Assign Dmitriy Rabotyagov as Freezer PTL (https://review.opendev.org/c/openstack/governance/+/915727) ** Move docs tools into TC governance (https://review.opendev.org/c/openstack/governance/+/914950) * Other governance proposals that will transition to "formal-vote" * 2025.1 Elections * TC Summary Email * Open Discussion and Reviews See you there! Thank you, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://releases.openstack.org/dalmatian/schedule.html [2] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

1 0

[neutron] Bug deputy report (week starting on April 15)
by Elvira Garcia Ruiz 22 Apr '24

22 Apr '24

Hi folks, This is the Neutron deputy report for the week of April 15 - 21. Thanks a lot to froyo for taking a look at a wave of ovn-octavia-provider bugs that appeared on our Launchpad during these last couple of hours! High ------ - https://bugs.launchpad.net/neutron/+bug/2062385 - [ovn-octavia-provider] Member with FIP not reachable Assigned to Fernando Royo - Might actually just be a known limitation of OVN. - https://bugs.launchpad.net/neutron/+bug/2061883 - [fwaas] Duplicate entry <uuid> for key default_firewall_groups.PRIMARY' Unassigned Medium ----------- - https://bugs.launchpad.net/neutron/+bug/2062536 - [ovn] agent_health_check does not work for ovn agents Assigned to Liu Xie. In progress. - https://bugs.launchpad.net/neutron/+bug/2062511 - [ovn] ovn metadata agent occasionally down if SB is snapshoting Fix proposed by Liu Xie https://review.opendev.org/c/openstack/neutron/+/916387 - https://bugs.launchpad.net/neutron/+bug/2062009 - Neutron-server + uwsgi deadlocks whenr unning rpc workers Fix proposed by sebageek https://review.opendev.org/c/openstack/neutron/+/916123 Incomplete --------------- - https://bugs.launchpad.net/neutron/+bug/2061350 - [ovn] The 'ovn_client' of the L3 plugin is occasionally NoneType Undecided -------------- These have been recently moved to neutron launchpad from the octavia one because they belong to the ovn-octavia-provider: - https://bugs.launchpad.net/neutron/+bug/2062965 - octavia/ovn: filedescriptor out of range in select() Unassigned - https://bugs.launchpad.net/neutron/+bug/2063043 - octavia/ovn: missed healthmon port cleanup Unassigned And this was just filed about an hour ago, also ovn-octavia-provider related: - https://bugs.launchpad.net/neutron/+bug/2063081 - [ovn] connectivity to load balancer with FIP from a server connected to a router with external gateway Unassigned Currently Fernando Royo is taking a look at the three of them for further triage. Best regards, Elvira

1 0

[tc] Congratulations, Goutham Pacha Ravi is new TC Chair
by Jay Faulkner 20 Apr '24

20 Apr '24

Hi all, I'd like to congratulate Goutham Pacha Ravi on becoming the TC Chair for 2024.2 cycle [1]. Thanks for the opportunity to serve as chair for the last six months; please treat Goutham kindly during his time in the role as you did me. Thanks, Jay Faulkner OpenStack TC 1: https://review.opendev.org/c/openstack/governance/+/916024

9 8