April 2024 - openstack-discuss - lists.openstack.org

[ironic][PTG] Ironic 2024.2 PTG summary
by Riccardo Pittau 16 Apr '24

16 Apr '24

Hello Ironicers! Here's a summary of the discussions around the major topics at the Ironic PTG 2024.2 # Caracal retrospective # We took our time to look back at the Caracal development cycle, how it was for the ironic project, celebrate what we've accomplished, and of course also understand if some of the work started during Caracal needed to be continued in Dalmatian. We're quite happy to say that most of the goals we've set for Caracal have been reached, and we've actually been able to do much more than planned! For example we delivered the HttpBoot support for Redfish, we've been able to tackle a lot of old bugs and keep pace with the migration from storyboard to launchpad, almost finished the merge of Inspector into Ironic, and marked multiple old drivers for removal. # Present and Future of Ironic # We've analyzed the current status of Ironic and asked ourselves: "Where do we want Ironic to go?" In the past years we started looking beyond Openstack and we've took some steps to get into that direction, and we can say that Ironic went over the Openstack borders, becoming more independent. Currently Ironic can be used in three ways: - Openstack Integrated, the more clear and established way - as the core engine of Metal3, with good documentation for various use cases, but still growing and with margin for improvement - standalone, which still requires some work, a much clearer path forward and requirements We really want to keep investing in standalone Ironic more in the future, but to do that we need to take some decisions based on the vision we have on server provisioning from here to 5 years. Lots of questions we asked ourselves on how to resolve some problems connected to having a standalone bare metal provisioning service, mostly related to networking (e.g. secureboot, virtualmedia), will expand and find possible answers in other topics. # Redfish topics # Talking about present and future, Redfish is the current constantly evolving and improving standard protocol to directly communicate with controllers of bare metal devices. Ironic supports it since a while and of course there's more and more demand for new features to support or to expand the current capabilities. The discussion was mainly around an alternative, and possibly easier way to provide images for firmware updates and virtual media boot using a "push" model instead of the current "pull", with two proposals: - using the UpdateService already present in the Redfish standard - using the Virtual Media Push as a new feature currently under discussion with DMTF Another major topic is the support for TLS connection to the BMC, which already has a design proposal from one major contributor, Fujitsu. In addition we'll also have to explore the "other side" of the connection, so from the BMC to the ironic services, which will be one of the goals for Dalmatian. # CI related topics# CI has always had great importance for the Ironic project being one of the major consumers in terms of resources. It's not a case that that first related topic was how to slim down the CI usage! During the Caracal cycle we've been able to reduce the number of jobs run by the Ironic project, but we've also added some more. We came to the conclusion that we need to take an approach where we stack more tests in fewer jobs, trying to consolidate down jobs as much as possible and minimize boot interface variation testing. During the Dalmatian cycle we'll work first on trying to have Redfish and ipmi in a single ironic job, and update the list of jobs to understand where we can avoid duplication. At some point we'll also have to make space for some more deep tests on ARM machines. We've also decided to give Codespell tests a try, and explore the possibility of having some pre-commit hooks, and, after some technical considerations, move away from Tinycore as our base for the IPA ramdisk in CI, gentoo being the most likely candidate as a replacement. # xPUs # With xPUs in this case we mean IPU and DPU (not to be confused with TPU or GPU) which are some Processing Units that are designed for specific applications and data types. Usually DPUs are embedded in edge network cards, while IPUs are specifically designed for Artificial Intelligence/Machine Learning. There are multiple challenges in order to support those devices, depending on the "vision" of the vendors, some view them having full Operating Systems, some as having an OS just as "firmware", most or all of them would like remote management, which is where Ironic can really make an impact. Even though there are multiple use cases and scenarios where the use of xPUs is highly valued, and the wishlist of potential features is quite long, the first step we need to take is to get some real hardware and start testing, which is the real challenge we'll have to deal with next. # Secure Boot # We've made huge steps forward in supporting Secure Boot, the next big one would be supporting custom keys and providing means to upload and update them. Before doing that we need to clarify some aspects of Secure Boot, especially the "security vs convenience". In the Dalmatian cycle will focus mainly on updating our documentation, provide a better picture to users, and start exploring possible solutions for the secure custom keys management. # Future of Networking # Reconnecting to the future of Ironic, networking represents the next step for a truly standalone Ironic. This means finding alternatives to Openstack-integrated scenarios and therefore to Neutron. In Metal3 we're solving the problem using a separate IPAM, but this is still not enough for a complete Ironic standalone, in the end what we really need is flat network control, which at the moment we don't really have an alternative for. We realized that the topic is vast, we've just scratched the surface and we need to invest much more time on it, having more discussions to go deeper into it and start researching alternatives and tools for our goal. That is (almost) all folks! You can find all the topics and more details in the discussions notes here: https://etherpad.opendev.org/p/ironic-ptg-april-202 Feel free to answer directly in this thread in case you have questions or comments, or contact the ironic team on IRC. Thanks to everyone who participated in the PTG! Have a great Dalmatian development cycle! Ciao Riccardo

1 0

[publiccloud-sig] Reminder - next meeting April 17th - 0700 UTC
by Felix Kronlage-Dammers 16 Apr '24

16 Apr '24

Hi everyone, This Wednesday (April 17th) the next meeting of the public cloud sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG See you tomorrow! felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[all] ORM apocalypse mostly averted (a.k.a. SQLAlchemy 2.0 is here)
by Stephen Finucane 15 Apr '24

15 Apr '24

o/ A whole 2 years and 8 months to the day after I first communicated about it [1], it would appear the requirements bump to SQLAlchemy 2.0 [2] is finally ready to go. I suspect most if not all service projects are familiar with SQLAlchemy 2.0 at this point, having needed to fix whatever compatibility issues were seen. For those that aren't, I suggest reading my previous emails on the matter. So what happens next? Hopefully for the vast majority of projects the answer should be "nothing": many projects now have sqlalchemy-tips jobs that are designed to run unit and/or integration tests with the latest versions of SQLAlchemy, Alembic and oslo.db so the switchover should be seamless. I foresee three situations: * For projects that continue working as-is post-bump, you may wish to delete the sqlalchemy-tips jobs now. The jobs were designed as a stop gap measure to prevent projects regressing on SQLAlchemy 2.0 support while we got other projects over the line. While there is some limited value in keeping these jobs around, they do consume resources and - to the best of my knowledge - the SQLAlchemy community is not planning to release a 3.0 version of SQLAlchemy (or 2.0 version of Alembic) any time soon :) The requirements jobs gates on many projects and should identify compatibility issues with these libraries going forward, so it's probably best to drop them from the projects. I'll push a couple of patches shortly to do just this. Feel free to accept or reject them as you see fit. * For projects that are supposed to be SQLAlchemy 2.0 Ready (TM) but start seeing issues that the sqlalchemy-tips jobs didn't catch (because, say, the job was running unit tests rather than functional tests and the unit tests are missing coverage), I'd encourage you to look through what is now a fairly extensive repository of patches [3] that we've built up while addressing SQLAlchemy 2.0 compatibility issues. These are mostly all tagged with the sqlalchemy-20 topic and there's a very good chance that a patch addressing your specific issue has already been submitted to another project. * For projects that were not in the gating queue for requirements bumps, have not prepped for SQLAlchemy, and start failing catastrophically post-bump: have you not been listening to me?! :) You've got some work to do but fortunately that work should be relatively simple (if a bit tedious). As noted above, there are a *lot* of patches that you can look through to identify solutions to your compatibility issues. My preferred strategy - which is the one also advocated for by the SQLAlchemy 2.x docs [4] - has been to configure the warnings filter to raise an error for all SQLAlchemy deprecation warnings then filter out those that we actually hit and iteratively work to remove these filters. That won't work now that SQLAlchemy 2.0 is in upper-constraints so you're going to have to fix it all in one go. You may wish to adopt the standard approach locally though, before squashing all the commits upon review. This has already consumed far (far!) too much of my time so I won't offer to undertake the migration of any outstanding projects. I am, however, happy to field questions and point to existing patches that resolve a given issue. Hopefully none of it will be necessary though. Cheers, Stephen [1] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.… [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20 [4] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html

11 13

[openstack-helm] 2024.2 PTG summary
by Vladimir Kozhukalov 15 Apr '24

15 Apr '24

Dear Openstack-helmers, Here is the short summary of our discussion during PTG 2024.2. If you would like to further discuss some of these topics or have any other questions you are welcome to continue the discussion either in this mailing thread or start a new one. ## Migration from legacy Ceph OSH charts to Rook During 2024.1 we developed the migration procedure which seems to work well at least for some configurations. At the moment we have a doc [1] that describes the procedure and also there is a script [2] which can be used as an example. The procedure is obviously quite dangerous and we decided to put some efforts to create a test job for this. Anyway, at this point it is recommended to use Rook for managing Ceph clusters. Once we have the migration test job and we are sure it works well enough we will officially deprecate the legacy Ceph charts with the deprecation period of one or two release cycles. ## MariaDB operator At the moment the OSH MariaDB chart [3] is recommended by default for managing MariaDB clusters. We are not yet ready to switch to the MariaDB-operator [4] (and the mariadb-cluster OSH chart [5]) by default for our test jobs. We will continue supporting MariaDB-operator test job and will try to work closer with the operator community. Ideally, we would like the MariaDB-operator project to provide their own Helm chart which we could utilize. ## OVN Our OVN test deployment jobs seem to work well and we can recommend using OVN as the default Openstack networking backend. However, migrating from OVS to OVN is not trivial and assumes addressing at least the following two points: - Currently we don't have the migration procedure like, for example, this [6], [7] in the context of OSH. It would be great to have a volunteer to contribute this. - Some of our users have OVS customizations which seem not to have 1-1 feature parity with OVN. So, even if we have the migration strategy, the feature alignment is going to take some time. We agreed to continue supporting OVS and OVN jobs 50/50 in this release cycle. Also it seems to worth it to initiate a wider discussion about the migration itself and about the timeline. ## Tacker documentation and testing During 2023.2 we added the Tacker chart [8] but it is not mentioned in the user deployment documetation. We agreed that the Tacker team will contribute the Tacker and the Barbican sections to the OSH deployment documentation (Barbican is needed for Tacker). There is a deployment test job for Tacker [9] although it is not currently a part of the check pipeline. We agreed to improve it so it not only deploys the Tacker project but also it should run some test scenarios against it. ## Third party RabbitMQ chart Our long term strategy is to focus more on Openstack charts and to use third parties for manging backend infrastructure wherever possible. So it looks like it worth it to do research regarding possible usage of the RabbitMQ operator [10]. We agreed that we are going to put efforts during 2024.2 release cycle to give it a try and probably to create a test job that will utilize this operator. ## Improve end user experience Most of our users consume the OSH charts as part of their CI/CD infrastructure and consider OSH as a Openstack managing framework rather than end-to-end solution. However, for the new users who would like to try the Openstack running on top of K8s with the minimal efforts it is important to have an experience similar to what they usually have with other Helm charts. This assumes the following - a user has a K8s cluster (possibly with some prerequisites) and the K8s API endpoint - a user has kubectl/helm command line tools configured (to connect to K8s API endpoint) and ready to use (probably from a remote host) - a user is not supposed to clone any git repositories to deploy the Openstack - a user is able to consume Helm charts published on a Helm repo During 2024.1 we have been working to imporove the deploy-env Ansible role to support single/multi-node environments, to make it possible to run kubectl/helm CLI tools from a remote node, to expose Openstack services via Metallb, etc. At this point we are ready to have a test job that will imitate the end user behavior with the only exception that for test jobs we heavily rely on two scripts `get-values-overrides.sh` [11] and `wait-for-pods.sh` [12]. They are not absolutely necessary but still users can benefit from using them and it looks like we can easily move this functionality to a Helm plugin. So, eventually the deployment can be performed with minimal efforts using pre-published charts. ## Widen the range of K8s configurations used for Openstack deployment Now all the test jobs are run with the same K8s configuration (Calico as a CNI, Ingress-Nginx as a ingress controller) and we are trying to stay as less K8s details dependent as possible. At the same time we could probably create test jobs that will exercise some other K8s configurations like for example - Cilium/Kube-Router as a CNI implementation - HAProxy as an ingress controller This seems to be minor, but still volunteers are welcome to contribute. [1] https://docs.openstack.org/openstack-helm/latest/troubleshooting/migrate-ce… [2] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… [3] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/mariadb [4] https://github.com/mariadb-operator/mariadb-operator [5] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/mariad… [6] https://docs.openstack.org/neutron/latest/ovn/migration.html [7] https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration [8] https://opendev.org/openstack/openstack-helm/src/branch/master/tacker [9] https://opendev.org/openstack/openstack-helm/src/branch/master/zuul.d/base.… [10] https://www.rabbitmq.com/kubernetes/operator/install-operator [11] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… [12] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… -- Best regards, Kozhukalov Vladimir

1 0

[tc] Next Meeting of Technical Committee Tomorrow, April 16
by Jay Faulkner 15 Apr '24

15 Apr '24

Hi all, The next scheduled meeting of the OpenStack Technical Committee is Tomorrow, Tuesday April 16 at 1800 UTC in #openstack-tc on OFTC. The proposed agenda is: * Roll call * Follow up on tracked action items ** gmann to add grenade skip level job simplification topic in QA PTG ** gmann will send an email to the openstack-discuss list to cancel next week's TC IRC meeting * Gate health check * TC vPTG 2024.2 Follow-ups ** https://etherpad.opendev.org/p/apr2024-ptg-os-tc * TC Chair Election ** https://governance.openstack.org/tc/reference/charter.html#tc-chair * Open Discussion and Reviews See you there! -- Jay Faulkner OpenStack TC Chair

1 0

Red hat Configuration with Ceph(Cephadm tool on ubuntu
by Adivya Singh 15 Apr '24

15 Apr '24

Hi All, Regarding red hat open stack integration with ubuntu ceph installed with ceph adm tool Can you please guide me on this, What file changes need to be in Storage -config, Storage Environment for a Deployer script to pick it up regarding Cinder integration with Ceph Regards Adivya Singh

3 7

[neutron] Bug deputy report (week starting on April 8)
by Bernard Cafarelli 15 Apr '24

15 Apr '24

Hey neutrinos, here is the list of bugs filled during the PTG week - not too long as apparently people were busy attending the PTG sessions! The first 2 bugs are currently unassigned Medium * OVN DNS not working as documented - https://bugs.launchpad.net/neutron/+bug/2059405 Needs further triaging * [OVN] Changing public network MTU does not update router port gateway_mtu option - https://bugs.launchpad.net/neutron/+bug/2060828 Unassigned, reproducer described in bug Low * [ML2][OVS] more precise flow table cleaning - https://bugs.launchpad.net/neutron/+bug/2060587 Patch in progress https://review.opendev.org/c/openstack/neutron/+/915302 Opened RFE * Add 'trusted_vif' field to the port attributes - https://bugs.launchpad.net/neutron/+bug/2060916/ Opinion * already associated floating ip can be associated to another server without check or warning - https://bugs.launchpad.net/neutron/+bug/2060808 * removing floating ip from server does not check the server - https://bugs.launchpad.net/neutron/+bug/2060812 Linked bugs (one for removal, one for reassigning), discussed last week in PTG, waiting on nova team opinion Needs more info * neutron-dhcp-agent attemps to read pid.haproxy but can't - https://bugs.launchpad.net/neutron/+bug/2060974 Discussed on IRC last week, waiting for confirmation but this should soon be closed -- Bernard Cafarelli

1 0

[release] Release countdown for week R-24
by Thierry Carrez 12 Apr '24

12 Apr '24

Welcome back to the release countdown emails! These will be sent at major points in the 2024.2 "Dalmatian" development cycle, which should conclude with a final release on October 2nd, 2024. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the 2024.2 "Dalmatian" development cycle, assessing Dalmatian community goals and approving Dalmatian specs. General Information ------------------- The Dalmatian cycle is of pretty standard length, 26 weeks. In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/dalmatian/schedule.html By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… By default, all your team deliverables from the 2024.1 "Caracal" release are continued in 2024.2 "Dalmatian" with a similar release model. Upcoming Deadlines & Dates -------------------------- Dalmatian-1 milestone: May 16 -- Thierry Carrez (ttx)

1 0

[cinder][vmt] updating the cinder core security contacts team
by Brian Rosmaita 12 Apr '24

12 Apr '24

Greetings to all people interested in cinder security, The “Cinder Core security contacts” team [0] is a proper subset of the cinder core team who are notified when the OpenStack VMT assigns a private security bug to cinder following the vulnerability management process [1]. The VMT prefers that this group be kept small, so often adding someone entails removing someone else. I propose the following changes: add: - Jon Bernard (current cinder core, and Cinder PTL for Dalmatian) remove: the following people, who have not worked on security bugs over the past few development cycles: - Ivan Kolodyazhny - Sean McGinnis - Walt Boring The Cinder project team thanks Ivan, Sean, and Walt for their past security service to the project. If you have any comments or concerns, please reply on the mailing list before 12:00 UTC Friday 12 April 2024. cheers, brian [0] https://launchpad.net/~cinder-coresec/+members [1] https://security.openstack.org/vmt-process.html

3 3

[openstack][kolla-ansible]
by Nguyễn Hữu Khôi 12 Apr '24

12 Apr '24

Hello. I am testing a case when I lost all my openstack cluster. I just have a full db backup and volumes(my storage node still alive). I deploy a new cluster with same node's name after that I recover db from backup and the cluster is up but I cannot see my instances. When I click on instance tab or volume tab, I see errors: [image: image.png] I hope you will share experiences. Nguyen Huu Khoi

1 2