April 2024 - openstack-discuss - lists.openstack.org

[manila] 2024.2 Dalmatian PTG summary
by Carlos Silva 16 Apr '24

16 Apr '24

Hello everyone, Thank you for participating in the PTG last week! You can check the notes in our PTG etherpad [0], and the recording for all sessions is available in the OpenStack Manila youtube channel [1]. Here is a brief summary of what we discussed and the action items: *# Caracal Retrospective:* *Highlights:* Great internship rounds (manila-ui, docs, SDK and bandit testing). SQLAlchemy 2.0 migration is close to completion and we are planning to backport it. Doc bug squash was a success. We managed to close almost 70% of the bugs we signed up for. FFEs are allowing little time for testing and we are willing to take some different actions. *AIs:* - Schedule a mid cycle check-in with our PTG etherpad to ensure all the topics we brought up are being addressed. - Collaborative review sessions now should be proposed alongside the feature when submitting the features to gerrit. - Bug squashes and hackathons will be announced earlier, so we can try to have more traction. *# Enhance PoolWeigher to respect current share network* The PoolWeigher doesn't take into consideration the share server per share network allocation. As a result, the weigher will take one of these approaches: - Stack: create all shares in a pool even though a newly added pool is available until it is full - Spread: create shares in the pool with fewer shares Instead of looking if there is already any share server on the pool, weigh according to if there is already any share server of the same share network on the pool *AIs:* - carthaca will create a launchpad blueprint with these notes for permanence. - Enhance the poolweigher to consider the share network in the request. *# NetApp Snaplock/WORM feature* An RFE has been filed for supporting WORM shares [2] Retention time should be configurable. We can implement such things through extra specs or new APIs for such shares. *AIs*: - NetApp/SAP will ask around if there are other vendors that have the same feature. - If they don't, we should take the extra spec route and add the NetApp specific metadata keys. *# Cross-service-request-id + Enhancements to logging* We need to enable x-openstack-request-id, and use it when making cross service requests (to cinder/glance/neutron) for better tracking in the cross-service logs. We could have an implementation in Manila similar to what is available in Nova and Cinder [3]. This could become an internship theme. *AIs:* - carthaca will report a bug to document this request *# Backend driver operations based on resource metadata update API* Metadata currently is being used as tags to refer to db resources and when it is updated, we don't check if the share back end needs to update anything in the share. The backends should be notified of metadata modifications in case there is need to update something in the shares. Where metadata changes should be reflected to Backend drivers: snapshot policy, show-mount option *AIs:* - kpdev will document the proposed new behavior into a specification *# Technical Debt - SRBAC* gouthamr talked us through all of the work that is complete and what are the next steps in the next few cycles. *AIs:* - Increase test coverage. - Set "[oslo_policy]/enforce_new_defaults=True" as a default and see if it works. - Introduce the manager role for reset-state, promote out of sync replicas, force-delete resources. *# Retrospective on internship projects* We discussed what we have been working on with the ~10 interns we had in Manila at the end of the Caracal cycle, as well as our plans for Dalmatian There are also some mentoring opportunities in case people are interested. Please reach out :) *# All things CephFS* ashrodri talked us through the updates from the previous cycle and all the testing she has been doing with CI and enhancing the CI jobs, as well as introducing the ingress daemon service and so on. We are hitting some package issues for testing, and we are trying to figure those out. On new features we are planning: *- Manage/unmanage implementation in the CephFS driver* - This is already available to other drivers but we need to implement this to the CephFS drivers. We expect to be able to start on this work during the Dalmatian cycle*.* *- **Impact of the new ensure shares API in the CephFS driver* - Ensure shares comes from a request from CERN, while they had their Ceph Mon IPs changing but they needed to restart the services to reconstruct the export locations. - It ended up becoming a feature that will be used by many people and we are trying to make this as generic as possible, as it can also be useful for backend migrations. - A spec is being worked on by carloss [4] and we expect to complete this work during the Dalmatian cycle. *- DHSS=True implementation with the CephFS driver*. - We received several requests to have DHSS=True available for the CephFS driver and we are willing to start on this work after another request showed up [5]. - *AI*: gouthamr will propose a spec to document how it will work. *- NFSv3 known issue* - NFSv3 has a large number of deficiencies that NFS-Ganesha can't fill - there are external services like rpcbind, statd that handle portions of the NFS protocol that are natively supported in NFSv4.1+ - However, Microsoft Windows doesn't have a native NFSv4.1+ client; which means Windows guests can't mount CephFS/NFS shares - until now. - With Ceph Reef, the NFS service enables v4 and v3 export rules; which means a share can be mounted with NFS v3 or NFS v4.1+ - It isn't advised to mount with both protocols at the same time, nor is it advised to use NFSv3 outside of windows - because there's no recovery in case there's a failover in the NFS server *- Async mirroring* - This feature will use cephfs mirroring [6], which sounds like share replication but it has some differences. - It works similarly to the way it was designed in Cinder [7]. - *AI*: ashrodri will work on a blueprint to track this effort. *# Backup APIs* Backups are available for some releases, and NetApp + SAP are planning some enhancements and are willing to include a new backup type entity in Manila, similar to share types We agreed that the backup type properties should reuse the metadata mechanism to reuse code and UX, so we will end up also having backup type metadata. Migrations can be a concern, as the backup types were configuration blocks in the manila.conf file, so we will rely on administrators to properly migrate that, or even provide a script to do such migration. *AIs*: - NetApp will work on a specification for the backup updates *# Backup driver implementation with CBACK* CERN is working on adding a driver that integrates with their backup solution (CBACK) to manila, which uses Ceph storages as the backups backend. Zach showed us some findings during the implementation and talked about his plans and some challenges. The plan is to introduce the new backup driver during the Dalmatian cycle and also start testing on CI with it. *# Bug management stats and cycle reports* vhari shared the bug stats after the previous cycle and how encouraging they are. We reflected on the data and had a suggestion to expand our bugsquash events to two in a cycle, so that can help us reduce the bug backlog quicker. We had great stats on reducing doc bugs backlog with the help of a bugsquash late in the previous cycle. We also talked about possible role rotation and discussed what it means to be the Manila bug representative. *# Provide support for efficiency policy on NetApp shares* NetApp is figuring out a way to determine efficiency policies during the creation of shares individually. They want to implement this by introducing new share type extra specs to configure such things. DHSS=True is still being discussed, and this feature will be worked on for DHSS=False first. NetApp intends to implement this feature during the Dalmatian cycle. *# Technical Debt - sqlalchemy 2.0* Thanks to stephenfin and zzzek, all changes for sqlalchemy 2.0 were merged in the beginning of the Dalmatian release, as some issues showed up in the process and we couldn't figure them out. We will be backporting the changes that didn't make it to Caracal, so sqlalchemy 2.0 will also be supported by Manila. There are no current plans for backporting this further. We also need to evaluate removing the backref to back_populates. *AIs:* - Complete the backport of the changes to 2024.1 caracal *# Add response schema validation* The SDK team would like to generate OpenAPI schemas stored in-tree, to ensure things are complete and up-to-date, and to avoid landing another large deliverable on the SDK team, and to allow service teams to fix their own issues. OpenAPI 3.1 is a superset of JSONSchema, which means we can use the same tooling we currently use for this. *AIs*: - stephenfin will work on a spec and on bootstraping it, and more people would be able to pick it up. *# Shares encryption* We had a brief discussion about the design of this feature for Manila, which the spec is already merged. kpdev has already proposed two patches for this change [8] *AIs:* schedule a collaborative review early in the cycle so we can start reviewing it as early as possible. *# DB migration script squash * tkajinam brough to our attention during the previous cycle that there was room for enhancement in our migration scripts, as there are 70+ of them now and we could squash them. We are also considering automated ways to do this with sqlalchemy. *AIs*: - gouthamr/tkajinam will reach out to zzzek and stephenfin and ask if there is an automated way to make this process easier. [0] https://etherpad.opendev.org/p/dalmatian-ptg-manila [1] https://www.youtube.com/@openstackmanila/ [2] https://bugs.launchpad.net/manila/+bug/2060027 [3] https://blueprints.launchpad.net/nova/+spec/cross-service-request-id [4] https://review.opendev.org/c/openstack/manila-specs/+/915480 [5] https://github.com/nfs-ganesha/nfs-ganesha/issues/1087 [6] https://docs.ceph.com/en/latest/dev/cephfs-mirroring/ [7] https://docs.openstack.org/cinder/latest/contributor/replication.html [8] https://review.opendev.org/q/topic:%22bp/share-encryption%22 If you have questions, please feel free to reach out. Looking forward to a very productive release cycle with you! Regards, carloss

1 0

[kolla-ansible][Antelope][Heat] Heat UpdateWaitConditionHandle Issues
by Devin Patterson 16 Apr '24

16 Apr '24

Hello, We are creating a workflow to use Heat auto-scaling groups with wait conditions. The ideal goal would be to have the wait condition clear and require another signal when the Heat stack is updated. Trying to use the resource OS::Heat::UpdateWaitConditionHandle instead of OS::Heat::WaitConditionHandle. This works during initial creation, but when I udpdate the stack with a new configuration, I get the following error. UpdateWaitConditionHandle.update() got an unexpected keyword argument 'new_template_id' Any ideas on how to use wait conditions during stack updates or to fix the above error. Thanks in advance. Devin

2 2

[Glance] Dalmatian PTG Summary
by Pranali Deore 16 Apr '24

16 Apr '24

Hello Everyone, We had our Dalmatian virtual PTG between 9th APRIL to 12th APRIL 2024. Thanks to everyone who joined the virtual PTG sessions. Using Opendev Meetpad we had all the discussion around different topics for glance. Here is the short summary of the discussions. Tuesday , APRIL 9th 2024 # Caracal Retrospective On a positive note we have completed few important things like adding support of new centralized_db cache driver which will be enabled default for caching related operations and stop using sqlite database from caracal release onwards and deprecated Sqlite cache driver, we also have deprecated glance-cache-manage command in favor of the new Cache API, glance scrubber utility and its associated configuration options, location_strategy option since in Bobcat, we added new weighing mechanism which made the location strategy no longer required and continued working on adding new location API support which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. On the other side we decided to speed up the work we've had in flight and speed up the reviews. # Add missing CLI support for some Glance API in Openstack Client The migration work for adding missing CLI was initiated in the antelope cycle. We have added the SDK support for cache API , images API and metadef API There are image-create-via-import, image-upload, member commands & metadef tags patches are under review. This cycle we are hoping to get all the above mentioned patches merged and mark this work complete. # Introduce new sub-store NFS for filesystem store Currently in glance when we configure nfs store, we do not store any information about what server nfs is configured, what mount point is configured, if it is available or not. From the beginning nfs is treated as a local FS store in glance. So if the NFS server is down , we don't have any idea about that, we write the image to the local FS store and it doesn't get reflected to the nfs store if nfs is down. We decided to have this support similar to cinder, when it configures nfs backend it checks at the initial level whether mount point is available or not and again checks during volume upload whether connected to nfs mount point We will enhance file system store by introducing some config parameters to store this information by creating separate ini file which can have information related to nfs server store with which we can check nfs server is reachable or not with the use of os-brick library. # Driver test coverage Currently glance has CI coverage for 'cinder', 'rbd', 'swift' stores and functional tests for FS store. Vmware driver is deprecated in 2024.1 but we decided to keep it and revise the status after updating the existing thread with the outcome in manila ptg. For S3 driver, we decided to ask for any interest in maintaining s3 driver (with adding CI, fixing bugs, etc) and depending on the response we will go ahead and add the coverage. # DB Migration script cleanup Glance has been using alembic migrations on the upgrading database schema from old version to the new version, currently glance maintains migration scripts from liberty and upgraded as we moved ahead cycle wise. So we decided to squash some of these[1] until the oldest release not EOLed and which is wallaby currently and decided to document this and from now onwards we will support migration scripts post EOL cycle and scrub the migration once the cycle is EOL. Wednesday April 10th , 2024 # Migrate existing images to new store at once Glance currently supports copying existing images to another available store via import workflow but one image at a time. If the deployer wants to desommision the existing store and add a new one, then all the images need to be copied to the new store and delete those one by one from the existing one. In onder to solve this problem we decided to provide glance-manage command line utility by introducing new operation copy-images to copy all or selective images in newly added store and if we add a flag 'delete_from_source' and set it to true we can delete all the images from the source store Example: glance-manage copy_images --source-store store_1 --destination-store store_2 --delete_from_source After all images are copied we will print the stat on the console - copied images, deleted images, ignored- images ( if user deletes any image in between the copy operation then, we will raise and catch the NotFound exception and add it to the ignored stores. Admin can check the ignored images issue and incase it's present then the process can be reinitiated) # Introduce common configuration option(s) at base store level which can be used/accessible as a common for all available stores? Since glance supports separate configuration options for each store incase of multi store enabled with the same store type. So we decided to introduce common configuration options at base store level which can be used/accessible as a common for all available stores similar to cinder [2] # Generic Image migration + Optimization for same store migration Currently the preferred way of migration in glance is two step 1. Copy the image from source store to destination store 2. delete the image from source store With the above approach there are 2 problems: 1. requires manual intervention from operators after waiting for image copy to finish and then delete the image from source store 2. No way for stores to optimize the operation We decided to address it by introducing a migration operation in glance which will have two features [3] 1. A generic migration workflow where we will perform the image copy and delete in the same API operation 2. Allow an interface for glance store methods to optimize the migration if possible else it will fall back to the generic workflow # New Location APIs We have decided to have support of new location apis which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. The implementation patches are up for review along with the unit/functional tests, documentation, api version bump, client & sdk side support and tempest coverage[2] The Poc patches of nova and cinder are also there to test this new location api functionality. During last cycle we have noticed few issues with cinder store while testing the cinder side POC patches, so the glance_store & cinder side fix is up for review, and once those patches are merged and glance_store is released, we can merged the New Location API patches after getting approval from Nova & Cinder side since both are the consumers of these 2 new APIs. We also decided to have a new CI job where all newly added tempest tests for these new APIs are passed and add command line utility to re-calculate the hash of the active images which are without hash if hash calculation fails during location-import. Thursday April 11th , 2024 # Add support to test whether swift is deployed or not We have noticed that when glance is configured to use swift as a glance store and swift service is not enabled/deployed/down then glance api service starts but image creation fails and image will remain in importing state forever. In case of import we upload the data at staging store first and then we try to import it to swift store, at that time We decided to add the support at the store side to check at the startup if swift is up and running and if it's not running then log a warning and disabled the store for adding & delete opearetion. # Deprecate and remove 'metadata_encryption_key' and related functionality 'metadata_encryption_key' config option and its related functionality was added quite a long back ago and even though as per the description it encrypts the location metadata, it actually encrypts location url. It encrypts the location url only for image upload/import/download/show APIs, doesn't encrypt url on location APIs. If it gets enabled during upgrade then it will break the existing deployment since existing image urls are not being encrypted. Hence we decided to deprecate it in this cycle and if there is no objection seen then we will remove it in the next cycle or may this the current cycle as well. # Shelved instance image protection This is an Optional feature proposal to prevent deletion of shelved instance's image. Currently when an instance is shelved its image can be removed and afterwards an instance cannot be booted again. Glance team has suggested checking the implementation of 'cinder_encryption_key_deletion_policy' and check whether it is possible to implement on a similar count and update the existing spec[4] accordingly. Apart from above topics you can find other miscellaneous topics discussed in PTG etherpad [5] and the recordings of all the sessions and milestone wise priorities at the end. If you have any questions/suggestions then please join us in our weekly meeting(each Thursday on #openstack-meeting irc channel at 1400 UTC) [1]: https://review.opendev.org/q/topic:%22db-migration-squash%22 [2]: https://opendev.org/openstack/cinder/src/commit/935f08e032b31d4628881d3a053… [3] : https://review.opendev.org/c/openstack/glance-specs/+/914639 [4]: https://review.opendev.org/c/openstack/glance-specs/+/915475/1/specs/2024.2… [5]: https://etherpad.opendev.org/p/apr2024-ptg-glance#L1

1 0

[ironic][PTG] Ironic 2024.2 PTG summary
by Riccardo Pittau 16 Apr '24

16 Apr '24

Hello Ironicers! Here's a summary of the discussions around the major topics at the Ironic PTG 2024.2 # Caracal retrospective # We took our time to look back at the Caracal development cycle, how it was for the ironic project, celebrate what we've accomplished, and of course also understand if some of the work started during Caracal needed to be continued in Dalmatian. We're quite happy to say that most of the goals we've set for Caracal have been reached, and we've actually been able to do much more than planned! For example we delivered the HttpBoot support for Redfish, we've been able to tackle a lot of old bugs and keep pace with the migration from storyboard to launchpad, almost finished the merge of Inspector into Ironic, and marked multiple old drivers for removal. # Present and Future of Ironic # We've analyzed the current status of Ironic and asked ourselves: "Where do we want Ironic to go?" In the past years we started looking beyond Openstack and we've took some steps to get into that direction, and we can say that Ironic went over the Openstack borders, becoming more independent. Currently Ironic can be used in three ways: - Openstack Integrated, the more clear and established way - as the core engine of Metal3, with good documentation for various use cases, but still growing and with margin for improvement - standalone, which still requires some work, a much clearer path forward and requirements We really want to keep investing in standalone Ironic more in the future, but to do that we need to take some decisions based on the vision we have on server provisioning from here to 5 years. Lots of questions we asked ourselves on how to resolve some problems connected to having a standalone bare metal provisioning service, mostly related to networking (e.g. secureboot, virtualmedia), will expand and find possible answers in other topics. # Redfish topics # Talking about present and future, Redfish is the current constantly evolving and improving standard protocol to directly communicate with controllers of bare metal devices. Ironic supports it since a while and of course there's more and more demand for new features to support or to expand the current capabilities. The discussion was mainly around an alternative, and possibly easier way to provide images for firmware updates and virtual media boot using a "push" model instead of the current "pull", with two proposals: - using the UpdateService already present in the Redfish standard - using the Virtual Media Push as a new feature currently under discussion with DMTF Another major topic is the support for TLS connection to the BMC, which already has a design proposal from one major contributor, Fujitsu. In addition we'll also have to explore the "other side" of the connection, so from the BMC to the ironic services, which will be one of the goals for Dalmatian. # CI related topics# CI has always had great importance for the Ironic project being one of the major consumers in terms of resources. It's not a case that that first related topic was how to slim down the CI usage! During the Caracal cycle we've been able to reduce the number of jobs run by the Ironic project, but we've also added some more. We came to the conclusion that we need to take an approach where we stack more tests in fewer jobs, trying to consolidate down jobs as much as possible and minimize boot interface variation testing. During the Dalmatian cycle we'll work first on trying to have Redfish and ipmi in a single ironic job, and update the list of jobs to understand where we can avoid duplication. At some point we'll also have to make space for some more deep tests on ARM machines. We've also decided to give Codespell tests a try, and explore the possibility of having some pre-commit hooks, and, after some technical considerations, move away from Tinycore as our base for the IPA ramdisk in CI, gentoo being the most likely candidate as a replacement. # xPUs # With xPUs in this case we mean IPU and DPU (not to be confused with TPU or GPU) which are some Processing Units that are designed for specific applications and data types. Usually DPUs are embedded in edge network cards, while IPUs are specifically designed for Artificial Intelligence/Machine Learning. There are multiple challenges in order to support those devices, depending on the "vision" of the vendors, some view them having full Operating Systems, some as having an OS just as "firmware", most or all of them would like remote management, which is where Ironic can really make an impact. Even though there are multiple use cases and scenarios where the use of xPUs is highly valued, and the wishlist of potential features is quite long, the first step we need to take is to get some real hardware and start testing, which is the real challenge we'll have to deal with next. # Secure Boot # We've made huge steps forward in supporting Secure Boot, the next big one would be supporting custom keys and providing means to upload and update them. Before doing that we need to clarify some aspects of Secure Boot, especially the "security vs convenience". In the Dalmatian cycle will focus mainly on updating our documentation, provide a better picture to users, and start exploring possible solutions for the secure custom keys management. # Future of Networking # Reconnecting to the future of Ironic, networking represents the next step for a truly standalone Ironic. This means finding alternatives to Openstack-integrated scenarios and therefore to Neutron. In Metal3 we're solving the problem using a separate IPAM, but this is still not enough for a complete Ironic standalone, in the end what we really need is flat network control, which at the moment we don't really have an alternative for. We realized that the topic is vast, we've just scratched the surface and we need to invest much more time on it, having more discussions to go deeper into it and start researching alternatives and tools for our goal. That is (almost) all folks! You can find all the topics and more details in the discussions notes here: https://etherpad.opendev.org/p/ironic-ptg-april-202 Feel free to answer directly in this thread in case you have questions or comments, or contact the ironic team on IRC. Thanks to everyone who participated in the PTG! Have a great Dalmatian development cycle! Ciao Riccardo

1 0

[publiccloud-sig] Reminder - next meeting April 17th - 0700 UTC
by Felix Kronlage-Dammers 16 Apr '24

16 Apr '24

Hi everyone, This Wednesday (April 17th) the next meeting of the public cloud sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG See you tomorrow! felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[all] ORM apocalypse mostly averted (a.k.a. SQLAlchemy 2.0 is here)
by Stephen Finucane 15 Apr '24

15 Apr '24

o/ A whole 2 years and 8 months to the day after I first communicated about it [1], it would appear the requirements bump to SQLAlchemy 2.0 [2] is finally ready to go. I suspect most if not all service projects are familiar with SQLAlchemy 2.0 at this point, having needed to fix whatever compatibility issues were seen. For those that aren't, I suggest reading my previous emails on the matter. So what happens next? Hopefully for the vast majority of projects the answer should be "nothing": many projects now have sqlalchemy-tips jobs that are designed to run unit and/or integration tests with the latest versions of SQLAlchemy, Alembic and oslo.db so the switchover should be seamless. I foresee three situations: * For projects that continue working as-is post-bump, you may wish to delete the sqlalchemy-tips jobs now. The jobs were designed as a stop gap measure to prevent projects regressing on SQLAlchemy 2.0 support while we got other projects over the line. While there is some limited value in keeping these jobs around, they do consume resources and - to the best of my knowledge - the SQLAlchemy community is not planning to release a 3.0 version of SQLAlchemy (or 2.0 version of Alembic) any time soon :) The requirements jobs gates on many projects and should identify compatibility issues with these libraries going forward, so it's probably best to drop them from the projects. I'll push a couple of patches shortly to do just this. Feel free to accept or reject them as you see fit. * For projects that are supposed to be SQLAlchemy 2.0 Ready (TM) but start seeing issues that the sqlalchemy-tips jobs didn't catch (because, say, the job was running unit tests rather than functional tests and the unit tests are missing coverage), I'd encourage you to look through what is now a fairly extensive repository of patches [3] that we've built up while addressing SQLAlchemy 2.0 compatibility issues. These are mostly all tagged with the sqlalchemy-20 topic and there's a very good chance that a patch addressing your specific issue has already been submitted to another project. * For projects that were not in the gating queue for requirements bumps, have not prepped for SQLAlchemy, and start failing catastrophically post-bump: have you not been listening to me?! :) You've got some work to do but fortunately that work should be relatively simple (if a bit tedious). As noted above, there are a *lot* of patches that you can look through to identify solutions to your compatibility issues. My preferred strategy - which is the one also advocated for by the SQLAlchemy 2.x docs [4] - has been to configure the warnings filter to raise an error for all SQLAlchemy deprecation warnings then filter out those that we actually hit and iteratively work to remove these filters. That won't work now that SQLAlchemy 2.0 is in upper-constraints so you're going to have to fix it all in one go. You may wish to adopt the standard approach locally though, before squashing all the commits upon review. This has already consumed far (far!) too much of my time so I won't offer to undertake the migration of any outstanding projects. I am, however, happy to field questions and point to existing patches that resolve a given issue. Hopefully none of it will be necessary though. Cheers, Stephen [1] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.… [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20 [4] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html

11 13

[openstack-helm] 2024.2 PTG summary
by Vladimir Kozhukalov 15 Apr '24

15 Apr '24

Dear Openstack-helmers, Here is the short summary of our discussion during PTG 2024.2. If you would like to further discuss some of these topics or have any other questions you are welcome to continue the discussion either in this mailing thread or start a new one. ## Migration from legacy Ceph OSH charts to Rook During 2024.1 we developed the migration procedure which seems to work well at least for some configurations. At the moment we have a doc [1] that describes the procedure and also there is a script [2] which can be used as an example. The procedure is obviously quite dangerous and we decided to put some efforts to create a test job for this. Anyway, at this point it is recommended to use Rook for managing Ceph clusters. Once we have the migration test job and we are sure it works well enough we will officially deprecate the legacy Ceph charts with the deprecation period of one or two release cycles. ## MariaDB operator At the moment the OSH MariaDB chart [3] is recommended by default for managing MariaDB clusters. We are not yet ready to switch to the MariaDB-operator [4] (and the mariadb-cluster OSH chart [5]) by default for our test jobs. We will continue supporting MariaDB-operator test job and will try to work closer with the operator community. Ideally, we would like the MariaDB-operator project to provide their own Helm chart which we could utilize. ## OVN Our OVN test deployment jobs seem to work well and we can recommend using OVN as the default Openstack networking backend. However, migrating from OVS to OVN is not trivial and assumes addressing at least the following two points: - Currently we don't have the migration procedure like, for example, this [6], [7] in the context of OSH. It would be great to have a volunteer to contribute this. - Some of our users have OVS customizations which seem not to have 1-1 feature parity with OVN. So, even if we have the migration strategy, the feature alignment is going to take some time. We agreed to continue supporting OVS and OVN jobs 50/50 in this release cycle. Also it seems to worth it to initiate a wider discussion about the migration itself and about the timeline. ## Tacker documentation and testing During 2023.2 we added the Tacker chart [8] but it is not mentioned in the user deployment documetation. We agreed that the Tacker team will contribute the Tacker and the Barbican sections to the OSH deployment documentation (Barbican is needed for Tacker). There is a deployment test job for Tacker [9] although it is not currently a part of the check pipeline. We agreed to improve it so it not only deploys the Tacker project but also it should run some test scenarios against it. ## Third party RabbitMQ chart Our long term strategy is to focus more on Openstack charts and to use third parties for manging backend infrastructure wherever possible. So it looks like it worth it to do research regarding possible usage of the RabbitMQ operator [10]. We agreed that we are going to put efforts during 2024.2 release cycle to give it a try and probably to create a test job that will utilize this operator. ## Improve end user experience Most of our users consume the OSH charts as part of their CI/CD infrastructure and consider OSH as a Openstack managing framework rather than end-to-end solution. However, for the new users who would like to try the Openstack running on top of K8s with the minimal efforts it is important to have an experience similar to what they usually have with other Helm charts. This assumes the following - a user has a K8s cluster (possibly with some prerequisites) and the K8s API endpoint - a user has kubectl/helm command line tools configured (to connect to K8s API endpoint) and ready to use (probably from a remote host) - a user is not supposed to clone any git repositories to deploy the Openstack - a user is able to consume Helm charts published on a Helm repo During 2024.1 we have been working to imporove the deploy-env Ansible role to support single/multi-node environments, to make it possible to run kubectl/helm CLI tools from a remote node, to expose Openstack services via Metallb, etc. At this point we are ready to have a test job that will imitate the end user behavior with the only exception that for test jobs we heavily rely on two scripts `get-values-overrides.sh` [11] and `wait-for-pods.sh` [12]. They are not absolutely necessary but still users can benefit from using them and it looks like we can easily move this functionality to a Helm plugin. So, eventually the deployment can be performed with minimal efforts using pre-published charts. ## Widen the range of K8s configurations used for Openstack deployment Now all the test jobs are run with the same K8s configuration (Calico as a CNI, Ingress-Nginx as a ingress controller) and we are trying to stay as less K8s details dependent as possible. At the same time we could probably create test jobs that will exercise some other K8s configurations like for example - Cilium/Kube-Router as a CNI implementation - HAProxy as an ingress controller This seems to be minor, but still volunteers are welcome to contribute. [1] https://docs.openstack.org/openstack-helm/latest/troubleshooting/migrate-ce… [2] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… [3] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/mariadb [4] https://github.com/mariadb-operator/mariadb-operator [5] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/mariad… [6] https://docs.openstack.org/neutron/latest/ovn/migration.html [7] https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration [8] https://opendev.org/openstack/openstack-helm/src/branch/master/tacker [9] https://opendev.org/openstack/openstack-helm/src/branch/master/zuul.d/base.… [10] https://www.rabbitmq.com/kubernetes/operator/install-operator [11] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… [12] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… -- Best regards, Kozhukalov Vladimir

1 0

[tc] Next Meeting of Technical Committee Tomorrow, April 16
by Jay Faulkner 15 Apr '24

15 Apr '24

Hi all, The next scheduled meeting of the OpenStack Technical Committee is Tomorrow, Tuesday April 16 at 1800 UTC in #openstack-tc on OFTC. The proposed agenda is: * Roll call * Follow up on tracked action items ** gmann to add grenade skip level job simplification topic in QA PTG ** gmann will send an email to the openstack-discuss list to cancel next week's TC IRC meeting * Gate health check * TC vPTG 2024.2 Follow-ups ** https://etherpad.opendev.org/p/apr2024-ptg-os-tc * TC Chair Election ** https://governance.openstack.org/tc/reference/charter.html#tc-chair * Open Discussion and Reviews See you there! -- Jay Faulkner OpenStack TC Chair

1 0

Red hat Configuration with Ceph(Cephadm tool on ubuntu
by Adivya Singh 15 Apr '24

15 Apr '24

Hi All, Regarding red hat open stack integration with ubuntu ceph installed with ceph adm tool Can you please guide me on this, What file changes need to be in Storage -config, Storage Environment for a Deployer script to pick it up regarding Cinder integration with Ceph Regards Adivya Singh

3 7

[neutron] Bug deputy report (week starting on April 8)
by Bernard Cafarelli 15 Apr '24

15 Apr '24

Hey neutrinos, here is the list of bugs filled during the PTG week - not too long as apparently people were busy attending the PTG sessions! The first 2 bugs are currently unassigned Medium * OVN DNS not working as documented - https://bugs.launchpad.net/neutron/+bug/2059405 Needs further triaging * [OVN] Changing public network MTU does not update router port gateway_mtu option - https://bugs.launchpad.net/neutron/+bug/2060828 Unassigned, reproducer described in bug Low * [ML2][OVS] more precise flow table cleaning - https://bugs.launchpad.net/neutron/+bug/2060587 Patch in progress https://review.opendev.org/c/openstack/neutron/+/915302 Opened RFE * Add 'trusted_vif' field to the port attributes - https://bugs.launchpad.net/neutron/+bug/2060916/ Opinion * already associated floating ip can be associated to another server without check or warning - https://bugs.launchpad.net/neutron/+bug/2060808 * removing floating ip from server does not check the server - https://bugs.launchpad.net/neutron/+bug/2060812 Linked bugs (one for removal, one for reassigning), discussed last week in PTG, waiting on nova team opinion Needs more info * neutron-dhcp-agent attemps to read pid.haproxy but can't - https://bugs.launchpad.net/neutron/+bug/2060974 Discussed on IRC last week, waiting for confirmation but this should soon be closed -- Bernard Cafarelli

1 0