- openstack-discuss - lists.openstack.org

Whitepaper Collaboration - Linux, OpenStack & Kubernetes
by Kendall Nelson 19 Apr '24

19 Apr '24

Hello Folks, As you hopefully know, the OpenInfra Foundation has a history of helping to define patterns and new usage patterns in our ecosystem through its collaboration on whitepapers. With the user survey, we have seen a distinct pattern of users leveraging an entirely open source stack to meet their needs - Kubernetes, OpenStack and Linux together. So, we are planning the compilation of a whitepaper further explaining this integration, planning to publish in September of this year. Continuing on the theme of collaboration, I would love help with both the writing and the reviewing of two sections in particular - the introduction and integration sections of the whitepaper. Other sections that will be included are model architectures and user case studies from OpenInfra member companies making use of this infrastructure pattern. If you are interested in collaborating on this whitepaper, please contact me directly and I will get you in the loop! Drop me an email to knelson(a)openinfra.dev -Kendall

1 0

[designate] Support for DNSSEC // Bump in the wire signer via BIND9 or Knot
by Christian Rohmann 19 Apr '24

19 Apr '24

Hello, I was wondering is anybody knew about the current state of DNSSEC signing for Designate-managed zones. Since Designate MDNS serves as primary they actually should do the signing / provide already signed zones via zone transfers. Adding support for DNSSEC this was last discussed for Kilo [1], but that spec was never finished, DNSSEC support never implemented. One approach to do this is using a bump in the wire signer [2][3][4] and have an intermediate BIND9 or Knot server doing the signing. Has anybody implemented something of this kind? If so, how do your users receive their initial DS / DNSKEY for the parent zone? Regards Christian [1] https://review.opendev.org/c/openstack/designate-specs/+/132571 [2] https://bind9.readthedocs.io/en/latest/dnssec-guide.html#bump-in-the-wire-s… [3] https://jpmens.net/2023/07/22/adieu-opendnssec-bienvenido-knot-dns/ [4] https://labs.ripe.net/author/anandb/dnssec-signer-migration/

4 7

[kolla] openstack centralized logging
by Eugen Block 18 Apr '24

18 Apr '24

Hi, I'm trying to find out how the "Venus" project could work. Unfortunately, there's not much to find yet. There's no installation guide as for other projects although it already exists since Zed release. I tried enabling it in my kolla all-in-one test deployment (2023.1), the api and manager container are running, but I have no idea how to get some data into or out of it. My conclusion is that it's not a mature project yet, could anyone comment on that? As an alternative, I enabled centralized logging in the same deployment as well, but unfortunately I can't use the dashboard because of [1]. So I don't really know what I can get out of opensearch in order to have a centralized logging. If possible, I would like to adapt that to a non-containerized openstack deployment. How can I change the opensearch version in my kolla deployment to be able to login and inspect what I got? Any pointers and comments are highly appreciated! Thanks! Eugen [1] https://bugs.launchpad.net/kolla-ansible/+bug/2060306

4 9

[tacker][ptg] Dalmatian PTG Summary
by Yasufumi Ogawa 18 Apr '24

18 Apr '24

Hi all, Here is a summary of Tacker team's two days discussion during vPTG 2024.2[1]. # Revise FTs We've tried to revise functional tests on zuul for the latest cycles to reduce the burden of rechecks for unexpected failures which are sometimes happened especially at the end of the cycles. We have several targets for the revise, such as clean dir structure, reduce the amount of test scenarios or so, and share the status of each tasks. # Progress report for Legacy API deprecating/obsoleting Done dropping all the regacy codes from tacker and its client, but still remained in tacker-horizon. We're going to complete this task in this release. # Opentelemetry plugin Support for opentelemetry's observability was proposed in Caracal, but couldn't be finished because of some more higher priority tasks. We've agreed to change the plan to release the feature is this release. # Support devstack install for Cilium as a CNI of Kubernetes VIM We released a sample VNF package using Cilium CNI for demonstrating VNF applications can communicate via Cilium's high performance connectivity for considering vRAN usecases. # Demonstration for tacker-horizon NFV Orchestration API v2.0 Shared the latest GUI support of tacker-horizon for managing NFV orchestration via ETSI SOL v2.0 APIs. # TOSCA 1.3 support Tacker supports TOSCA as a VNF descriptor defined in OASIS standards[2], but the latest version in the support is TOSCA 1.2. Although there are two choices for us for newer ones, 1.3 or 2.x, we've agree to introduce 1.3 which is considered as the most active version currently and shared the implementation plan in this release. # Proposal for adding Tacker installation with openstack-helm in offical OpenStack-helm document We added tacker support in openstack-helm without documentation in the previous cycle. So, we'll complete the remained task in Dalmatian. # Fix schema definition error of TST 2.6.1 repository TST is an ETSI compliance test suites and each of its versions is compatible with SOL specification's as a maintenance policy in the community. We're going to complete to implement TST 2.6.1. The problem is that there is unexpected data field `virtualLinkDescId` in TST is incompatible with SOL standards. It should be in TST while keeping the versions. We'll propose the issue and fix to the standards. # Performance improvement of "list" APIs There is a performance issue in APIs for retrieving entries from a user feedback and it can be a problem especially in commercial usecases managing large amount of objects. There are several possibility of the performance issue such as lack of database optimization or implementation design is not still well considered. For the issue, we've started to figure out the cause of the issue from some more detailed experimentation. [1] https://etherpad.opendev.org/p/apr2024-ptg-tacker [2] https://groups.oasis-open.org/communities/tc-community-home2?CommunityKey=f…

1 0

[all][unmaintained] Gate status of Unmaintained branches
by Elõd Illés 18 Apr '24

18 Apr '24

Hi, Recently, based on the 'Unmaintained replaces Extended Maintenance' resolution [1], old branches (Xena, Wallaby, and Victoria), originally in Extended Maintenance, moved to Unmaintained, together with Yoga branch that transitioned directly from maintained to Unmaintained as a first such branch. From now on, there is the possibility for folks who are interested in further maintaining these old branches (in the so called 'Unmaintained' state :)). For that, maintainers need to sign up to be a member of openstack-unmaintained-core group (or the project specific group, if such exists). There is no requirement to be part of that group, but we would like to ask 'unmaintainers' to follow the stable policy [2], if possible, for unmaintained/* branches as well (*if the given project follows it on maintained branches). To signal your interest, please write a mail to this mailing list, or ping openstack-unmaintained-core group members (for example me :)) on IRC. As a first important thing, i would like to call attention to the following part of the resolution: 'The CI for all branches must be in good standing at the time of opt-in. At a minimum this needs to contain all integrated jobs, unit tests, pep8, and functional testing. However, as this is a best-effort CI and to preserve resources, Unmaintained branches will include periodic jobs of no higher than monthly frequency.' [1] However, while many gates look healthy [3] (i.e. patches have merged), there are still a significant number of failing CI gates [4]. So, to these projects and their unmaintained/.* branches, to be eligible to kept open for further maintenance and be able to opt-in in the next cycle for maintenance, these gates have to be fixed as soon as possible. Otherwise project branches need to be deleted and transitioned to End of Life. As an example for a simple gate fix, many project/branch need to drop their failing 'lower-constraints' job (or fix them). Also a good starter is to fix zuul config errors [5] if there exists any. Final note from me: as the above quote shows, the concept of good standing CI is quite minimal. Any further CI jobs (devstack based tempest jobs) add value to these branches for those who consume them. So any such work is really appreciated. If you have any question, feel free to ask, either here, on the mailing list, or ping me or the TC on IRC. Thanks for reading this, Előd Illés irc: elodilles @ #openstack-release #openstack-stable #openstack-unmaintained [1] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… [2] https://docs.openstack.org/project-team-guide/stable-branches.html [3] https://review.opendev.org/q/branch:%5Eunmaintained/.*+label:Verified%252B2… [4] https://review.opendev.org/q/branch:%5Eunmaintained/.*+label:Verified-1,225 [5] https://zuul.opendev.org/t/openstack/config-errors?branch=unmaintained%2Fyo…

1 0

[heat] Parametrization of Neutron trunk subports scaling down
by Lajos Katona 17 Apr '24

17 Apr '24

Hi, With Hot templates we run into a scaling issue with Neutron trunk subports. The task is to create Neutron trunks with "vlan_count" number of subports, and with stack update be able to add or remove subports of the created trunk. We tried with resourceGroup, and that works well if we increase the number of subports. The problem comes with decrease, as with resourceGroups I can't (my lack of knowledge) tell heat_engine that first update the trunk and remove the subport and after that delete the port. And Neutron API gives back conflict as the port which heat_engine tries to delete is still used by the trunk. Do you know a way to accomplish such an operation with Hot templates? Thanks in advance for the help. Best wishes Lajos Katona (lajoskatona)

2 4

[magnum] Dalmatian PTG summary
by Jake Yip 17 Apr '24

17 Apr '24

Hi all, Magnum held our PTG on 2024-04-10 0600-0800 UTC. I have summarised some points that may be of interest. 1. Magnum team is small, we welcome help with reviews, documentation, deprecation. 2. In Caracal, we have done a lot of work improving the CI. This allowed us to confidently remove old code that has been dragging us down. A big shout out to mnasiadka for this effort. For Dalmatian, we are improving this further with sonobuoy tests, etc. 3. In Caracal, we enhanced the driver code to support multiple drivers. Historically, Magnum drivers were in tree. It was difficult to support out-of-tree drivers without clashing. With Caracal, now an operator can have multiple active drivers in their installation, and can also move to new drivers in the future. 4. ClusterAPI driver There are two ClusterAPI drivers developed by different organisations (outside Magnum). - magnum-capi-helm - out-of-tree, in governance - https://github.com/openstack/magnum-capi-helm - magnum-cluster-api - out-of-tree, outside goverance - https://github.com/vexxhost/magnum-cluster-api StackHPC as kindly offered to move their magnum-capi-helm driver to OpenStack governance, and will work with Magnum team to get it working in CI in Dalmatian. Thanks mnasiadka, mkjpryor, johnthetubaguy, travisholton, for this work. A number of users are also reporting success with magnum-cluster-api driver. Feel free to try this out and ask questions on ML. Thanks mnaser who have been active on the ML answering questions. 3. Heat driver Currently the Heat driver supports Kubernetes v1.27 in Caracal. We aim to get v1.28 supported in Dalmatian. v1.29 is already supported in both ClusterAPI drivers. Heat driver still needs to be supported for possibly one/two cycles once ClusterAPI driver is available, so we target F for removal of Heat driver. For more information, please see the etherpad [1], or join the Magnum meeting if you have any questions. Regards, Jake

1 0

[qa][ptg] QA Dalmatian vPTG Summary
by Martin Kopec 17 Apr '24

17 Apr '24

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Caracal Retrospective == We still manage making releases, doing code reviews and bug triaging even with only a few active contributors. Kudos to them. == Simplify grenade skip level jobs == It was agreed that we want to run grenade-skip-level job always as voting by default in template and project can opt-out for non-SLURP release. == Replace paramiko with libssh2 == It was agreed that the priority of getting rid of paramiko should take place at services' side first. After all services are done with this change, then we can proceed with this on tempest's side. In the meantime, we at least know that tempest works with the FIPS environment [2]. == Triage previous and write new priority items == The list of the priority items planned for this cycle as well as the ones which were carried over from the previous cycles, can be found here [3]. [1] https://etherpad.opendev.org/p/apr2024-ptg-qa [2] https://zuul.openstack.org/builds?job_name=tempest-centos9-stream-fips&skip… [3] https://etherpad.opendev.org/p/qa-caracal-priority Thank you all for attending the PTG and contributing to the discussion, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

volume in use
by kjme001＠gmail.com 17 Apr '24

17 Apr '24

helllo, I have the following problem that after deleting the instance, the volume remains in use and is connected to an instance that does not exist. I have to execute the command: cinder reset-state --state available --attach-status detached name_volume then the volume becomes available and is not connected to the instance. How to solve this problem? Regards

2 1

[manila] 2024.2 Dalmatian PTG summary
by Carlos Silva 17 Apr '24

17 Apr '24

Hello everyone, Thank you for participating in the PTG last week! You can check the notes in our PTG etherpad [0], and the recording for all sessions is available in the OpenStack Manila youtube channel [1]. Here is a brief summary of what we discussed and the action items: *# Caracal Retrospective:* *Highlights:* Great internship rounds (manila-ui, docs, SDK and bandit testing). SQLAlchemy 2.0 migration is close to completion and we are planning to backport it. Doc bug squash was a success. We managed to close almost 70% of the bugs we signed up for. FFEs are allowing little time for testing and we are willing to take some different actions. *AIs:* - Schedule a mid cycle check-in with our PTG etherpad to ensure all the topics we brought up are being addressed. - Collaborative review sessions now should be proposed alongside the feature when submitting the features to gerrit. - Bug squashes and hackathons will be announced earlier, so we can try to have more traction. *# Enhance PoolWeigher to respect current share network* The PoolWeigher doesn't take into consideration the share server per share network allocation. As a result, the weigher will take one of these approaches: - Stack: create all shares in a pool even though a newly added pool is available until it is full - Spread: create shares in the pool with fewer shares Instead of looking if there is already any share server on the pool, weigh according to if there is already any share server of the same share network on the pool *AIs:* - carthaca will create a launchpad blueprint with these notes for permanence. - Enhance the poolweigher to consider the share network in the request. *# NetApp Snaplock/WORM feature* An RFE has been filed for supporting WORM shares [2] Retention time should be configurable. We can implement such things through extra specs or new APIs for such shares. *AIs*: - NetApp/SAP will ask around if there are other vendors that have the same feature. - If they don't, we should take the extra spec route and add the NetApp specific metadata keys. *# Cross-service-request-id + Enhancements to logging* We need to enable x-openstack-request-id, and use it when making cross service requests (to cinder/glance/neutron) for better tracking in the cross-service logs. We could have an implementation in Manila similar to what is available in Nova and Cinder [3]. This could become an internship theme. *AIs:* - carthaca will report a bug to document this request *# Backend driver operations based on resource metadata update API* Metadata currently is being used as tags to refer to db resources and when it is updated, we don't check if the share back end needs to update anything in the share. The backends should be notified of metadata modifications in case there is need to update something in the shares. Where metadata changes should be reflected to Backend drivers: snapshot policy, show-mount option *AIs:* - kpdev will document the proposed new behavior into a specification *# Technical Debt - SRBAC* gouthamr talked us through all of the work that is complete and what are the next steps in the next few cycles. *AIs:* - Increase test coverage. - Set "[oslo_policy]/enforce_new_defaults=True" as a default and see if it works. - Introduce the manager role for reset-state, promote out of sync replicas, force-delete resources. *# Retrospective on internship projects* We discussed what we have been working on with the ~10 interns we had in Manila at the end of the Caracal cycle, as well as our plans for Dalmatian There are also some mentoring opportunities in case people are interested. Please reach out :) *# All things CephFS* ashrodri talked us through the updates from the previous cycle and all the testing she has been doing with CI and enhancing the CI jobs, as well as introducing the ingress daemon service and so on. We are hitting some package issues for testing, and we are trying to figure those out. On new features we are planning: *- Manage/unmanage implementation in the CephFS driver* - This is already available to other drivers but we need to implement this to the CephFS drivers. We expect to be able to start on this work during the Dalmatian cycle*.* *- **Impact of the new ensure shares API in the CephFS driver* - Ensure shares comes from a request from CERN, while they had their Ceph Mon IPs changing but they needed to restart the services to reconstruct the export locations. - It ended up becoming a feature that will be used by many people and we are trying to make this as generic as possible, as it can also be useful for backend migrations. - A spec is being worked on by carloss [4] and we expect to complete this work during the Dalmatian cycle. *- DHSS=True implementation with the CephFS driver*. - We received several requests to have DHSS=True available for the CephFS driver and we are willing to start on this work after another request showed up [5]. - *AI*: gouthamr will propose a spec to document how it will work. *- NFSv3 known issue* - NFSv3 has a large number of deficiencies that NFS-Ganesha can't fill - there are external services like rpcbind, statd that handle portions of the NFS protocol that are natively supported in NFSv4.1+ - However, Microsoft Windows doesn't have a native NFSv4.1+ client; which means Windows guests can't mount CephFS/NFS shares - until now. - With Ceph Reef, the NFS service enables v4 and v3 export rules; which means a share can be mounted with NFS v3 or NFS v4.1+ - It isn't advised to mount with both protocols at the same time, nor is it advised to use NFSv3 outside of windows - because there's no recovery in case there's a failover in the NFS server *- Async mirroring* - This feature will use cephfs mirroring [6], which sounds like share replication but it has some differences. - It works similarly to the way it was designed in Cinder [7]. - *AI*: ashrodri will work on a blueprint to track this effort. *# Backup APIs* Backups are available for some releases, and NetApp + SAP are planning some enhancements and are willing to include a new backup type entity in Manila, similar to share types We agreed that the backup type properties should reuse the metadata mechanism to reuse code and UX, so we will end up also having backup type metadata. Migrations can be a concern, as the backup types were configuration blocks in the manila.conf file, so we will rely on administrators to properly migrate that, or even provide a script to do such migration. *AIs*: - NetApp will work on a specification for the backup updates *# Backup driver implementation with CBACK* CERN is working on adding a driver that integrates with their backup solution (CBACK) to manila, which uses Ceph storages as the backups backend. Zach showed us some findings during the implementation and talked about his plans and some challenges. The plan is to introduce the new backup driver during the Dalmatian cycle and also start testing on CI with it. *# Bug management stats and cycle reports* vhari shared the bug stats after the previous cycle and how encouraging they are. We reflected on the data and had a suggestion to expand our bugsquash events to two in a cycle, so that can help us reduce the bug backlog quicker. We had great stats on reducing doc bugs backlog with the help of a bugsquash late in the previous cycle. We also talked about possible role rotation and discussed what it means to be the Manila bug representative. *# Provide support for efficiency policy on NetApp shares* NetApp is figuring out a way to determine efficiency policies during the creation of shares individually. They want to implement this by introducing new share type extra specs to configure such things. DHSS=True is still being discussed, and this feature will be worked on for DHSS=False first. NetApp intends to implement this feature during the Dalmatian cycle. *# Technical Debt - sqlalchemy 2.0* Thanks to stephenfin and zzzek, all changes for sqlalchemy 2.0 were merged in the beginning of the Dalmatian release, as some issues showed up in the process and we couldn't figure them out. We will be backporting the changes that didn't make it to Caracal, so sqlalchemy 2.0 will also be supported by Manila. There are no current plans for backporting this further. We also need to evaluate removing the backref to back_populates. *AIs:* - Complete the backport of the changes to 2024.1 caracal *# Add response schema validation* The SDK team would like to generate OpenAPI schemas stored in-tree, to ensure things are complete and up-to-date, and to avoid landing another large deliverable on the SDK team, and to allow service teams to fix their own issues. OpenAPI 3.1 is a superset of JSONSchema, which means we can use the same tooling we currently use for this. *AIs*: - stephenfin will work on a spec and on bootstraping it, and more people would be able to pick it up. *# Shares encryption* We had a brief discussion about the design of this feature for Manila, which the spec is already merged. kpdev has already proposed two patches for this change [8] *AIs:* schedule a collaborative review early in the cycle so we can start reviewing it as early as possible. *# DB migration script squash * tkajinam brough to our attention during the previous cycle that there was room for enhancement in our migration scripts, as there are 70+ of them now and we could squash them. We are also considering automated ways to do this with sqlalchemy. *AIs*: - gouthamr/tkajinam will reach out to zzzek and stephenfin and ask if there is an automated way to make this process easier. [0] https://etherpad.opendev.org/p/dalmatian-ptg-manila [1] https://www.youtube.com/@openstackmanila/ [2] https://bugs.launchpad.net/manila/+bug/2060027 [3] https://blueprints.launchpad.net/nova/+spec/cross-service-request-id [4] https://review.opendev.org/c/openstack/manila-specs/+/915480 [5] https://github.com/nfs-ganesha/nfs-ganesha/issues/1087 [6] https://docs.ceph.com/en/latest/dev/cephfs-mirroring/ [7] https://docs.openstack.org/cinder/latest/contributor/replication.html [8] https://review.opendev.org/q/topic:%22bp/share-encryption%22 If you have questions, please feel free to reach out. Looking forward to a very productive release cycle with you! Regards, carloss

1 0

[kolla-ansible][Antelope][Heat] Heat UpdateWaitConditionHandle Issues
by Devin Patterson 17 Apr '24

17 Apr '24

Hello, We are creating a workflow to use Heat auto-scaling groups with wait conditions. The ideal goal would be to have the wait condition clear and require another signal when the Heat stack is updated. Trying to use the resource OS::Heat::UpdateWaitConditionHandle instead of OS::Heat::WaitConditionHandle. This works during initial creation, but when I udpdate the stack with a new configuration, I get the following error. UpdateWaitConditionHandle.update() got an unexpected keyword argument 'new_template_id' Any ideas on how to use wait conditions during stack updates or to fix the above error. Thanks in advance. Devin

2 2

[Glance] Dalmatian PTG Summary
by Pranali Deore 17 Apr '24

17 Apr '24

Hello Everyone, We had our Dalmatian virtual PTG between 9th APRIL to 12th APRIL 2024. Thanks to everyone who joined the virtual PTG sessions. Using Opendev Meetpad we had all the discussion around different topics for glance. Here is the short summary of the discussions. Tuesday , APRIL 9th 2024 # Caracal Retrospective On a positive note we have completed few important things like adding support of new centralized_db cache driver which will be enabled default for caching related operations and stop using sqlite database from caracal release onwards and deprecated Sqlite cache driver, we also have deprecated glance-cache-manage command in favor of the new Cache API, glance scrubber utility and its associated configuration options, location_strategy option since in Bobcat, we added new weighing mechanism which made the location strategy no longer required and continued working on adding new location API support which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. On the other side we decided to speed up the work we've had in flight and speed up the reviews. # Add missing CLI support for some Glance API in Openstack Client The migration work for adding missing CLI was initiated in the antelope cycle. We have added the SDK support for cache API , images API and metadef API There are image-create-via-import, image-upload, member commands & metadef tags patches are under review. This cycle we are hoping to get all the above mentioned patches merged and mark this work complete. # Introduce new sub-store NFS for filesystem store Currently in glance when we configure nfs store, we do not store any information about what server nfs is configured, what mount point is configured, if it is available or not. From the beginning nfs is treated as a local FS store in glance. So if the NFS server is down , we don't have any idea about that, we write the image to the local FS store and it doesn't get reflected to the nfs store if nfs is down. We decided to have this support similar to cinder, when it configures nfs backend it checks at the initial level whether mount point is available or not and again checks during volume upload whether connected to nfs mount point We will enhance file system store by introducing some config parameters to store this information by creating separate ini file which can have information related to nfs server store with which we can check nfs server is reachable or not with the use of os-brick library. # Driver test coverage Currently glance has CI coverage for 'cinder', 'rbd', 'swift' stores and functional tests for FS store. Vmware driver is deprecated in 2024.1 but we decided to keep it and revise the status after updating the existing thread with the outcome in manila ptg. For S3 driver, we decided to ask for any interest in maintaining s3 driver (with adding CI, fixing bugs, etc) and depending on the response we will go ahead and add the coverage. # DB Migration script cleanup Glance has been using alembic migrations on the upgrading database schema from old version to the new version, currently glance maintains migration scripts from liberty and upgraded as we moved ahead cycle wise. So we decided to squash some of these[1] until the oldest release not EOLed and which is wallaby currently and decided to document this and from now onwards we will support migration scripts post EOL cycle and scrub the migration once the cycle is EOL. Wednesday April 10th , 2024 # Migrate existing images to new store at once Glance currently supports copying existing images to another available store via import workflow but one image at a time. If the deployer wants to desommision the existing store and add a new one, then all the images need to be copied to the new store and delete those one by one from the existing one. In onder to solve this problem we decided to provide glance-manage command line utility by introducing new operation copy-images to copy all or selective images in newly added store and if we add a flag 'delete_from_source' and set it to true we can delete all the images from the source store Example: glance-manage copy_images --source-store store_1 --destination-store store_2 --delete_from_source After all images are copied we will print the stat on the console - copied images, deleted images, ignored- images ( if user deletes any image in between the copy operation then, we will raise and catch the NotFound exception and add it to the ignored stores. Admin can check the ignored images issue and incase it's present then the process can be reinitiated) # Introduce common configuration option(s) at base store level which can be used/accessible as a common for all available stores? Since glance supports separate configuration options for each store incase of multi store enabled with the same store type. So we decided to introduce common configuration options at base store level which can be used/accessible as a common for all available stores similar to cinder [2] # Generic Image migration + Optimization for same store migration Currently the preferred way of migration in glance is two step 1. Copy the image from source store to destination store 2. delete the image from source store With the above approach there are 2 problems: 1. requires manual intervention from operators after waiting for image copy to finish and then delete the image from source store 2. No way for stores to optimize the operation We decided to address it by introducing a migration operation in glance which will have two features [3] 1. A generic migration workflow where we will perform the image copy and delete in the same API operation 2. Allow an interface for glance store methods to optimize the migration if possible else it will fall back to the generic workflow # New Location APIs We have decided to have support of new location apis which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. The implementation patches are up for review along with the unit/functional tests, documentation, api version bump, client & sdk side support and tempest coverage[2] The Poc patches of nova and cinder are also there to test this new location api functionality. During last cycle we have noticed few issues with cinder store while testing the cinder side POC patches, so the glance_store & cinder side fix is up for review, and once those patches are merged and glance_store is released, we can merged the New Location API patches after getting approval from Nova & Cinder side since both are the consumers of these 2 new APIs. We also decided to have a new CI job where all newly added tempest tests for these new APIs are passed and add command line utility to re-calculate the hash of the active images which are without hash if hash calculation fails during location-import. Thursday April 11th , 2024 # Add support to test whether swift is deployed or not We have noticed that when glance is configured to use swift as a glance store and swift service is not enabled/deployed/down then glance api service starts but image creation fails and image will remain in importing state forever. In case of import we upload the data at staging store first and then we try to import it to swift store, at that time We decided to add the support at the store side to check at the startup if swift is up and running and if it's not running then log a warning and disabled the store for adding & delete opearetion. # Deprecate and remove 'metadata_encryption_key' and related functionality 'metadata_encryption_key' config option and its related functionality was added quite a long back ago and even though as per the description it encrypts the location metadata, it actually encrypts location url. It encrypts the location url only for image upload/import/download/show APIs, doesn't encrypt url on location APIs. If it gets enabled during upgrade then it will break the existing deployment since existing image urls are not being encrypted. Hence we decided to deprecate it in this cycle and if there is no objection seen then we will remove it in the next cycle or may this the current cycle as well. # Shelved instance image protection This is an Optional feature proposal to prevent deletion of shelved instance's image. Currently when an instance is shelved its image can be removed and afterwards an instance cannot be booted again. Glance team has suggested checking the implementation of 'cinder_encryption_key_deletion_policy' and check whether it is possible to implement on a similar count and update the existing spec[4] accordingly. Apart from above topics you can find other miscellaneous topics discussed in PTG etherpad [5] and the recordings of all the sessions and milestone wise priorities at the end. If you have any questions/suggestions then please join us in our weekly meeting(each Thursday on #openstack-meeting irc channel at 1400 UTC) [1]: https://review.opendev.org/q/topic:%22db-migration-squash%22 [2]: https://opendev.org/openstack/cinder/src/commit/935f08e032b31d4628881d3a053… [3] : https://review.opendev.org/c/openstack/glance-specs/+/914639 [4]: https://review.opendev.org/c/openstack/glance-specs/+/915475/1/specs/2024.2… [5]: https://etherpad.opendev.org/p/apr2024-ptg-glance#L1

1 0

[ironic][PTG] Ironic 2024.2 PTG summary
by Riccardo Pittau 16 Apr '24

16 Apr '24

Hello Ironicers! Here's a summary of the discussions around the major topics at the Ironic PTG 2024.2 # Caracal retrospective # We took our time to look back at the Caracal development cycle, how it was for the ironic project, celebrate what we've accomplished, and of course also understand if some of the work started during Caracal needed to be continued in Dalmatian. We're quite happy to say that most of the goals we've set for Caracal have been reached, and we've actually been able to do much more than planned! For example we delivered the HttpBoot support for Redfish, we've been able to tackle a lot of old bugs and keep pace with the migration from storyboard to launchpad, almost finished the merge of Inspector into Ironic, and marked multiple old drivers for removal. # Present and Future of Ironic # We've analyzed the current status of Ironic and asked ourselves: "Where do we want Ironic to go?" In the past years we started looking beyond Openstack and we've took some steps to get into that direction, and we can say that Ironic went over the Openstack borders, becoming more independent. Currently Ironic can be used in three ways: - Openstack Integrated, the more clear and established way - as the core engine of Metal3, with good documentation for various use cases, but still growing and with margin for improvement - standalone, which still requires some work, a much clearer path forward and requirements We really want to keep investing in standalone Ironic more in the future, but to do that we need to take some decisions based on the vision we have on server provisioning from here to 5 years. Lots of questions we asked ourselves on how to resolve some problems connected to having a standalone bare metal provisioning service, mostly related to networking (e.g. secureboot, virtualmedia), will expand and find possible answers in other topics. # Redfish topics # Talking about present and future, Redfish is the current constantly evolving and improving standard protocol to directly communicate with controllers of bare metal devices. Ironic supports it since a while and of course there's more and more demand for new features to support or to expand the current capabilities. The discussion was mainly around an alternative, and possibly easier way to provide images for firmware updates and virtual media boot using a "push" model instead of the current "pull", with two proposals: - using the UpdateService already present in the Redfish standard - using the Virtual Media Push as a new feature currently under discussion with DMTF Another major topic is the support for TLS connection to the BMC, which already has a design proposal from one major contributor, Fujitsu. In addition we'll also have to explore the "other side" of the connection, so from the BMC to the ironic services, which will be one of the goals for Dalmatian. # CI related topics# CI has always had great importance for the Ironic project being one of the major consumers in terms of resources. It's not a case that that first related topic was how to slim down the CI usage! During the Caracal cycle we've been able to reduce the number of jobs run by the Ironic project, but we've also added some more. We came to the conclusion that we need to take an approach where we stack more tests in fewer jobs, trying to consolidate down jobs as much as possible and minimize boot interface variation testing. During the Dalmatian cycle we'll work first on trying to have Redfish and ipmi in a single ironic job, and update the list of jobs to understand where we can avoid duplication. At some point we'll also have to make space for some more deep tests on ARM machines. We've also decided to give Codespell tests a try, and explore the possibility of having some pre-commit hooks, and, after some technical considerations, move away from Tinycore as our base for the IPA ramdisk in CI, gentoo being the most likely candidate as a replacement. # xPUs # With xPUs in this case we mean IPU and DPU (not to be confused with TPU or GPU) which are some Processing Units that are designed for specific applications and data types. Usually DPUs are embedded in edge network cards, while IPUs are specifically designed for Artificial Intelligence/Machine Learning. There are multiple challenges in order to support those devices, depending on the "vision" of the vendors, some view them having full Operating Systems, some as having an OS just as "firmware", most or all of them would like remote management, which is where Ironic can really make an impact. Even though there are multiple use cases and scenarios where the use of xPUs is highly valued, and the wishlist of potential features is quite long, the first step we need to take is to get some real hardware and start testing, which is the real challenge we'll have to deal with next. # Secure Boot # We've made huge steps forward in supporting Secure Boot, the next big one would be supporting custom keys and providing means to upload and update them. Before doing that we need to clarify some aspects of Secure Boot, especially the "security vs convenience". In the Dalmatian cycle will focus mainly on updating our documentation, provide a better picture to users, and start exploring possible solutions for the secure custom keys management. # Future of Networking # Reconnecting to the future of Ironic, networking represents the next step for a truly standalone Ironic. This means finding alternatives to Openstack-integrated scenarios and therefore to Neutron. In Metal3 we're solving the problem using a separate IPAM, but this is still not enough for a complete Ironic standalone, in the end what we really need is flat network control, which at the moment we don't really have an alternative for. We realized that the topic is vast, we've just scratched the surface and we need to invest much more time on it, having more discussions to go deeper into it and start researching alternatives and tools for our goal. That is (almost) all folks! You can find all the topics and more details in the discussions notes here: https://etherpad.opendev.org/p/ironic-ptg-april-202 Feel free to answer directly in this thread in case you have questions or comments, or contact the ironic team on IRC. Thanks to everyone who participated in the PTG! Have a great Dalmatian development cycle! Ciao Riccardo

1 0

[publiccloud-sig] Reminder - next meeting April 17th - 0700 UTC
by Felix Kronlage-Dammers 16 Apr '24

16 Apr '24

Hi everyone, This Wednesday (April 17th) the next meeting of the public cloud sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG See you tomorrow! felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[all] ORM apocalypse mostly averted (a.k.a. SQLAlchemy 2.0 is here)
by Stephen Finucane 16 Apr '24

16 Apr '24

o/ A whole 2 years and 8 months to the day after I first communicated about it [1], it would appear the requirements bump to SQLAlchemy 2.0 [2] is finally ready to go. I suspect most if not all service projects are familiar with SQLAlchemy 2.0 at this point, having needed to fix whatever compatibility issues were seen. For those that aren't, I suggest reading my previous emails on the matter. So what happens next? Hopefully for the vast majority of projects the answer should be "nothing": many projects now have sqlalchemy-tips jobs that are designed to run unit and/or integration tests with the latest versions of SQLAlchemy, Alembic and oslo.db so the switchover should be seamless. I foresee three situations: * For projects that continue working as-is post-bump, you may wish to delete the sqlalchemy-tips jobs now. The jobs were designed as a stop gap measure to prevent projects regressing on SQLAlchemy 2.0 support while we got other projects over the line. While there is some limited value in keeping these jobs around, they do consume resources and - to the best of my knowledge - the SQLAlchemy community is not planning to release a 3.0 version of SQLAlchemy (or 2.0 version of Alembic) any time soon :) The requirements jobs gates on many projects and should identify compatibility issues with these libraries going forward, so it's probably best to drop them from the projects. I'll push a couple of patches shortly to do just this. Feel free to accept or reject them as you see fit. * For projects that are supposed to be SQLAlchemy 2.0 Ready (TM) but start seeing issues that the sqlalchemy-tips jobs didn't catch (because, say, the job was running unit tests rather than functional tests and the unit tests are missing coverage), I'd encourage you to look through what is now a fairly extensive repository of patches [3] that we've built up while addressing SQLAlchemy 2.0 compatibility issues. These are mostly all tagged with the sqlalchemy-20 topic and there's a very good chance that a patch addressing your specific issue has already been submitted to another project. * For projects that were not in the gating queue for requirements bumps, have not prepped for SQLAlchemy, and start failing catastrophically post-bump: have you not been listening to me?! :) You've got some work to do but fortunately that work should be relatively simple (if a bit tedious). As noted above, there are a *lot* of patches that you can look through to identify solutions to your compatibility issues. My preferred strategy - which is the one also advocated for by the SQLAlchemy 2.x docs [4] - has been to configure the warnings filter to raise an error for all SQLAlchemy deprecation warnings then filter out those that we actually hit and iteratively work to remove these filters. That won't work now that SQLAlchemy 2.0 is in upper-constraints so you're going to have to fix it all in one go. You may wish to adopt the standard approach locally though, before squashing all the commits upon review. This has already consumed far (far!) too much of my time so I won't offer to undertake the migration of any outstanding projects. I am, however, happy to field questions and point to existing patches that resolve a given issue. Hopefully none of it will be necessary though. Cheers, Stephen [1] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.… [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20 [4] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html

11 13

[openstack-helm] 2024.2 PTG summary
by Vladimir Kozhukalov 16 Apr '24

16 Apr '24

Dear Openstack-helmers, Here is the short summary of our discussion during PTG 2024.2. If you would like to further discuss some of these topics or have any other questions you are welcome to continue the discussion either in this mailing thread or start a new one. ## Migration from legacy Ceph OSH charts to Rook During 2024.1 we developed the migration procedure which seems to work well at least for some configurations. At the moment we have a doc [1] that describes the procedure and also there is a script [2] which can be used as an example. The procedure is obviously quite dangerous and we decided to put some efforts to create a test job for this. Anyway, at this point it is recommended to use Rook for managing Ceph clusters. Once we have the migration test job and we are sure it works well enough we will officially deprecate the legacy Ceph charts with the deprecation period of one or two release cycles. ## MariaDB operator At the moment the OSH MariaDB chart [3] is recommended by default for managing MariaDB clusters. We are not yet ready to switch to the MariaDB-operator [4] (and the mariadb-cluster OSH chart [5]) by default for our test jobs. We will continue supporting MariaDB-operator test job and will try to work closer with the operator community. Ideally, we would like the MariaDB-operator project to provide their own Helm chart which we could utilize. ## OVN Our OVN test deployment jobs seem to work well and we can recommend using OVN as the default Openstack networking backend. However, migrating from OVS to OVN is not trivial and assumes addressing at least the following two points: - Currently we don't have the migration procedure like, for example, this [6], [7] in the context of OSH. It would be great to have a volunteer to contribute this. - Some of our users have OVS customizations which seem not to have 1-1 feature parity with OVN. So, even if we have the migration strategy, the feature alignment is going to take some time. We agreed to continue supporting OVS and OVN jobs 50/50 in this release cycle. Also it seems to worth it to initiate a wider discussion about the migration itself and about the timeline. ## Tacker documentation and testing During 2023.2 we added the Tacker chart [8] but it is not mentioned in the user deployment documetation. We agreed that the Tacker team will contribute the Tacker and the Barbican sections to the OSH deployment documentation (Barbican is needed for Tacker). There is a deployment test job for Tacker [9] although it is not currently a part of the check pipeline. We agreed to improve it so it not only deploys the Tacker project but also it should run some test scenarios against it. ## Third party RabbitMQ chart Our long term strategy is to focus more on Openstack charts and to use third parties for manging backend infrastructure wherever possible. So it looks like it worth it to do research regarding possible usage of the RabbitMQ operator [10]. We agreed that we are going to put efforts during 2024.2 release cycle to give it a try and probably to create a test job that will utilize this operator. ## Improve end user experience Most of our users consume the OSH charts as part of their CI/CD infrastructure and consider OSH as a Openstack managing framework rather than end-to-end solution. However, for the new users who would like to try the Openstack running on top of K8s with the minimal efforts it is important to have an experience similar to what they usually have with other Helm charts. This assumes the following - a user has a K8s cluster (possibly with some prerequisites) and the K8s API endpoint - a user has kubectl/helm command line tools configured (to connect to K8s API endpoint) and ready to use (probably from a remote host) - a user is not supposed to clone any git repositories to deploy the Openstack - a user is able to consume Helm charts published on a Helm repo During 2024.1 we have been working to imporove the deploy-env Ansible role to support single/multi-node environments, to make it possible to run kubectl/helm CLI tools from a remote node, to expose Openstack services via Metallb, etc. At this point we are ready to have a test job that will imitate the end user behavior with the only exception that for test jobs we heavily rely on two scripts `get-values-overrides.sh` [11] and `wait-for-pods.sh` [12]. They are not absolutely necessary but still users can benefit from using them and it looks like we can easily move this functionality to a Helm plugin. So, eventually the deployment can be performed with minimal efforts using pre-published charts. ## Widen the range of K8s configurations used for Openstack deployment Now all the test jobs are run with the same K8s configuration (Calico as a CNI, Ingress-Nginx as a ingress controller) and we are trying to stay as less K8s details dependent as possible. At the same time we could probably create test jobs that will exercise some other K8s configurations like for example - Cilium/Kube-Router as a CNI implementation - HAProxy as an ingress controller This seems to be minor, but still volunteers are welcome to contribute. [1] https://docs.openstack.org/openstack-helm/latest/troubleshooting/migrate-ce… [2] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… [3] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/mariadb [4] https://github.com/mariadb-operator/mariadb-operator [5] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/mariad… [6] https://docs.openstack.org/neutron/latest/ovn/migration.html [7] https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration [8] https://opendev.org/openstack/openstack-helm/src/branch/master/tacker [9] https://opendev.org/openstack/openstack-helm/src/branch/master/zuul.d/base.… [10] https://www.rabbitmq.com/kubernetes/operator/install-operator [11] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… [12] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/… -- Best regards, Kozhukalov Vladimir

1 0

[tc] Next Meeting of Technical Committee Tomorrow, April 16
by Jay Faulkner 16 Apr '24

16 Apr '24

Hi all, The next scheduled meeting of the OpenStack Technical Committee is Tomorrow, Tuesday April 16 at 1800 UTC in #openstack-tc on OFTC. The proposed agenda is: * Roll call * Follow up on tracked action items ** gmann to add grenade skip level job simplification topic in QA PTG ** gmann will send an email to the openstack-discuss list to cancel next week's TC IRC meeting * Gate health check * TC vPTG 2024.2 Follow-ups ** https://etherpad.opendev.org/p/apr2024-ptg-os-tc * TC Chair Election ** https://governance.openstack.org/tc/reference/charter.html#tc-chair * Open Discussion and Reviews See you there! -- Jay Faulkner OpenStack TC Chair

1 0

Red hat Configuration with Ceph(Cephadm tool on ubuntu
by Adivya Singh 15 Apr '24

15 Apr '24

Hi All, Regarding red hat open stack integration with ubuntu ceph installed with ceph adm tool Can you please guide me on this, What file changes need to be in Storage -config, Storage Environment for a Deployer script to pick it up regarding Cinder integration with Ceph Regards Adivya Singh

3 7

[neutron] Bug deputy report (week starting on April 8)
by Bernard Cafarelli 15 Apr '24

15 Apr '24

Hey neutrinos, here is the list of bugs filled during the PTG week - not too long as apparently people were busy attending the PTG sessions! The first 2 bugs are currently unassigned Medium * OVN DNS not working as documented - https://bugs.launchpad.net/neutron/+bug/2059405 Needs further triaging * [OVN] Changing public network MTU does not update router port gateway_mtu option - https://bugs.launchpad.net/neutron/+bug/2060828 Unassigned, reproducer described in bug Low * [ML2][OVS] more precise flow table cleaning - https://bugs.launchpad.net/neutron/+bug/2060587 Patch in progress https://review.opendev.org/c/openstack/neutron/+/915302 Opened RFE * Add 'trusted_vif' field to the port attributes - https://bugs.launchpad.net/neutron/+bug/2060916/ Opinion * already associated floating ip can be associated to another server without check or warning - https://bugs.launchpad.net/neutron/+bug/2060808 * removing floating ip from server does not check the server - https://bugs.launchpad.net/neutron/+bug/2060812 Linked bugs (one for removal, one for reassigning), discussed last week in PTG, waiting on nova team opinion Needs more info * neutron-dhcp-agent attemps to read pid.haproxy but can't - https://bugs.launchpad.net/neutron/+bug/2060974 Discussed on IRC last week, waiting for confirmation but this should soon be closed -- Bernard Cafarelli

1 0

[release] Release countdown for week R-24
by Thierry Carrez 12 Apr '24

12 Apr '24

Welcome back to the release countdown emails! These will be sent at major points in the 2024.2 "Dalmatian" development cycle, which should conclude with a final release on October 2nd, 2024. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the 2024.2 "Dalmatian" development cycle, assessing Dalmatian community goals and approving Dalmatian specs. General Information ------------------- The Dalmatian cycle is of pretty standard length, 26 weeks. In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/dalmatian/schedule.html By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… By default, all your team deliverables from the 2024.1 "Caracal" release are continued in 2024.2 "Dalmatian" with a similar release model. Upcoming Deadlines & Dates -------------------------- Dalmatian-1 milestone: May 16 -- Thierry Carrez (ttx)

1 0

[cinder][vmt] updating the cinder core security contacts team
by Brian Rosmaita 12 Apr '24

12 Apr '24

Greetings to all people interested in cinder security, The “Cinder Core security contacts” team [0] is a proper subset of the cinder core team who are notified when the OpenStack VMT assigns a private security bug to cinder following the vulnerability management process [1]. The VMT prefers that this group be kept small, so often adding someone entails removing someone else. I propose the following changes: add: - Jon Bernard (current cinder core, and Cinder PTL for Dalmatian) remove: the following people, who have not worked on security bugs over the past few development cycles: - Ivan Kolodyazhny - Sean McGinnis - Walt Boring The Cinder project team thanks Ivan, Sean, and Walt for their past security service to the project. If you have any comments or concerns, please reply on the mailing list before 12:00 UTC Friday 12 April 2024. cheers, brian [0] https://launchpad.net/~cinder-coresec/+members [1] https://security.openstack.org/vmt-process.html

3 3

[openstack][kolla-ansible]
by Nguyễn Hữu Khôi 12 Apr '24

12 Apr '24

Hello. I am testing a case when I lost all my openstack cluster. I just have a full db backup and volumes(my storage node still alive). I deploy a new cluster with same node's name after that I recover db from backup and the cluster is up but I cannot see my instances. When I click on instance tab or volume tab, I see errors: [image: image.png] I hope you will share experiences. Nguyen Huu Khoi

1 2

[openstacksdk][older pythons] Reverting a small mypy change in openstackapi
by Michael Still 12 Apr '24

12 Apr '24

Hi, I discovered the issue fixed by https://review.opendev.org/c/openstack/openstacksdk/+/915199 accidentally while rebased Kerbside patches, and I am wondering who I should ask nicely for a review. Its a relatively small change, but does regress mypy support in return for working correctly on python versions below python 3.10, which we promise to support right now. Thanks, Michael

1 0

[Glance] Dalmatian vPTG Updates
by Pranali Deore 12 Apr '24

12 Apr '24

Hello Everyone, We have concluded Glance Dalmatian vPTG yesterday. I will share the summary early next week. You can find tentative milestone wise priorities for glance in PTG etherpad[1]. Join us in the weekly meeting on Thursday(1400 UTC) if you have any doubts/suggestions. Thanks & Regards, Pranali Deore [1: https://etherpad.opendev.org/p/apr2024-ptg-glance#L390

1 0

[i18n][PTG] I18n SIG 2024.2 (Dalmatian) PTG Planning
by Ian Y. Choi 12 Apr '24

12 Apr '24

Hi, PTG is happening this week. @seongsoo and I have blocked time slots (Thu 21-22 UTC & Fri 14-16 UTC). Anyone can check via https://ptg.opendev.org/ptg.html and please let me know if more discussion is needed during PTG. Those are topics what I am thinking currently but it is open - feel free to suggest any topic to be discussed: - Moving forward on open reviews for i18n: at least, there are several pending items on openstack/i18n repo - openstack/i18n repository structure discussion to embrace the use of weblate - wlc library on tools/weblate and others including the addition of tox environment and corresponding requirements.txt - Pilot work progress discussion on governance translation support on weblate platform - Overall feasible migration plan discussion to sunset translate.openstack.org earlier than zanata.translate.org Discussions will be shared via Etherpad: https://etherpad.opendev.org/p/apr2024-ptg-i18n . Lastly, please don't forget to register if you have forgotten: https://ptg2024.openinfra.dev/ :) Looking forward to a productive PTG with I18n. Thank you, /Ian

2 1