Hello Everyone,
We had our Dalmatian virtual PTG between 9th APRIL to 12th APRIL 2024. Thanks to everyone who joined the virtual PTG sessions. Using Opendev Meetpad we had all the discussion around different topics for glance.
Here is the short summary of the discussions.
Tuesday , APRIL 9th 2024
# Caracal Retrospective On a positive note we have completed few important things like adding support of new centralized_db cache driver which will be enabled default for caching related operations and stop using sqlite database from caracal release onwards and deprecated Sqlite cache driver, we also have deprecated glance-cache-manage command in favor of the new Cache API, glance scrubber utility and its associated configuration options, location_strategy option since in Bobcat, we added new weighing mechanism which made the location strategy no longer required and continued working on adding new location API support which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. On the other side we decided to speed up the work we've had in flight and speed up the reviews.
# Add missing CLI support for some Glance API in Openstack Client The migration work for adding missing CLI was initiated in the antelope cycle. We have added the SDK support for cache API , images API and metadef API There are image-create-via-import, image-upload, member commands & metadef tags patches are under review. This cycle we are hoping to get all the above mentioned patches merged and mark this work complete.
# Introduce new sub-store NFS for filesystem store Currently in glance when we configure nfs store, we do not store any information about what server nfs is configured, what mount point is configured, if it is available or not. From the beginning nfs is treated as a local FS store in glance. So if the NFS server is down , we don't have any idea about that, we write the image to the local FS store and it doesn't get reflected to the nfs store if nfs is down. We decided to have this support similar to cinder, when it configures nfs backend it checks at the initial level whether mount point is available or not and again checks during volume upload whether connected to nfs mount point We will enhance file system store by introducing some config parameters to store this information by creating separate ini file which can have information related to nfs server store with which we can check nfs server is reachable or not with the use of os-brick library.
# Driver test coverage Currently glance has CI coverage for 'cinder', 'rbd', 'swift' stores and functional tests for FS store. Vmware driver is deprecated in 2024.1 but we decided to keep it and revise the status after updating the existing thread with the outcome in manila ptg. For S3 driver, we decided to ask for any interest in maintaining s3 driver (with adding CI, fixing bugs, etc) and depending on the response we will go ahead and add the coverage.
# DB Migration script cleanup Glance has been using alembic migrations on the upgrading database schema from old version to the new version, currently glance maintains migration scripts from liberty and upgraded as we moved ahead cycle wise. So we decided to squash some of these[1] until the oldest release not EOLed and which is wallaby currently and decided to document this and from now onwards we will support migration scripts post EOL cycle and scrub the migration once the cycle is EOL.
Wednesday April 10th , 2024
# Migrate existing images to new store at once Glance currently supports copying existing images to another available store via import workflow but one image at a time. If the deployer wants to desommision the existing store and add a new one, then all the images need to be copied to the new store and delete those one by one from the existing one. In onder to solve this problem we decided to provide glance-manage command line utility by introducing new operation copy-images to copy all or selective images in newly added store and if we add a flag 'delete_from_source' and set it to true we can delete all the images from the source store
Example: glance-manage copy_images --source-store store_1 --destination-store store_2 --delete_from_source
After all images are copied we will print the stat on the console - copied images, deleted images, ignored- images ( if user deletes any image in between the copy operation then, we will raise and catch the NotFound exception and add it to the ignored stores. Admin can check the ignored images issue and incase it's present then the process can be reinitiated)
# Introduce common configuration option(s) at base store level which can be used/accessible as a common for all available stores? Since glance supports separate configuration options for each store incase of multi store enabled with the same store type. So we decided to introduce common configuration options at base store level which can be used/accessible as a common for all available stores similar to cinder [2]
# Generic Image migration + Optimization for same store migration Currently the preferred way of migration in glance is two step 1. Copy the image from source store to destination store 2. delete the image from source store With the above approach there are 2 problems: 1. requires manual intervention from operators after waiting for image copy to finish and then delete the image from source store 2. No way for stores to optimize the operation We decided to address it by introducing a migration operation in glance which will have two features [3] 1. A generic migration workflow where we will perform the image copy and delete in the same API operation 2. Allow an interface for glance store methods to optimize the migration if possible else it will fall back to the generic workflow
# New Location APIs We have decided to have support of new location apis which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. The implementation patches are up for review along with the unit/functional tests, documentation, api version bump, client & sdk side support and tempest coverage[2] The Poc patches of nova and cinder are also there to test this new location api functionality. During last cycle we have noticed few issues with cinder store while testing the cinder side POC patches, so the glance_store & cinder side fix is up for review, and once those patches are merged and glance_store is released, we can merged the New Location API patches after getting approval from Nova & Cinder side since both are the consumers of these 2 new APIs. We also decided to have a new CI job where all newly added tempest tests for these new APIs are passed and add command line utility to re-calculate the hash of the active images which are without hash if hash calculation fails during location-import.
Thursday April 11th , 2024
# Add support to test whether swift is deployed or not We have noticed that when glance is configured to use swift as a glance store and swift service is not enabled/deployed/down then glance api service starts but image creation fails and image will remain in importing state forever. In case of import we upload the data at staging store first and then we try to import it to swift store, at that time We decided to add the support at the store side to check at the startup if swift is up and running and if it's not running then log a warning and disabled the store for adding & delete opearetion.
# Deprecate and remove 'metadata_encryption_key' and related functionality 'metadata_encryption_key' config option and its related functionality was added quite a long back ago and even though as per the description it encrypts the location metadata, it actually encrypts location url. It encrypts the location url only for image upload/import/download/show APIs, doesn't encrypt url on location APIs. If it gets enabled during upgrade then it will break the existing deployment since existing image urls are not being encrypted. Hence we decided to deprecate it in this cycle and if there is no objection seen then we will remove it in the next cycle or may this the current cycle as well.
# Shelved instance image protection This is an Optional feature proposal to prevent deletion of shelved instance's image. Currently when an instance is shelved its image can be removed and afterwards an instance cannot be booted again. Glance team has suggested checking the implementation of 'cinder_encryption_key_deletion_policy' and check whether it is possible to implement on a similar count and update the existing spec[4] accordingly.
Apart from above topics you can find other miscellaneous topics discussed in PTG etherpad [5] and the recordings of all the sessions and milestone wise priorities at the end. If you have any questions/suggestions then please join us in our weekly meeting(each Thursday on #openstack-meeting irc channel at 1400 UTC)
[1]: https://review.opendev.org/q/topic:%22db-migration-squash%22 [2]: https://opendev.org/openstack/cinder/src/commit/935f08e032b31d4628881d3a053b... [3] : https://review.opendev.org/c/openstack/glance-specs/+/914639 [4]: https://review.opendev.org/c/openstack/glance-specs/+/915475/1/specs/2024.2/... [5]: https://etherpad.opendev.org/p/apr2024-ptg-glance#L1