On Wed, Jan 31, 2024 at 2:36 PM Christian Rohmann <christian.rohmann@inovex.de> wrote:
Hey all,


On Wed, Jan 31, 2024 at 3:55 AM Shrishail K <nkshrishail@gmail.com> wrote:

Hi Team,


I have a use case where I need to replicate images from file store to S3 store.

While image creation directly on S3 works fine with large files over 5G.

Since Glance moved all stores to the glance_store project, this should be the driver at work here:
https://github.com/openstack/glance_store/commits/master/glance_store/_drivers/s3.py

And there you see AWS boto3 lib is used, explicitly using multipart uploads.
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/create_multipart_upload.html

 

The same fails when attempted via image-import with copy-image method.

The failure is due to AWS rejecting a single object upload operation where the object size is over 5GB.

 

What I see from logs is that direct image creation on S3 uses multipart upload when images are large. However, I fail to see the same happening when image import was used to copy an image from file to s3.

 

Is this a known issue? Or Am I missing some configs related to image-import?


On 31.01.24 09:13, Abhishek Kekane wrote:
By design copy-image import workflow uses a common uploading mechanism for all stores, so yes it is a known limitation if it is not using multipart upload for s3 backend. Feel free to propose enhancement for the same or participate in the upcoming PTG 'April 8-12, 2024' to discuss the improvements for this behavior.
Abishek, I suppose the copy-image is done via this helper here and which are were referring to?
https://github.com/openstack/glance/blob/master/glance/async_/flows/_internal_plugins/copy_image.py

Hi Christian,

The helper you mention above is responsible to download the existing data at common storage known as staging area (configured using os_glance_staging_store in glance-api.conf) and from there it will be imported to the destination/target store. However debugging further I found that it internally calls store.add method, which means in fact it is using a particular driver call only.

I suspect [1] is where it is using single_part as an upload for s3 while copying the image, because we are not passing the size of an existing image to the import call.

I think this is driver specific improvement, and requires additional effort to make it work.

[1] https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/s3.py#L628

Thanks,

Abhishek 

Can you elaborate on how this is NOT currently able to use the glance_store drivers and the abstraction they actually provide?

So in essence this is not just about S3, but about adding support for using the glance_store drivers in the copy-image functionality.
That indeed would be a nice improvement.


Regards


Christian