On Mon, Feb 5, 2024 at 10:03 PM Shrishail K <nkshrishail@gmail.com> wrote:
Hi Shrishail,

 
Hi Folks,

Happy Monday. Just checking back if there were any updates on the logs I shared with you.
I was able to put a small hack (just to unblock my work), which determines the size of the image from the staging store(whenever image_size is set to 0 in s3 driver).
I haven't got time to look further into it, unfortunately we are approaching towards end of current development cycle and I need to looked at current tasks in hand for the same.
I will take this up in upcoming development cycle PTG for discussion. The other problem for me is I don't have active s3 setup so cannot verify the fix at my end. 

 

On the other hand, while I was playing with the image-import mechanism, I saw couple other issues,

1. It looks like the image-import command fails in CLI validation when --wait option is not provided.

root@1071c7a187a8:/muon# openstack image import --method copy-image --store file 2fa366df-d28c-43ec-b10c-00454fc8b3d8

usage: openstack image import [-h] [-f {json,shell,table,value,yaml}] [-c COLUMN] [--noindent] [--prefix PREFIX] [--max-width <integer>] [--fit-width] [--print-empty] [--method <method>] [--uri <uri>]

                              [--remote-image <REMOTE_IMAGE>] [--remote-region <REMOTE_GLANCE_REGION>] [--remote-service-interface <REMOTE_SERVICE_INTERFACE>] [--store [<STORE> ...] | --all-stores ALL_STORES] [--allow-failure]

                              [--disallow-failure] [--wait]

                              <image>

openstack image import: error: the following arguments are required: <image>


2. The --wait option when used does not really block the CLI till the import is completed. 


This bug is from openstack client and not glance client. If you use glance client command  (glance image-import or image-create-via-import) it will work since we don't have any support for --wait option.


Thanks,

Shrishail


On Thu, 1 Feb 2024 at 11:49, Shrishail K <nkshrishail@gmail.com> wrote:
[Replying-all]

Sure, it looks like setting image size enables the call to multipart upload, here is what I see 

1. With DEBUGs enabled, with no code changes, attempting to import an image of around 300MB.

2024-02-01 15:50:28.516
2024-02-01 15:50:28.323 732 DEBUG glance.async_.flows.api_image_import [-] Image c0b3654d-476f-41b7-8fcf-4eb73075bb3d os_glance_importing_to_stores= merge_store_list /var/lib/kolla/venv/lib/python3.10/site-packages/glance/async_/flows/api_image_import.py:278
2024-02-01 15:50:28.516
2024-02-01 15:50:28.288 732 DEBUG glance_store._drivers.s3 [-] Wrote 374656512 bytes to S3 key named c0b3654d-476f-41b7-8fcf-4eb73075bb3d with checksum 34ae3c2ddfe1ed0bfab30d7d6ce69f78 _add_singlepart /var/lib/kolla/venv/lib/python3.10/site-packages/glance_store/_drivers/s3.py:690
2024-02-01 15:50:28.516
2024-02-01 15:50:28.288 732 DEBUG botocore.hooks [-] Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7fda44492620>> _emit /var/lib/kolla/venv/lib/python3.10/site-packages/botocore/hooks.py:238
2024-02-01 15:50:28.516
2024-02-01 15:50:28.288 732 DEBUG botocore.retryhandler [-] No retry needed. __call__ /var/lib/kolla/venv/lib/python3.10/site-packages/botocore/retryhandler.py:211
2024-02-01 15:50:28.516
2024-02-01 15:50:28.287 732 DEBUG botocore.hooks [-] Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7fda44492560> _emit /var/lib/kolla/venv/lib/python3.10/site-packages/botocore/hooks.py:238

2. With DEBUGs enabled, with hardcoding the image size, attempting to import an image of around 300MB (with multipart configs left default)

2024-02-01 16:35:30.511
2024-02-01 16:35:30.299 731 DEBUG glance_store._drivers.s3 [-] Multipart initiate key=c0b3654d-476f-41b7-8fcf-4eb73075bb3d, UploadId=7HLARegXPw.iQVH72rcAKfGWb4GURDa.F_.tQJQgazGS8BjizAaXogifzck1Pqx6jnL7ENoEqmTEwFpNOBEdIqGb0gmmMKK51zIGmZp8jiDkpbHlb_Xh2XuOLehXGXgoUyLRk9S7kLufUnuxz4YnT69ezd08dIJX.GtrWl.6S_L9pB1Aak.hSFhuB8HfkVjH _add_multipart /var/lib/kolla/venv/lib/python3.10/site-packages/glance_store/_drivers/s3.py:721
2024-02-01 16:35:30.260
2024-02-01 16:35:30.117 731 DEBUG glance_store._drivers.s3 [-] NKSHRISHAIL: IMAGE SIZE IS ZERO... SETTING IT TO 374656512 add /var/lib/kolla/venv/lib/python3.10/site-packages/glance_store/_drivers/s3.py:631

On Thu, 1 Feb 2024 at 03:14, Abhishek Kekane <akekane@redhat.com> wrote:


On Thu, Feb 1, 2024 at 1:19 PM Christian Rohmann <christian.rohmann@inovex.de> wrote:

On 31.01.24 16:36, Abhishek Kekane wrote:

2. Following the discussion, I was getting a sense that the code changes may not be a lot, in that case, if it's possible to provide a patch,
    I can patch my openstack and see how it goes. 

Even though changes are trivial, glance uses onion architecture, so it needed changes at a lot places which will take some time. Ideally it’s not possible to provide quick fix for this.

Since we have invested some time looking into this, would it not make sense to write this up in a bug and maybe you could add some of your implementation knowledge as well?
This then seems like a fruit (be it low-hanging or not) that someone could pick.


Otherwise this bug / missing feature discussion just ends up in the ML archive.

Hi Shrishail,
CC: Christian

Is it possible for you to hardcode image_size at [1] and check not passing image_size is the real issue?
I have reported a bug [2] and mentioned it as a possible solution, but if that is not the case then we need to investigate further.



Thanks,

Abhishek

 


Regards


Christian