Hi Massimo,Glance multiple store is still experimental and it does not have support for location API in rocky and stein, we are making it compatible in Train release.There is still one bug open for location compatibility with multiple stores and patch for it is not merged yet, I will also encourage you to deploy latest master and apply this patch in your environment and test it.NOTE:I have tested this on current master after applying patch to local environment and the scenario you mentioned is working fine.Thanks & Best Regards,Abhishek KekaneOn Tue, Aug 27, 2019 at 7:47 PM Abhishek Kekane <akekane@redhat.com> wrote:Hi Massimo,Cool, thank you for confirmation, I will try to reproduce this in my environment. It will take some time, will revert back to you once done. Meanwhile could you please try the same with single store (disabling the multiple stores)?Thank you,Abhishek--On Tue, 27 Aug 2019 at 6:33 PM, Massimo Sgaravatto <massimo.sgaravatto@gmail.com> wrote:Hi AbhishekYes, before trying the:glance location-add --url rbd://8162f291-00b6-4b40-a8b4-1981a8c09b64/images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628/snap 6bcc4eab-ed35-42dc-88bd-1d45de73b628command, I uploaded the image to ceph. And indeed I can see it:[root@ceph-mon-01 ~]# rbd info images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628
rbd image '6bcc4eab-ed35-42dc-88bd-1d45de73b628':
size 10GiB in 1280 objects
order 23 (8MiB objects)
block_name_prefix: rbd_data.b7b56aa1e4f0bc
format: 2
features: layering
flags:
create_timestamp: Tue Aug 27 11:42:32 2019
[root@ceph-mon-01 ~]# rbd snap ls images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628
SNAPID NAME SIZE TIMESTAMP
455500 snap 10GiB Tue Aug 27 11:46:37 2019
[root@ceph-mon-01 ~]#Cheers, MassimoOn Tue, Aug 27, 2019 at 2:41 PM Abhishek Kekane <akekane@redhat.com> wrote:Hi Massimo,I need to reproduce this first, but pretty much sure this issue is not because you configured multiple backends. What you are doing is downloading existing image to your local storage and then uploading it to glance using add-location operation.So before adding this using add-location operation have you manually uploaded this image to ceph/rbd? if not then how you are building your location url which you are mentioning in the add-location command? If this location is not existing then it might be a problem.Thank you,Abhishek--On Tue, 27 Aug 2019 at 5:09 PM, Massimo Sgaravatto <massimo.sgaravatto@gmail.com> wrote:I have a Rocky installation where glance is configured with multiple backends. This [*] is the relevant part of the glance configuration.I now need to dismiss the file backend and move the images-snapshots stored there to rbd.I had in mind to follow this procedure (already successfully tested with an older version of OpenStack):1) download the image from the file backend (glance image-download --file <file> <imageid>2) upload the file <file> to rbd, relying on this function: https://github.com/openstack/glance_store/blob/stable/rocky/glance_store/_drivers/rbd.py#L4453) glance location-add --url rbd://xyz <imageid>4) glance location-delete --url file://abc <imageid>I have problems with 3:# glance --debug location-add --url rbd://8162f291-00b6-4b40-a8b4-1981a8c09b64/images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628/snap 6bcc4eab-ed35-42dc-88bd-1d45de73b628......DEBUG:keystoneauth.session:PATCH call to image for https://cloud-areapd.pd.infn.it:9292/v2/images/6bcc4eab-ed35-42dc-88bd-1d45de73b628 used request id req-6c0598cc-582c-4ce7-a14c-5d1bb6ec4f14
Request returned failure status 400.
DEBUG:glanceclient.common.http:Request returned failure status 400.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/glanceclient/shell.py", line 687, in main
OpenStackImagesShell().main(argv)
File "/usr/lib/python2.7/site-packages/glanceclient/shell.py", line 591, in main
args.func(client, args)
File "/usr/lib/python2.7/site-packages/glanceclient/v2/shell.py", line 749, in do_location_add
image = gc.images.add_location(args.id, args.url, metadata)
File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 448, in add_location
response = self._send_image_update_request(image_id, add_patch)
File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", line 598, in inner
return RequestIdProxy(wrapped(*args, **kwargs))
File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line 432, in _send_image_update_request
data=json.dumps(patch_body))
File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 340, in patch
return self.request(url, 'PATCH', **kwargs)
File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 377, in request
return self._handle_response(resp)
File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", line 126, in _handle_response
raise exc.from_response(resp, resp.content)
HTTPBadRequest: 400 Bad Request: Invalid location (HTTP 400)
400 Bad Request: Invalid location (HTTP 400)As far as I can see with the "openstack" client there is not something to add/delete a location.So I guess it is necessary to change the 'direct_url' and 'locations' properties.If I try to change the direct_url property:# openstack image set --property direct_url='rbd://8162f291-00b6-4b40-a8b4-1981a8c09b64/images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628/snap' 6bcc4eab-ed35-42dc-88bd-1d45de73b628
403 Forbidden: Attribute 'direct_url' is read-only. (HTTP 403)Any hints ?Thanks, Massimo[*][default]...enabled_backends = file:file,http:http,rbd:rbd
show_image_direct_url = true
show_multiple_locations = true[glance_store]
default_backend = rbd[file]
filesystem_store_datadir = /var/lib/glance/images/
[rbd]
rbd_store_chunk_size = 8
rbd_store_ceph_conf = /etc/ceph/ceph.conf
rbd_store_user = glance-prod
rbd_store_pool = images-prodThanks & Best Regards,Abhishek KekaneThanks & Best Regards,Abhishek Kekane