Re: [GLANCE][VICTORIA] - Multi-backend support with ceph backed swift.
Sure! However it is specified two times within the official glance configuration, once in glance_store and once in glance.store.swift.store but that last section is really not that well documented. https://docs.openstack.org/glance/latest/configuration/glance_api.html#glanc... Le mar. 24 mai 2022 à 17:54, Abhishek Kekane <akekane@redhat.com> a écrit :
Hi Gaël,
Can you try moving default_swift_reference from glance_store section to particular swift store section, i.e. cold as per your configs.
I guess that is the issue.
Abhishek
On Tue, 24 May 2022 at 7:30 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Hi everyone,
I'm currently enabling glance multi backend support for one of our platforms and we're facing a weird situation.
This platform is a CentOS 8 Stream based Openstack VICTORIA and when we enable the multi-backends feature, the swift store isn't working properly.
When we try to send an image to this store we end up with the following error message: *"ERROR glance_store._drivers.swift.store - A value for swift_store_auth_address is required."*
Here is the complete log trace: https://paste.opendev.org/show/bGeCJjSB1C5GwD6JYFvu/
I'm a bit surprised because according to the error message and the glance_store code it means that either my *default_swift_reference* or *auth_address *fromthe *default_swift_reference *isn't set.
https://opendev.org/openstack/glance_store/src/branch/stable/victoria/glance...
However as you can see on my glance-api.conf file I've (seems so) correctly set the glance_store directive and the store settings:
glance-api.conf: https://paste.opendev.org/show/bjFAPCjlCG6MqINGsIlK/ glance-swift.conf: https://paste.opendev.org/show/bcCfBQDj32T3qgiJj3P7/
I'm only using a single tenant glance container within the admin project. What's really weird is this issue still arise and exist with any known (to me) combination of configuration.
I've tried to declare the store right within the glance-api.conf file, but it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive instead of the auth_address one, it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive plus the auth_address one, it's not working. I've tried to replace auth_address directive with the deprecated swift_store_auth_address within the glance-swift.conf, same error and issue again.
When I replace the default_backend value with performance, uploading an image works perfectly.
I know that my swift CEPH RGW backed endpoint works as I use it already to store data. I've even tried to push an image on it both using CLI (openstack object) and cURL API call and it works. I'm a bit lost here, so if anyone is having thoughts on this I would be happy to hear you!
Thanks a lot everyone!
-- Thanks & Best Regards,
Abhishek Kekane
On Tue, 24 May 2022 at 9:34 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Sure! However it is specified two times within the official glance configuration, once in glance_store and once in glance.store.swift.store but that last section is really not that well documented.
https://docs.openstack.org/glance/latest/configuration/glance_api.html#glanc...
Ack, that needs to be fixed then, important thing is if you are using multiple stores then store specific parameters must go under specific store section otherwise they won’t get recognised. Abhishek
Le mar. 24 mai 2022 à 17:54, Abhishek Kekane <akekane@redhat.com> a écrit :
Hi Gaël,
Can you try moving default_swift_reference from glance_store section to particular swift store section, i.e. cold as per your configs.
I guess that is the issue.
Abhishek
On Tue, 24 May 2022 at 7:30 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Hi everyone,
I'm currently enabling glance multi backend support for one of our platforms and we're facing a weird situation.
This platform is a CentOS 8 Stream based Openstack VICTORIA and when we enable the multi-backends feature, the swift store isn't working properly.
When we try to send an image to this store we end up with the following error message: *"ERROR glance_store._drivers.swift.store - A value for swift_store_auth_address is required."*
Here is the complete log trace: https://paste.opendev.org/show/bGeCJjSB1C5GwD6JYFvu/
I'm a bit surprised because according to the error message and the glance_store code it means that either my *default_swift_reference* or *auth_address *fromthe *default_swift_reference *isn't set.
https://opendev.org/openstack/glance_store/src/branch/stable/victoria/glance...
However as you can see on my glance-api.conf file I've (seems so) correctly set the glance_store directive and the store settings:
glance-api.conf: https://paste.opendev.org/show/bjFAPCjlCG6MqINGsIlK/ glance-swift.conf: https://paste.opendev.org/show/bcCfBQDj32T3qgiJj3P7/
I'm only using a single tenant glance container within the admin project. What's really weird is this issue still arise and exist with any known (to me) combination of configuration.
I've tried to declare the store right within the glance-api.conf file, but it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive instead of the auth_address one, it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive plus the auth_address one, it's not working. I've tried to replace auth_address directive with the deprecated swift_store_auth_address within the glance-swift.conf, same error and issue again.
When I replace the default_backend value with performance, uploading an image works perfectly.
I know that my swift CEPH RGW backed endpoint works as I use it already to store data. I've even tried to push an image on it both using CLI (openstack object) and cURL API call and it works. I'm a bit lost here, so if anyone is having thoughts on this I would be happy to hear you!
Thanks a lot everyone!
-- Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards, Abhishek Kekane
Ok ok, I was kinda assuming that but was a bit lost by both doc and code as doc specify it multiple time within config but also within the configuring page but this time with third alternative location ^^ Will test that and let you know. Thanks a lot for this first review. Le mar. 24 mai 2022 à 18:10, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:34 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Sure! However it is specified two times within the official glance configuration, once in glance_store and once in glance.store.swift.store but that last section is really not that well documented.
https://docs.openstack.org/glance/latest/configuration/glance_api.html#glanc...
Ack, that needs to be fixed then, important thing is if you are using multiple stores then store specific parameters must go under specific store section otherwise they won’t get recognised.
Abhishek
Le mar. 24 mai 2022 à 17:54, Abhishek Kekane <akekane@redhat.com> a écrit :
Hi Gaël,
Can you try moving default_swift_reference from glance_store section to particular swift store section, i.e. cold as per your configs.
I guess that is the issue.
Abhishek
On Tue, 24 May 2022 at 7:30 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Hi everyone,
I'm currently enabling glance multi backend support for one of our platforms and we're facing a weird situation.
This platform is a CentOS 8 Stream based Openstack VICTORIA and when we enable the multi-backends feature, the swift store isn't working properly.
When we try to send an image to this store we end up with the following error message: *"ERROR glance_store._drivers.swift.store - A value for swift_store_auth_address is required."*
Here is the complete log trace: https://paste.opendev.org/show/bGeCJjSB1C5GwD6JYFvu/
I'm a bit surprised because according to the error message and the glance_store code it means that either my *default_swift_reference* or *auth_address *fromthe *default_swift_reference *isn't set.
https://opendev.org/openstack/glance_store/src/branch/stable/victoria/glance...
However as you can see on my glance-api.conf file I've (seems so) correctly set the glance_store directive and the store settings:
glance-api.conf: https://paste.opendev.org/show/bjFAPCjlCG6MqINGsIlK/ glance-swift.conf: https://paste.opendev.org/show/bcCfBQDj32T3qgiJj3P7/
I'm only using a single tenant glance container within the admin project. What's really weird is this issue still arise and exist with any known (to me) combination of configuration.
I've tried to declare the store right within the glance-api.conf file, but it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive instead of the auth_address one, it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive plus the auth_address one, it's not working. I've tried to replace auth_address directive with the deprecated swift_store_auth_address within the glance-swift.conf, same error and issue again.
When I replace the default_backend value with performance, uploading an image works perfectly.
I know that my swift CEPH RGW backed endpoint works as I use it already to store data. I've even tried to push an image on it both using CLI (openstack object) and cURL API call and it works. I'm a bit lost here, so if anyone is having thoughts on this I would be happy to hear you!
Thanks a lot everyone!
-- Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards,
Abhishek Kekane
On Tue, 24 May 2022 at 9:42 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Ok ok, I was kinda assuming that but was a bit lost by both doc and code as doc specify it multiple time within config but also within the configuring page but this time with third alternative location ^^
Will test that and let you know.
Thanks a lot for this first review.
No worries, please let me know if you have any questions. Abhishek
Le mar. 24 mai 2022 à 18:10, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:34 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Sure! However it is specified two times within the official glance configuration, once in glance_store and once in glance.store.swift.store but that last section is really not that well documented.
https://docs.openstack.org/glance/latest/configuration/glance_api.html#glanc...
Ack, that needs to be fixed then, important thing is if you are using multiple stores then store specific parameters must go under specific store section otherwise they won’t get recognised.
Abhishek
Le mar. 24 mai 2022 à 17:54, Abhishek Kekane <akekane@redhat.com> a écrit :
Hi Gaël,
Can you try moving default_swift_reference from glance_store section to particular swift store section, i.e. cold as per your configs.
I guess that is the issue.
Abhishek
On Tue, 24 May 2022 at 7:30 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Hi everyone,
I'm currently enabling glance multi backend support for one of our platforms and we're facing a weird situation.
This platform is a CentOS 8 Stream based Openstack VICTORIA and when we enable the multi-backends feature, the swift store isn't working properly.
When we try to send an image to this store we end up with the following error message: *"ERROR glance_store._drivers.swift.store - A value for swift_store_auth_address is required."*
Here is the complete log trace: https://paste.opendev.org/show/bGeCJjSB1C5GwD6JYFvu/
I'm a bit surprised because according to the error message and the glance_store code it means that either my *default_swift_reference* or *auth_address *fromthe *default_swift_reference *isn't set.
https://opendev.org/openstack/glance_store/src/branch/stable/victoria/glance...
However as you can see on my glance-api.conf file I've (seems so) correctly set the glance_store directive and the store settings:
glance-api.conf: https://paste.opendev.org/show/bjFAPCjlCG6MqINGsIlK/ glance-swift.conf: https://paste.opendev.org/show/bcCfBQDj32T3qgiJj3P7/
I'm only using a single tenant glance container within the admin project. What's really weird is this issue still arise and exist with any known (to me) combination of configuration.
I've tried to declare the store right within the glance-api.conf file, but it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive instead of the auth_address one, it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive plus the auth_address one, it's not working. I've tried to replace auth_address directive with the deprecated swift_store_auth_address within the glance-swift.conf, same error and issue again.
When I replace the default_backend value with performance, uploading an image works perfectly.
I know that my swift CEPH RGW backed endpoint works as I use it already to store data. I've even tried to push an image on it both using CLI (openstack object) and cURL API call and it works. I'm a bit lost here, so if anyone is having thoughts on this I would be happy to hear you!
Thanks a lot everyone!
-- Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards, Abhishek Kekane
All right, just tested to move the directive and it now works like a charm! Thanks a lot! Le mar. 24 mai 2022 à 19:54, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:42 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Ok ok, I was kinda assuming that but was a bit lost by both doc and code as doc specify it multiple time within config but also within the configuring page but this time with third alternative location ^^
Will test that and let you know.
Thanks a lot for this first review.
No worries, please let me know if you have any questions.
Abhishek
Le mar. 24 mai 2022 à 18:10, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:34 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Sure! However it is specified two times within the official glance configuration, once in glance_store and once in glance.store.swift.store but that last section is really not that well documented.
https://docs.openstack.org/glance/latest/configuration/glance_api.html#glanc...
Ack, that needs to be fixed then, important thing is if you are using multiple stores then store specific parameters must go under specific store section otherwise they won’t get recognised.
Abhishek
Le mar. 24 mai 2022 à 17:54, Abhishek Kekane <akekane@redhat.com> a écrit :
Hi Gaël,
Can you try moving default_swift_reference from glance_store section to particular swift store section, i.e. cold as per your configs.
I guess that is the issue.
Abhishek
On Tue, 24 May 2022 at 7:30 PM, Gaël THEROND < gael.therond@bitswalk.com> wrote:
Hi everyone,
I'm currently enabling glance multi backend support for one of our platforms and we're facing a weird situation.
This platform is a CentOS 8 Stream based Openstack VICTORIA and when we enable the multi-backends feature, the swift store isn't working properly.
When we try to send an image to this store we end up with the following error message: *"ERROR glance_store._drivers.swift.store - A value for swift_store_auth_address is required."*
Here is the complete log trace: https://paste.opendev.org/show/bGeCJjSB1C5GwD6JYFvu/
I'm a bit surprised because according to the error message and the glance_store code it means that either my *default_swift_reference* or *auth_address *fromthe *default_swift_reference *isn't set.
https://opendev.org/openstack/glance_store/src/branch/stable/victoria/glance...
However as you can see on my glance-api.conf file I've (seems so) correctly set the glance_store directive and the store settings:
glance-api.conf: https://paste.opendev.org/show/bjFAPCjlCG6MqINGsIlK/ glance-swift.conf: https://paste.opendev.org/show/bcCfBQDj32T3qgiJj3P7/
I'm only using a single tenant glance container within the admin project. What's really weird is this issue still arise and exist with any known (to me) combination of configuration.
I've tried to declare the store right within the glance-api.conf file, but it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive instead of the auth_address one, it's not working. I've tried to declare the store right within the glance-api.conf file and with the deprecated swift_store_auth_address directive plus the auth_address one, it's not working. I've tried to replace auth_address directive with the deprecated swift_store_auth_address within the glance-swift.conf, same error and issue again.
When I replace the default_backend value with performance, uploading an image works perfectly.
I know that my swift CEPH RGW backed endpoint works as I use it already to store data. I've even tried to push an image on it both using CLI (openstack object) and cURL API call and it works. I'm a bit lost here, so if anyone is having thoughts on this I would be happy to hear you!
Thanks a lot everyone!
-- Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards,
Abhishek Kekane
Great, thanks for the update. I will try to improve the documentation for the same. Abhishek On Tue, 24 May 2022 at 11:33 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
All right, just tested to move the directive and it now works like a charm!
Thanks a lot!
Le mar. 24 mai 2022 à 19:54, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:42 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Ok ok, I was kinda assuming that but was a bit lost by both doc and code as doc specify it multiple time within config but also within the configuring page but this time with third alternative location ^^
Will test that and let you know.
Thanks a lot for this first review.
No worries, please let me know if you have any questions.
Abhishek
Le mar. 24 mai 2022 à 18:10, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:34 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Sure! However it is specified two times within the official glance configuration, once in glance_store and once in glance.store.swift.store but that last section is really not that well documented.
https://docs.openstack.org/glance/latest/configuration/glance_api.html#glanc...
Ack, that needs to be fixed then, important thing is if you are using multiple stores then store specific parameters must go under specific store section otherwise they won’t get recognised.
Abhishek
Le mar. 24 mai 2022 à 17:54, Abhishek Kekane <akekane@redhat.com> a écrit :
Hi Gaël,
Can you try moving default_swift_reference from glance_store section to particular swift store section, i.e. cold as per your configs.
I guess that is the issue.
Abhishek
On Tue, 24 May 2022 at 7:30 PM, Gaël THEROND < gael.therond@bitswalk.com> wrote:
> Hi everyone, > > I'm currently enabling glance multi backend support for one of our > platforms and we're facing a weird situation. > > This platform is a CentOS 8 Stream based Openstack VICTORIA and when > we enable the multi-backends feature, the swift store isn't working > properly. > > When we try to send an image to this store we end up with the > following error message: > *"ERROR glance_store._drivers.swift.store - A value for > swift_store_auth_address is required."* > > Here is the complete log trace: > https://paste.opendev.org/show/bGeCJjSB1C5GwD6JYFvu/ > > I'm a bit surprised because according to the error message and the > glance_store code it means that either my *default_swift_reference* > or *auth_address *fromthe *default_swift_reference *isn't set. > > > https://opendev.org/openstack/glance_store/src/branch/stable/victoria/glance... > > However as you can see on my glance-api.conf file I've (seems so) > correctly set the glance_store directive and the store settings: > > glance-api.conf: > https://paste.opendev.org/show/bjFAPCjlCG6MqINGsIlK/ > glance-swift.conf: > https://paste.opendev.org/show/bcCfBQDj32T3qgiJj3P7/ > > I'm only using a single tenant glance container within the admin > project. > What's really weird is this issue still arise and exist with any > known (to me) combination of configuration. > > I've tried to declare the store right within the glance-api.conf > file, but it's not working. > I've tried to declare the store right within the glance-api.conf > file and with the deprecated swift_store_auth_address directive instead of > the auth_address one, it's not working. > I've tried to declare the store right within the glance-api.conf > file and with the deprecated swift_store_auth_address directive plus the > auth_address one, it's not working. > I've tried to replace auth_address directive with the deprecated > swift_store_auth_address within the glance-swift.conf, same error and issue > again. > > When I replace the default_backend value with performance, uploading > an image works perfectly. > > I know that my swift CEPH RGW backed endpoint works as I use it > already to store data. I've even tried to push an image on it both using > CLI (openstack object) and cURL API call and it works. > I'm a bit lost here, so if anyone is having thoughts on this I would > be happy to hear you! > > Thanks a lot everyone! > -- Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards,
Abhishek Kekane
-- Thanks & Best Regards,
Abhishek Kekane
Hi Abhishek, if it can help here are both documentation URLs that are mixin up things and so get me confused with what’s in code. The default glance documentation: https://docs.openstack.org/glance/latest/configuration/configuring.html#conf... Here the doc state that swift_store_config_file should be in [DEFAULT] section. If it’s true when not using multiple backend store we could probably add a disclaimer paragraph for multiple backend stores setup. The default glance-api.conf configuration file: https://docs.openstack.org/glance/latest/configuration/glance_api.html Here the doc state that the directive should be placed within either [glance_store] or [glance.store.swift.store] as for upper if it’s true for setup not using multiple backends we probably should add a disclaimer paragraph and add a more explicit paragraph about backend naming reference within the config file as well as a more specific exemple for the value of default_backend directive as with multiple backend it HAVE to relate to the backend key value you want to target. Anyway, thanks a lot for your kind help, let me know if you want me to fix this doc. Cheers! Le mer. 25 mai 2022 à 04:08, Abhishek Kekane <akekane@redhat.com> a écrit :
Great, thanks for the update. I will try to improve the documentation for the same.
Abhishek
On Tue, 24 May 2022 at 11:33 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
All right, just tested to move the directive and it now works like a charm!
Thanks a lot!
Le mar. 24 mai 2022 à 19:54, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:42 PM, Gaël THEROND <gael.therond@bitswalk.com> wrote:
Ok ok, I was kinda assuming that but was a bit lost by both doc and code as doc specify it multiple time within config but also within the configuring page but this time with third alternative location ^^
Will test that and let you know.
Thanks a lot for this first review.
No worries, please let me know if you have any questions.
Abhishek
Le mar. 24 mai 2022 à 18:10, Abhishek Kekane <akekane@redhat.com> a écrit :
On Tue, 24 May 2022 at 9:34 PM, Gaël THEROND < gael.therond@bitswalk.com> wrote:
Sure! However it is specified two times within the official glance configuration, once in glance_store and once in glance.store.swift.store but that last section is really not that well documented.
https://docs.openstack.org/glance/latest/configuration/glance_api.html#glanc...
Ack, that needs to be fixed then, important thing is if you are using multiple stores then store specific parameters must go under specific store section otherwise they won’t get recognised.
Abhishek
Le mar. 24 mai 2022 à 17:54, Abhishek Kekane <akekane@redhat.com> a écrit :
> Hi Gaël, > > Can you try moving default_swift_reference from glance_store section > to particular swift store section, i.e. cold as per your configs. > > I guess that is the issue. > > Abhishek > > On Tue, 24 May 2022 at 7:30 PM, Gaël THEROND < > gael.therond@bitswalk.com> wrote: > >> Hi everyone, >> >> I'm currently enabling glance multi backend support for one of our >> platforms and we're facing a weird situation. >> >> This platform is a CentOS 8 Stream based Openstack VICTORIA and >> when we enable the multi-backends feature, the swift store isn't working >> properly. >> >> When we try to send an image to this store we end up with the >> following error message: >> *"ERROR glance_store._drivers.swift.store - A value for >> swift_store_auth_address is required."* >> >> Here is the complete log trace: >> https://paste.opendev.org/show/bGeCJjSB1C5GwD6JYFvu/ >> >> I'm a bit surprised because according to the error message and the >> glance_store code it means that either my *default_swift_reference* >> or *auth_address *fromthe *default_swift_reference *isn't set. >> >> >> https://opendev.org/openstack/glance_store/src/branch/stable/victoria/glance... >> >> However as you can see on my glance-api.conf file I've (seems so) >> correctly set the glance_store directive and the store settings: >> >> glance-api.conf: >> https://paste.opendev.org/show/bjFAPCjlCG6MqINGsIlK/ >> glance-swift.conf: >> https://paste.opendev.org/show/bcCfBQDj32T3qgiJj3P7/ >> >> I'm only using a single tenant glance container within the admin >> project. >> What's really weird is this issue still arise and exist with any >> known (to me) combination of configuration. >> >> I've tried to declare the store right within the glance-api.conf >> file, but it's not working. >> I've tried to declare the store right within the glance-api.conf >> file and with the deprecated swift_store_auth_address directive instead of >> the auth_address one, it's not working. >> I've tried to declare the store right within the glance-api.conf >> file and with the deprecated swift_store_auth_address directive plus the >> auth_address one, it's not working. >> I've tried to replace auth_address directive with the deprecated >> swift_store_auth_address within the glance-swift.conf, same error and issue >> again. >> >> When I replace the default_backend value with performance, >> uploading an image works perfectly. >> >> I know that my swift CEPH RGW backed endpoint works as I use it >> already to store data. I've even tried to push an image on it both using >> CLI (openstack object) and cURL API call and it works. >> I'm a bit lost here, so if anyone is having thoughts on this I >> would be happy to hear you! >> >> Thanks a lot everyone! >> > -- > Thanks & Best Regards, > > Abhishek Kekane > --
Thanks & Best Regards,
Abhishek Kekane
--
Thanks & Best Regards,
Abhishek Kekane
-- Thanks & Best Regards,
Abhishek Kekane
participants (2)
-
Abhishek Kekane
-
Gaël THEROND