I looked at tcib definitions but we have never added boto3 even in master. However, according to the glance_store code, I expect glance_store would be able to ignore the error even if boto3 is not enabled unless you enable s3 backend somehow. https://github.com/openstack/glance_store/blob/64e25979a904e468e1e89d8727e95... If you can find the log line "Failed to load driver ..." in glance-api.log then the cause would be different and we have to check the full logs(both /var/log/containers/glance/glance-api.log and probably /var/log/containers/sdtouts/glance_api.log) to find out what is actually causing the crash. On Tue, Feb 21, 2023 at 4:05 PM Pranali Deore <pdeore@redhat.com> wrote:
This doesn't seem like a fatal error, so the issue might be something else. What podman logs glance_api shows?
On Tue, Feb 21, 2023 at 12:14 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Abhishek, Yes, this feels like the same bug, as per the comment from Lars Wiegman (lars-fuga) <https://launchpad.net/~lars-fuga> my glance api container is in reboot loop, when glance multi store is configured.
So I have been searching for a patch for tripleo.
With regards, Swogat Pradhan
On Tue, Feb 21, 2023 at 12:09 PM Abhishek Kekane <akekane@redhat.com> wrote:
Hi Swogat,
I think this looks relevant, kindly go through this and confirm. https://bugs.launchpad.net/kolla/+bug/1884259
Thanks & Best Regards,
Abhishek Kekane
On Tue, Feb 21, 2023 at 11:58 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi pranali, Those configs('central', 'cinder_store') were added when i had started deploying the cloud, those parameters have been removed from the yaml now, but exist in the config file. Right now i have reverted the config so i am not getting any erros in the glance api logs. But when i configure the multistore i get the following errors on both sides (central and edge) in glance_api.log: 2023-02-13 13:02:19.878 7 INFO glance.async_ [-] Threadpool model set to 'EventletThreadPoolModel' 2023-02-13 13:02:19.969 7 INFO keyring.backend [-] Loading KWallet 2023-02-13 13:02:19.977 7 INFO keyring.backend [-] Loading SecretService 2023-02-13 13:02:19.982 7 INFO keyring.backend [-] Loading Windows 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading chainer 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading macOS 2023-02-13 13:02:20.571 7 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2023-02-13 13:02:20.711 7 ERROR stevedore.extension [-] Could not load 'glance.store.s3.Store': No module named 'boto3': ModuleNotFoundError: No module named 'boto3' 2023-02-13 13:02:20.719 7 ERROR stevedore.extension [-] Could not load 's3': No module named 'boto3': ModuleNotFoundError: No module named 'boto3'
I get no error whatsoever other than the above.
With regards, Swogat Pradhan
On Tue, Feb 21, 2023 at 11:44 AM Pranali Deore <pdeore@redhat.com> wrote:
Hey Swogat,
By looking at the conf files you've shared, not sure from where the extra store sections 'central', 'cinder_store' are added since you have not added those in any of your yaml files. Could you please share glance-api logs from both the nodes to check more details? Probably the HA configuration is causing this kind of issue but need more information related to deployment & log to understand the root cause.
Thanks, ~Pranali
On Mon, Feb 20, 2023 at 4:45 PM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Pranali, Thank you for your response. The glance.yaml i had shared is for the central site. I am attaching the required files for your reference.
With regards, Swogat Pradhan
On Mon, Feb 20, 2023 at 4:29 PM Pranali Deore <pdeore@redhat.com> wrote:
> Hi Swogat, > > The glance.yaml you've shared is for central site or edge site? > Is it possible for you to share your glance-api.conf just to check > how configuration values are set? > > > > Thanks, > Pranali > > On Tue, Feb 14, 2023 at 12:35 PM Swogat Pradhan < > swogatpradhan22@gmail.com> wrote: > >> Additional INFO: >> The glance_api container is in reboot loop after adding multi store >> config. >> Which is why the image service has become unreachable. >> >> >> On Mon, Feb 13, 2023 at 6:39 PM Swogat Pradhan < >> swogatpradhan22@gmail.com> wrote: >> >>> Hi, >>> Openstack wallaby | tripleo >>> >>> I am trying to setup DCN where i want to setup a glance multistore >>> config in both central and edge sites as per the documentation. >>> But when ever i run stack update with the following config for >>> multistore: >>> >>> [stack@hkg2director ~]$ cat workplace/glance.yaml >>> parameter_defaults: >>> GlanceShowMultipleLocations: true >>> GlanceBackend: rbd >>> GlanceEnabledImportMethods: web-download,copy-image >>> GlanceBackendID: ceph >>> GlanceMultistoreConfig: >>> dcn01: >>> GlanceBackend: rbd >>> GlanceStoreDescription: 'dcn01 rbd glance store' >>> CephClusterName: dcn01 >>> >>> The image service becomes inaccessible. Giving the following error: >>> >>> (overcloud) [stack@hkg2director ~]$ openstack image list >>> Failed to contact the endpoint at https://bolt.bdxworld.com:13292 >>> for discovery. Fallback to using that endpoint as the base url. >>> Failed to contact the endpoint at https://bolt.bdxworld.com:13292 >>> for discovery. Fallback to using that endpoint as the base url. >>> The image service for :regionOne exists but does not have any >>> supported versions. >>> >>> >>> glance api.log from controller nodes: >>> >>> 2023-02-13 13:02:19.878 7 INFO glance.async_ [-] Threadpool model >>> set to 'EventletThreadPoolModel' >>> 2023-02-13 13:02:19.969 7 INFO keyring.backend [-] Loading KWallet >>> 2023-02-13 13:02:19.977 7 INFO keyring.backend [-] Loading >>> SecretService >>> 2023-02-13 13:02:19.982 7 INFO keyring.backend [-] Loading Windows >>> 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading chainer >>> 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading macOS >>> 2023-02-13 13:02:20.571 7 WARNING keystonemiddleware.auth_token >>> [-] AuthToken middleware is set with >>> keystone_authtoken.service_token_roles_required set to False. This is >>> backwards compatible but deprecated behaviour. Please set this to True. >>> 2023-02-13 13:02:20.711 7 ERROR stevedore.extension [-] Could not >>> load 'glance.store.s3.Store': No module named 'boto3': ModuleNotFoundError: >>> No module named 'boto3' >>> 2023-02-13 13:02:20.719 7 ERROR stevedore.extension [-] Could not >>> load 's3': No module named 'boto3': ModuleNotFoundError: No module named >>> 'boto3' >>> >>> Is there a patch to fix this issue?? >>> >>