I looked at tcib definitions but we have never added boto3 even in master.

However, according to the glance_store code, I expect glance_store would be able to ignore the error
even if boto3 is not enabled unless you enable s3 backend somehow.

https://github.com/openstack/glance_store/blob/64e25979a904e468e1e89d8727e9554662596acf/glance_store/backend.py#L222-L224

If you can find the log line "Failed to load driver ..." in glance-api.log then the cause would be different
and we have to check the full logs(both /var/log/containers/glance/glance-api.log and probably
/var/log/containers/sdtouts/glance_api.log) to find out what is actually causing the crash.

On Tue, Feb 21, 2023 at 4:05 PM Pranali Deore <pdeore@redhat.com> wrote:
This doesn't seem like a fatal error, so the issue might be something else.
What podman logs glance_api shows? 

On Tue, Feb 21, 2023 at 12:14 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Abhishek,
Yes, this feels like the same bug, as per the comment from Lars Wiegman (lars-fuga) my glance api container is in reboot loop, when glance multi store is configured.

So I have been searching for a patch for tripleo.

With regards,
Swogat Pradhan

On Tue, Feb 21, 2023 at 12:09 PM Abhishek Kekane <akekane@redhat.com> wrote:
Hi Swogat,

I think this looks relevant, kindly go through this and confirm.
https://bugs.launchpad.net/kolla/+bug/1884259


Thanks & Best Regards,

Abhishek Kekane


On Tue, Feb 21, 2023 at 11:58 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi pranali,
Those configs('central', 'cinder_store') were added when i had started deploying the cloud, those parameters have been removed from the yaml now, but exist in the config file.
Right now i have reverted the config so i am not getting any erros in the glance api logs.
But when i configure the multistore i get the following errors on both sides (central and edge) in glance_api.log:
2023-02-13 13:02:19.878 7 INFO glance.async_ [-] Threadpool model set to 'EventletThreadPoolModel'
2023-02-13 13:02:19.969 7 INFO keyring.backend [-] Loading KWallet
2023-02-13 13:02:19.977 7 INFO keyring.backend [-] Loading SecretService
2023-02-13 13:02:19.982 7 INFO keyring.backend [-] Loading Windows
2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading chainer
2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading macOS
2023-02-13 13:02:20.571 7 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True.
2023-02-13 13:02:20.711 7 ERROR stevedore.extension [-] Could not load 'glance.store.s3.Store': No module named 'boto3': ModuleNotFoundError: No module named 'boto3'
2023-02-13 13:02:20.719 7 ERROR stevedore.extension [-] Could not load 's3': No module named 'boto3': ModuleNotFoundError: No module named 'boto3'

I get no error whatsoever other than the above.

With regards,
Swogat Pradhan

On Tue, Feb 21, 2023 at 11:44 AM Pranali Deore <pdeore@redhat.com> wrote:
Hey Swogat,

By looking at the conf files you've shared, not sure from where the extra store sections 'central', 'cinder_store' are added since you have not added those in any of your yaml files.
Could you please share glance-api logs from both the nodes to check more details?
Probably the HA configuration is causing this kind of issue but need more information related to deployment & log to understand the root cause.



Thanks,
~Pranali
 

On Mon, Feb 20, 2023 at 4:45 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Pranali,
Thank you for your response.
The glance.yaml i had shared is for the central site.
I am attaching the required files for your reference.

With regards,
Swogat Pradhan

On Mon, Feb 20, 2023 at 4:29 PM Pranali Deore <pdeore@redhat.com> wrote:
Hi Swogat,

The glance.yaml you've shared is for central site or edge site?
Is it possible for you to share your glance-api.conf just to check how configuration values are set?



Thanks,
Pranali

On Tue, Feb 14, 2023 at 12:35 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Additional INFO:
The glance_api container is in reboot loop after adding multi store config.
Which is why the image service has become unreachable.


On Mon, Feb 13, 2023 at 6:39 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi,
Openstack wallaby | tripleo

I am trying to setup DCN where i want to setup a glance multistore config in both central and edge sites as per the documentation.
But when ever i run stack update with the following config for multistore:

[stack@hkg2director ~]$ cat workplace/glance.yaml
parameter_defaults:
  GlanceShowMultipleLocations: true
  GlanceBackend: rbd
  GlanceEnabledImportMethods: web-download,copy-image
  GlanceBackendID: ceph
  GlanceMultistoreConfig:
    dcn01:
      GlanceBackend: rbd
      GlanceStoreDescription: 'dcn01 rbd glance store'
      CephClusterName: dcn01

The image service becomes inaccessible. Giving the following error:

(overcloud) [stack@hkg2director ~]$ openstack image list
Failed to contact the endpoint at https://bolt.bdxworld.com:13292 for discovery. Fallback to using that endpoint as the base url.
Failed to contact the endpoint at https://bolt.bdxworld.com:13292 for discovery. Fallback to using that endpoint as the base url.
The image service for :regionOne exists but does not have any supported versions.


glance api.log from controller nodes:

2023-02-13 13:02:19.878 7 INFO glance.async_ [-] Threadpool model set to 'EventletThreadPoolModel'
2023-02-13 13:02:19.969 7 INFO keyring.backend [-] Loading KWallet
2023-02-13 13:02:19.977 7 INFO keyring.backend [-] Loading SecretService
2023-02-13 13:02:19.982 7 INFO keyring.backend [-] Loading Windows
2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading chainer
2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading macOS
2023-02-13 13:02:20.571 7 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True.
2023-02-13 13:02:20.711 7 ERROR stevedore.extension [-] Could not load 'glance.store.s3.Store': No module named 'boto3': ModuleNotFoundError: No module named 'boto3'
2023-02-13 13:02:20.719 7 ERROR stevedore.extension [-] Could not load 's3': No module named 'boto3': ModuleNotFoundError: No module named 'boto3'

Is there a patch to fix this issue??