Open Stack

Hi All,
I have set up glance multi store in the central site and the same issue is
present.
Attaching the logs.

With regards,
Swogat Pradhan

On Tue, Feb 21, 2023 at 10:04 PM Swogat Pradhan <swogatpradhan22 at gmail.com>
wrote:

> Hi,
> I am currently working on client approval to run the stack update.
> I will share the logs shortly.
>
> With regards,
> Swogat Pradhan
>
> On Tue, Feb 21, 2023 at 7:58 PM John Fulton <johfulto at redhat.com> wrote:
>
>> I agree with Pranali and Takashi.
>>
>> I have also seen "No module named 'boto3'" in my logs while setting up
>> DCN with glance successfully. So that is not the cause of your fatal
>> error. You should look in your logs for something else as the root
>> cause of this issue.
>>
>>   John
>>
>> On Tue, Feb 21, 2023 at 2:20 AM Takashi Kajinami <tkajinam at redhat.com>
>> wrote:
>> >
>> > I looked at tcib definitions but we have never added boto3 even in
>> master.
>> >
>> > However, according to the glance_store code, I expect glance_store
>> would be able to ignore the error
>> > even if boto3 is not enabled unless you enable s3 backend somehow.
>> >
>> >
>> https://github.com/openstack/glance_store/blob/64e25979a904e468e1e89d8727e9554662596acf/glance_store/backend.py#L222-L224
>> >
>> > If you can find the log line "Failed to load driver ..." in
>> glance-api.log then the cause would be different
>> > and we have to check the full logs(both
>> /var/log/containers/glance/glance-api.log and probably
>> > /var/log/containers/sdtouts/glance_api.log) to find out what is
>> actually causing the crash.
>> >
>> > On Tue, Feb 21, 2023 at 4:05 PM Pranali Deore <pdeore at redhat.com>
>> wrote:
>> >>
>> >> This doesn't seem like a fatal error, so the issue might be something
>> else.
>> >> What podman logs glance_api shows?
>> >>
>> >> On Tue, Feb 21, 2023 at 12:14 PM Swogat Pradhan <
>> swogatpradhan22 at gmail.com> wrote:
>> >>>
>> >>> Hi Abhishek,
>> >>> Yes, this feels like the same bug, as per the comment from Lars
>> Wiegman (lars-fuga) my glance api container is in reboot loop, when glance
>> multi store is configured.
>> >>>
>> >>> So I have been searching for a patch for tripleo.
>> >>>
>> >>> With regards,
>> >>> Swogat Pradhan
>> >>>
>> >>> On Tue, Feb 21, 2023 at 12:09 PM Abhishek Kekane <akekane at redhat.com>
>> wrote:
>> >>>>
>> >>>> Hi Swogat,
>> >>>>
>> >>>> I think this looks relevant, kindly go through this and confirm.
>> >>>> https://bugs.launchpad.net/kolla/+bug/1884259
>> >>>>
>> >>>>
>> >>>> Thanks & Best Regards,
>> >>>>
>> >>>> Abhishek Kekane
>> >>>>
>> >>>>
>> >>>> On Tue, Feb 21, 2023 at 11:58 AM Swogat Pradhan <
>> swogatpradhan22 at gmail.com> wrote:
>> >>>>>
>> >>>>> Hi pranali,
>> >>>>> Those configs('central', 'cinder_store') were added when i had
>> started deploying the cloud, those parameters have been removed from the
>> yaml now, but exist in the config file.
>> >>>>> Right now i have reverted the config so i am not getting any erros
>> in the glance api logs.
>> >>>>> But when i configure the multistore i get the following errors on
>> both sides (central and edge) in glance_api.log:
>> >>>>> 2023-02-13 13:02:19.878 7 INFO glance.async_ [-] Threadpool model
>> set to 'EventletThreadPoolModel'
>> >>>>> 2023-02-13 13:02:19.969 7 INFO keyring.backend [-] Loading KWallet
>> >>>>> 2023-02-13 13:02:19.977 7 INFO keyring.backend [-] Loading
>> SecretService
>> >>>>> 2023-02-13 13:02:19.982 7 INFO keyring.backend [-] Loading Windows
>> >>>>> 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading chainer
>> >>>>> 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading macOS
>> >>>>> 2023-02-13 13:02:20.571 7 WARNING keystonemiddleware.auth_token [-]
>> AuthToken middleware is set with
>> keystone_authtoken.service_token_roles_required set to False. This is
>> backwards compatible but deprecated behaviour. Please set this to True.
>> >>>>> 2023-02-13 13:02:20.711 7 ERROR stevedore.extension [-] Could not
>> load 'glance.store.s3.Store': No module named 'boto3': ModuleNotFoundError:
>> No module named 'boto3'
>> >>>>> 2023-02-13 13:02:20.719 7 ERROR stevedore.extension [-] Could not
>> load 's3': No module named 'boto3': ModuleNotFoundError: No module named
>> 'boto3'
>> >>>>>
>> >>>>> I get no error whatsoever other than the above.
>> >>>>>
>> >>>>> With regards,
>> >>>>> Swogat Pradhan
>> >>>>>
>> >>>>> On Tue, Feb 21, 2023 at 11:44 AM Pranali Deore <pdeore at redhat.com>
>> wrote:
>> >>>>>>
>> >>>>>> Hey Swogat,
>> >>>>>>
>> >>>>>> By looking at the conf files you've shared, not sure from where
>> the extra store sections 'central', 'cinder_store' are added since you have
>> not added those in any of your yaml files.
>> >>>>>> Could you please share glance-api logs from both the nodes to
>> check more details?
>> >>>>>> Probably the HA configuration is causing this kind of issue but
>> need more information related to deployment & log to understand the root
>> cause.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> ~Pranali
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Feb 20, 2023 at 4:45 PM Swogat Pradhan <
>> swogatpradhan22 at gmail.com> wrote:
>> >>>>>>>
>> >>>>>>> Hi Pranali,
>> >>>>>>> Thank you for your response.
>> >>>>>>> The glance.yaml i had shared is for the central site.
>> >>>>>>> I am attaching the required files for your reference.
>> >>>>>>>
>> >>>>>>> With regards,
>> >>>>>>> Swogat Pradhan
>> >>>>>>>
>> >>>>>>> On Mon, Feb 20, 2023 at 4:29 PM Pranali Deore <pdeore at redhat.com>
>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi Swogat,
>> >>>>>>>>
>> >>>>>>>> The glance.yaml you've shared is for central site or edge site?
>> >>>>>>>> Is it possible for you to share your glance-api.conf just to
>> check how configuration values are set?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> Pranali
>> >>>>>>>>
>> >>>>>>>> On Tue, Feb 14, 2023 at 12:35 PM Swogat Pradhan <
>> swogatpradhan22 at gmail.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Additional INFO:
>> >>>>>>>>> The glance_api container is in reboot loop after adding multi
>> store config.
>> >>>>>>>>> Which is why the image service has become unreachable.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Feb 13, 2023 at 6:39 PM Swogat Pradhan <
>> swogatpradhan22 at gmail.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Hi,
>> >>>>>>>>>> Openstack wallaby | tripleo
>> >>>>>>>>>>
>> >>>>>>>>>> I am trying to setup DCN where i want to setup a glance
>> multistore config in both central and edge sites as per the documentation.
>> >>>>>>>>>> But when ever i run stack update with the following config for
>> multistore:
>> >>>>>>>>>>
>> >>>>>>>>>> [stack at hkg2director ~]$ cat workplace/glance.yaml
>> >>>>>>>>>> parameter_defaults:
>> >>>>>>>>>>   GlanceShowMultipleLocations: true
>> >>>>>>>>>>   GlanceBackend: rbd
>> >>>>>>>>>>   GlanceEnabledImportMethods: web-download,copy-image
>> >>>>>>>>>>   GlanceBackendID: ceph
>> >>>>>>>>>>   GlanceMultistoreConfig:
>> >>>>>>>>>>     dcn01:
>> >>>>>>>>>>       GlanceBackend: rbd
>> >>>>>>>>>>       GlanceStoreDescription: 'dcn01 rbd glance store'
>> >>>>>>>>>>       CephClusterName: dcn01
>> >>>>>>>>>>
>> >>>>>>>>>> The image service becomes inaccessible. Giving the following
>> error:
>> >>>>>>>>>>
>> >>>>>>>>>> (overcloud) [stack at hkg2director ~]$ openstack image list
>> >>>>>>>>>> Failed to contact the endpoint at
>> https://bolt.bdxworld.com:13292 for discovery. Fallback to using that
>> endpoint as the base url.
>> >>>>>>>>>> Failed to contact the endpoint at
>> https://bolt.bdxworld.com:13292 for discovery. Fallback to using that
>> endpoint as the base url.
>> >>>>>>>>>> The image service for :regionOne exists but does not have any
>> supported versions.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> glance api.log from controller nodes:
>> >>>>>>>>>>
>> >>>>>>>>>> 2023-02-13 13:02:19.878 7 INFO glance.async_ [-] Threadpool
>> model set to 'EventletThreadPoolModel'
>> >>>>>>>>>> 2023-02-13 13:02:19.969 7 INFO keyring.backend [-] Loading
>> KWallet
>> >>>>>>>>>> 2023-02-13 13:02:19.977 7 INFO keyring.backend [-] Loading
>> SecretService
>> >>>>>>>>>> 2023-02-13 13:02:19.982 7 INFO keyring.backend [-] Loading
>> Windows
>> >>>>>>>>>> 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading
>> chainer
>> >>>>>>>>>> 2023-02-13 13:02:19.983 7 INFO keyring.backend [-] Loading
>> macOS
>> >>>>>>>>>> 2023-02-13 13:02:20.571 7 WARNING
>> keystonemiddleware.auth_token [-] AuthToken middleware is set with
>> keystone_authtoken.service_token_roles_required set to False. This is
>> backwards compatible but deprecated behaviour. Please set this to True.
>> >>>>>>>>>> 2023-02-13 13:02:20.711 7 ERROR stevedore.extension [-] Could
>> not load 'glance.store.s3.Store': No module named 'boto3':
>> ModuleNotFoundError: No module named 'boto3'
>> >>>>>>>>>> 2023-02-13 13:02:20.719 7 ERROR stevedore.extension [-] Could
>> not load 's3': No module named 'boto3': ModuleNotFoundError: No module
>> named 'boto3'
>> >>>>>>>>>>
>> >>>>>>>>>> Is there a patch to fix this issue??
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230222/54b650c0/attachment.htm>