[Triple0] [Wallaby] External Ceph Integration getting failed

John Fulton johfulto at redhat.com
Mon Aug 29 13:19:54 UTC 2022


On Fri, Aug 26, 2022 at 11:03 PM Lokendra Rathour <lokendrarathour at gmail.com>
wrote:

> Hi John,
> It got resolved, reason was NTP.
> The NTP time was not in sync., i noticed thay recently the NTP is not
> getting configured properly on the controller and xompute nodes.
> After enabling thr time sync and validation we redeployed and it worked
> fine.
>
> I have another querry w.r.t to storage integration with a tripleo.
>
> We have noticed that only passing the external-ceph.yaml is not doing the
> deployment, we also need to pass ceph parameters in container-prepare.
> We did see some containers getting downloaded as well but after the
> deployment is done we do not see them anywhere.
> What can be the reason for such containers if not used ?
> Any point would help me further ensure 100% offline tripleO
>

You'll need the ceph container if:

1. If you're using NFS Ganesha with external ceph
2. If you're using  ceph-ansible with external ceph

You should be using Wallaby however as per [1]. If you're only using RBD
you shouldn't need the ceph container. This role should set up your ceph
conf and key files.


https://github.com/openstack/tripleo-ansible/tree/stable/wallaby/tripleo_ansible/roles/tripleo_ceph_client

For offline tripleo, you need overcloud containers (regardless of if the
ceph container is one of them). The solution to that problem is to use the
undercloud as a container registry as per [2].

  John

[1]
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/ceph_external.html
[2]
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployment/container_image_prepare.html


>
> On Thu, 4 Aug 2022, 17:07 Lokendra Rathour, <lokendrarathour at gmail.com>
> wrote:
>
>> Hi Team,
>> I was trying to integrate External Ceph with Triple0 Wallaby, and at the
>> end of deployment in step4 getting the below error:
>>
>> 2022-08-03 18:37:21,158 p=507732 u=stack n=ansible | 2022-08-03
>> 18:37:21.157962 | 525400fe-86b8-65d9-d100-0000000080d2 |       TASK |
>> Create containers from
>> /var/lib/tripleo-config/container-startup-config/step_4
>> 2022-08-03 18:37:21,239 p=507732 u=stack n=ansible | 2022-08-03
>> 18:37:21.238718 | 69e98219-f748-4af7-a6d0-f8f73680ce9b |   INCLUDED |
>> /usr/share/ansible/roles/tripleo_container_manage/tasks/create.yml |
>> overcloud-controller-2
>> 2022-08-03 18:37:21,273 p=507732 u=stack n=ansible | 2022-08-03
>> 18:37:21.272340 | 525400fe-86b8-65d9-d100-0000000086d9 |       TASK |
>> Create containers managed by Podman for
>> /var/lib/tripleo-config/container-startup-config/step_4
>> 2022-08-03 18:37:24,532 p=507732 u=stack n=ansible | 2022-08-03
>> 18:37:24.530812 |                                      |    WARNING |
>> ERROR: Can't run container nova_libvirt_init_secret
>> stderr:
>> 2022-08-03 18:37:24,533 p=507732 u=stack n=ansible | 2022-08-03
>> 18:37:24.532811 | 525400fe-86b8-65d9-d100-0000000082ec |      FATAL |
>> Create containers managed by Podman for
>> /var/lib/tripleo-config/container-startup-config/step_4 |
>> overcloud-novacompute-0 | error={"changed": false, "msg": "Failed
>> containers: nova_libvirt_init_secret"}
>> 2022-08-03 18:37:44,282 p=507732 u
>>
>>
>> *external-ceph.conf:*
>>
>> parameter_defaults:
>>   # Enable use of RBD backend in nova-compute
>>   NovaEnableRbdBackend: True
>>   # Enable use of RBD backend in cinder-volume
>>   CinderEnableRbdBackend: True
>>   # Backend to use for cinder-backup
>>   CinderBackupBackend: ceph
>>   # Backend to use for glance
>>   GlanceBackend: rbd
>>   # Name of the Ceph pool hosting Nova ephemeral images
>>   NovaRbdPoolName: vms
>>   # Name of the Ceph pool hosting Cinder volumes
>>   CinderRbdPoolName: volumes
>>   # Name of the Ceph pool hosting Cinder backups
>>   CinderBackupRbdPoolName: backups
>>   # Name of the Ceph pool hosting Glance images
>>   GlanceRbdPoolName: images
>>   # Name of the user to authenticate with the external Ceph cluster
>>   CephClientUserName: admin
>>   # The cluster FSID
>>   CephClusterFSID: 'ca3080-aaaa-4d1a-b1fd-4aaaa9a9ea4c'
>>   # The CephX user auth key
>>   CephClientKey: 'AQDgRjhiuLMnAxAAnYwgERERFy0lzH6ufSl70A=='
>>   # The list of Ceph monitors
>>   CephExternalMonHost:
>> 'abcd:abcd:abcd::11,abcd:abcd:abcd::12,abcd:abcd:abcd::13'
>> ~
>>
>>
>> Have tried checking and validating the ceph client details and they seem
>> to be correct, further digging the container log I could see something like
>> this :
>>
>> [root at overcloud-novacompute-0 containers]# tail -f
>> nova_libvirt_init_secret.log
>> tail: cannot open 'nova_libvirt_init_secret.log' for reading: No such
>> file or directory
>> tail: no files remaining
>> [root at overcloud-novacompute-0 containers]# tail -f
>> stdouts/nova_libvirt_init_secret.log
>> 2022-08-04T11:48:47.689898197+05:30 stdout F
>> ------------------------------------------------
>> 2022-08-04T11:48:47.690002011+05:30 stdout F Initializing virsh secrets
>> for: ceph:admin
>> 2022-08-04T11:48:47.690590594+05:30 stdout F Error: /etc/ceph/ceph.conf
>> was not found
>> 2022-08-04T11:48:47.690625088+05:30 stdout F Path to
>> nova_libvirt_init_secret was ceph:admin
>> 2022-08-04T16:20:29.643785538+05:30 stdout F
>> ------------------------------------------------
>> 2022-08-04T16:20:29.643785538+05:30 stdout F Initializing virsh secrets
>> for: ceph:admin
>> 2022-08-04T16:20:29.644785532+05:30 stdout F Error: /etc/ceph/ceph.conf
>> was not found
>> 2022-08-04T16:20:29.644785532+05:30 stdout F Path to
>> nova_libvirt_init_secret was ceph:admin
>> ^C
>> [root at overcloud-novacompute-0 containers]# tail -f
>> stdouts/nova_compute_init_log.log
>>
>> --
>> ~ Lokendra
>> skype: lokendrarathour
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220829/bb4541a8/attachment-0001.htm>


More information about the openstack-discuss mailing list