<div dir="ltr">Hi Thanks, <div>for the inputs, we could see the miss, </div><div>now we have added the required miss :</div><div>"<span style="font-family:monospace">TripleO resource </span>OS::TripleO::Services::CephExternal: ../deployment/cephadm/ceph-client.yaml"</div><div><br></div><div>Now with this setting if we deploy the setup in wallaby, we are getting error as:</div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br>PLAY [External deployment step 1] **********************************************<br>2022-08-11 08:33:20.183104 | 525400d4-7124-4a42-664c-0000000000a8 | TASK | External deployment step 1<br>2022-08-11 08:33:20.211821 | 525400d4-7124-4a42-664c-0000000000a8 | OK | External deployment step 1 | undercloud -> localhost | result={<br> "changed": false,<br> "msg": "Use --start-at-task 'External deployment step 1' to resume from this task"<br>}<br>[WARNING]: ('undercloud -> localhost', '525400d4-7124-4a42-664c-0000000000a8')<br>missing from stats<br>2022-08-11 08:33:20.254775 | 525400d4-7124-4a42-664c-0000000000a9 | TIMING | include_tasks | undercloud | 0:05:01.151528 | 0.03s<br>2022-08-11 08:33:20.304290 | 730cacb3-fa5a-4dca-9730-9a8ce54fb5a3 | INCLUDED | /home/stack/overcloud-deploy/overcloud/config-download/overcloud/external_deploy_steps_tasks_step1.yaml | undercloud<br>2022-08-11 08:33:20.322079 | 525400d4-7124-4a42-664c-0000000048d0 | TASK | Set some tripleo-ansible facts<br>2022-08-11 08:33:20.350423 | 525400d4-7124-4a42-664c-0000000048d0 | OK | Set some tripleo-ansible facts | undercloud<br>2022-08-11 08:33:20.351792 | 525400d4-7124-4a42-664c-0000000048d0 | TIMING | Set some tripleo-ansible facts | undercloud | 0:05:01.248558 | 0.03s<br>2022-08-11 08:33:20.366717 | 525400d4-7124-4a42-664c-0000000048d7 | TASK | Container image prepare<br>2022-08-11 08:34:32.486108 | 525400d4-7124-4a42-664c-0000000048d7 | FATAL | Container image prepare | <b>undercloud | error={"changed": false, "error": "None: Max retries exceeded with url: /v2/ (Caused by None)", "msg": "Error running container image prepare: None: Max retries exceeded with url: /v2/ (Caused by None)", "params": {}, "success": false}</b><br>2022-08-11 08:34:32.488845 | 525400d4-7124-4a42-664c-0000000048d7 | TIMING | tripleo_container_image_prepare : Container image prepare | undercloud | 0:06:13.385607 | 72.12s<br><br></div></blockquote>This gets failed at step 1, As this is wallaby and based on the document (<a href="https://docs.openstack.org/project-deploy-guide/tripleo-docs/wallaby/features/ceph_external.html">Use an external Ceph cluster with the Overcloud — TripleO 3.0.0 documentation (openstack.org)</a>) we should only pass this external-ceph.yaml for the external ceph intergration. </div><div>But it is not happening.</div><div><br></div><div><br></div><div>Few things to note:</div><div>1. Container Prepare:</div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>(undercloud) [stack@undercloud ~]$ cat containers-prepare-parameter.yaml</div><div># Generated with the following on 2022-06-28T18:56:38.642315</div><div>#</div><div># openstack tripleo container image prepare default --local-push-destination --output-env-file /home/stack/containers-prepare-parameter.yaml</div><div>#</div></blockquote><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>parameter_defaults:</div><div> ContainerImagePrepare:</div><div> - push_destination: true</div><div> set:</div><div> name_prefix: openstack-</div><div> name_suffix: ''</div><div> namespace: <a href="http://myserver.com:5000/tripleowallaby">myserver.com:5000/tripleowallaby</a></div><div> neutron_driver: ovn</div><div> rhel_containers: false</div><div> tag: current-tripleo</div><div> tag_from_label: rdo_version</div><div>(undercloud) [stack@undercloud ~]$</div></blockquote>2. this is SSL based deployment. <div><br></div><div>Any idea for the error, the issue is seen only once we have the external ceph integration enabled. </div><div><br></div><div>Best Regards,</div><div>Lokendra</div><div><br><div><div><br></div><div><br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 4, 2022 at 7:22 PM Francesco Pantano <<a href="mailto:fpantano@redhat.com">fpantano@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:monospace">Hi,<br>ceph is supposed to be configured by this tripleo-ansible role [1], which is triggered by tht on external_deploy_steps [2].<br>In theory adding [3] should just work, assuming you customize the ceph cluster mon ip addresses, fsid and a few other related variables.<br>From your previous email I suspect in your external-ceph.yaml you missed the TripleO resource <span style="font-family:Arial,Helvetica,sans-serif">OS::TripleO::Services::CephExternal</span><span style="font-family:Arial,Helvetica,sans-serif">: </span><span style="font-family:Arial,Helvetica,sans-serif">../deployment/cephadm/ceph-client.yaml</span></div><div class="gmail_default">(see [3]).<br><br>Thanks,<br>Francesco</div><div class="gmail_default" style="font-family:monospace"><br></div><div class="gmail_default" style="font-family:monospace"><br></div><div class="gmail_default" style="font-family:monospace">[1] <a href="https://github.com/openstack/tripleo-ansible/tree/master/tripleo_ansible/roles/tripleo_ceph_client" target="_blank">https://github.com/openstack/tripleo-ansible/tree/master/tripleo_ansible/roles/tripleo_ceph_client</a></div><div class="gmail_default" style="font-family:monospace">[2] <a href="https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/cephadm/ceph-client.yaml#L93" target="_blank">https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/cephadm/ceph-client.yaml#L93</a></div><div class="gmail_default" style="font-family:monospace">[3] <a href="https://github.com/openstack/tripleo-heat-templates/blob/master/environments/external-ceph.yaml" target="_blank">https://github.com/openstack/tripleo-heat-templates/blob/master/environments/external-ceph.yaml</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 4, 2022 at 2:01 PM Lokendra Rathour <<a href="mailto:lokendrarathour@gmail.com" target="_blank">lokendrarathour@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Team,<div>I was trying to integrate External Ceph with Triple0 Wallaby, and at the end of deployment in step4 getting the below error:</div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)">2022-08-03 18:37:21,158 p=507732 u=stack n=ansible | 2022-08-03 18:37:21.157962 | 525400fe-86b8-65d9-d100-0000000080d2 | TASK | Create containers from /var/lib/tripleo-config/container-startup-config/step_4</span><br style="box-sizing:border-box;color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)"><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)">2022-08-03 18:37:21,239 p=507732 u=stack n=ansible | 2022-08-03 18:37:21.238718 | 69e98219-f748-4af7-a6d0-f8f73680ce9b | INCLUDED | /usr/share/ansible/roles/tripleo_container_manage/tasks/create.yml | overcloud-controller-2</span><br style="box-sizing:border-box;color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)"><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)">2022-08-03 18:37:21,273 p=507732 u=stack n=ansible | 2022-08-03 18:37:21.272340 | 525400fe-86b8-65d9-d100-0000000086d9 | TASK | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_4</span><br style="box-sizing:border-box;color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)"><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)">2022-08-03 18:37:24,532 p=507732 u=stack n=ansible | 2022-08-03 18:37:24.530812 | | WARNING | ERROR: Can't run container nova_libvirt_init_secret</span><br style="box-sizing:border-box;color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)"><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)">stderr:</span><br style="box-sizing:border-box;color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)"><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)">2022-08-03 18:37:24,533 p=507732 u=stack n=ansible | 2022-08-03 18:37:24.532811 | 525400fe-86b8-65d9-d100-0000000082ec | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-</span><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(255,255,0)">config/step_4 | overcloud-novacompute-0 | error={"changed": false, "msg": "Failed containers: nova_libvirt_init_secret"}</span><br style="box-sizing:border-box;color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)"><span style="color:rgb(36,36,36);font-family:"Segoe UI",system-ui,"Apple Color Emoji","Segoe UI Emoji",sans-serif;font-size:14px;background-color:rgb(232,235,250)">2022-08-03 18:37:44,282 p=507732 u</span><br></div></blockquote></div><div><br></div><div><b>external-ceph.conf:</b></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>parameter_defaults:</div><div> # Enable use of RBD backend in nova-compute</div><div> NovaEnableRbdBackend: True</div><div> # Enable use of RBD backend in cinder-volume</div><div> CinderEnableRbdBackend: True</div><div> # Backend to use for cinder-backup</div><div> CinderBackupBackend: ceph</div><div> # Backend to use for glance</div><div> GlanceBackend: rbd</div><div> # Name of the Ceph pool hosting Nova ephemeral images</div><div> NovaRbdPoolName: vms</div><div> # Name of the Ceph pool hosting Cinder volumes</div><div> CinderRbdPoolName: volumes</div><div> # Name of the Ceph pool hosting Cinder backups</div><div> CinderBackupRbdPoolName: backups</div><div> # Name of the Ceph pool hosting Glance images</div><div> GlanceRbdPoolName: images</div><div> # Name of the user to authenticate with the external Ceph cluster</div><div> CephClientUserName: admin</div><div> # The cluster FSID</div><div> CephClusterFSID: 'ca3080-aaaa-4d1a-b1fd-4aaaa9a9ea4c'</div><div> # The CephX user auth key</div><div> CephClientKey: 'AQDgRjhiuLMnAxAAnYwgERERFy0lzH6ufSl70A=='</div><div> # The list of Ceph monitors</div><div> CephExternalMonHost: 'abcd:abcd:abcd::11,abcd:abcd:abcd::12,abcd:abcd:abcd::13'</div><div>~</div></blockquote><div><div><br></div><div>Have tried checking and validating the ceph client details and they seem to be correct, further digging the container log I could see something like this :</div><div><br></div><div>[root@overcloud-novacompute-0 containers]# tail -f nova_libvirt_init_secret.log<br>tail: cannot open 'nova_libvirt_init_secret.log' for reading: No such file or directory<br>tail: no files remaining<br>[root@overcloud-novacompute-0 containers]# tail -f stdouts/nova_libvirt_init_secret.log<br>2022-08-04T11:48:47.689898197+05:30 stdout F ------------------------------------------------<br>2022-08-04T11:48:47.690002011+05:30 stdout F Initializing virsh secrets for: ceph:admin<br>2022-08-04T11:48:47.690590594+05:30 stdout F Error: /etc/ceph/ceph.conf was not found<br>2022-08-04T11:48:47.690625088+05:30 stdout F Path to nova_libvirt_init_secret was ceph:admin<br>2022-08-04T16:20:29.643785538+05:30 stdout F ------------------------------------------------<br>2022-08-04T16:20:29.643785538+05:30 stdout F Initializing virsh secrets for: ceph:admin<br>2022-08-04T16:20:29.644785532+05:30 stdout F Error: /etc/ceph/ceph.conf was not found<br>2022-08-04T16:20:29.644785532+05:30 stdout F Path to nova_libvirt_init_secret was ceph:admin<br>^C<br>[root@overcloud-novacompute-0 containers]# tail -f stdouts/nova_compute_init_log.log<br></div><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">~ Lokendra<br></div><div>skype: lokendrarathour</div><div dir="ltr"><img width="200" height="41" src="https://ci3.googleusercontent.com/mail-sig/AIorK4zyd6LpJOGqagxmzUlY59eMQx0-FN0t8HtjdtGE7VLZSKIxBUz3bI7z-MBqbgDVg1-XbtvHgN_ATJ10N6bonyO-JSGTtl5s_mNSbDoXBg"><br></div><div dir="ltr"><br></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><span><font color="#888888"><font face="monospace">Francesco Pantano<br>
GPG KEY: F41BD75C</font><br></font></span></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr">~ Lokendra<br></div><div>skype: lokendrarathour</div><div dir="ltr"><img width="200" height="41" src="https://ci3.googleusercontent.com/mail-sig/AIorK4zyd6LpJOGqagxmzUlY59eMQx0-FN0t8HtjdtGE7VLZSKIxBUz3bI7z-MBqbgDVg1-XbtvHgN_ATJ10N6bonyO-JSGTtl5s_mNSbDoXBg"><br></div><div dir="ltr"><br></div></div></div></div>