<div dir="ltr">I just now realized, that I have seen in some log, messages, missing some puppet-dependencies... Cannot find log again...</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 29 Apr 2020 at 09:39, Ruslanas Gžibovskis <<a href="mailto:ruslanas@lpic.lt">ruslanas@lpic.lt</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">podman ps -a = clean, no containers at all.<div>I have a watch -d "sudo podman ps -a ; sudo podman images -a ; sudo df -h"<br><div><br></div><div>paunch.log is empty. (I did several reinstallations).<br></div><div><br></div><div>I found in image logs:</div><div>2020-04-29 08:52:49,854 140572 DEBUG urllib3.connectionpool [ ] <a href="https://registry-1.docker.io:443" target="_blank">https://registry-1.docker.io:443</a> "GET /v2/ HTTP/1.1" 401 87<br>2020-04-29 08:52:49,855 140572 DEBUG tripleo_common.image.image_uploader [ ] <a href="https://registry-1.docker.io/v2/" target="_blank">https://registry-1.docker.io/v2/</a> status code 401<br>2020-04-29 08:52:49,855 140572 DEBUG tripleo_common.image.image_uploader [ ] Token parameters: params {'scope': 'repository:tripleotrain/centos-binary-zaqar-wsgi:pull', 'service': '<a href="http://registry.docker.io" target="_blank">registry.docker.io</a>'}<br></div><div>2020-04-29 08:52:49,731 140572 DEBUG urllib3.connectionpool [ ] <a href="https://registry-1.docker.io:443" target="_blank">https://registry-1.docker.io:443</a> "GET /v2/ HTTP/1.1" 401 87<br>2020-04-29 08:52:49,732 140572 DEBUG tripleo_common.image.image_uploader [ ] <a href="https://registry-1.docker.io/v2/" target="_blank">https://registry-1.docker.io/v2/</a> status code 401<br>2020-04-29 08:52:49,732 140572 DEBUG tripleo_common.image.image_uploader [ ] Token parameters: params {'scope': 'repository:tripleotrain/centos-binary-rsyslog:pull', 'service': '<a href="http://registry.docker.io" target="_blank">registry.docker.io</a>'}</div><div>2020-04-29 08:52:49,583 140572 DEBUG urllib3.connectionpool [ ] <a href="https://registry-1.docker.io:443" target="_blank">https://registry-1.docker.io:443</a> "GET /v2/ HTTP/1.1" 401 87<br>2020-04-29 08:52:49,584 140572 DEBUG tripleo_common.image.image_uploader [ ] <a href="https://registry-1.docker.io/v2/" target="_blank">https://registry-1.docker.io/v2/</a> status code 401<br>2020-04-29 08:52:49,584 140572 DEBUG tripleo_common.image.image_uploader [ ] Token parameters: params {'scope': 'repository:tripleotrain/centos-binary-swift-proxy-server:pull', 'service': '<a href="http://registry.docker.io" target="_blank">registry.docker.io</a>'}<br>2020-04-29 08:52:49,586 140572 DEBUG urllib3.connectionpool [ ] Starting new HTTPS connection (1): <a href="http://auth.docker.io:443" target="_blank">auth.docker.io:443</a><br>2020-04-29 08:52:49,606 140572 DEBUG urllib3.connectionpool [ ] <a href="https://registry-1.docker.io:443" target="_blank">https://registry-1.docker.io:443</a> "GET /v2/ HTTP/1.1" 401 87<br>2020-04-29 08:52:49,607 140572 DEBUG tripleo_common.image.image_uploader [ ] <a href="https://registry-1.docker.io/v2/" target="_blank">https://registry-1.docker.io/v2/</a> status code 401<br>2020-04-29 08:52:49,607 140572 DEBUG tripleo_common.image.image_uploader [ ] Token parameters: params {'scope': 'repository:tripleotrain/centos-binary-swift-object:pull', 'service': '<a href="http://registry.docker.io" target="_blank">registry.docker.io</a>'}<br></div><div><br></div><div>Later I saw connectionpool retrying, but I have not seen "
tripleo_common.image.image_uploader" with same.</div></div><div><br></div><div>Every 2.0s: sudo podman ps -a ; sudo podman images -a ; sudo df -h Wed Apr 29 09:38:26 2020<br><br>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES<br>REPOSITORY TAG IMAGE ID CREATED SIZE<br><a href="http://docker.io/tripleotrain/centos-binary-nova-api" target="_blank">docker.io/tripleotrain/centos-binary-nova-api</a> current-tripleo e32831544953 2 days ago 1.39 GB<br><a href="http://docker.io/tripleotrain/centos-binary-glance-api" target="_blank">docker.io/tripleotrain/centos-binary-glance-api</a> current-tripleo edbb7dff6427 2 days ago 1.31 GB<br><a href="http://docker.io/tripleotrain/centos-binary-mistral-api" target="_blank">docker.io/tripleotrain/centos-binary-mistral-api</a> current-tripleo bcb3e95028a3 2 days ago 1.54 GB<br><a href="http://docker.io/tripleotrain/centos-binary-ironic-pxe" target="_blank">docker.io/tripleotrain/centos-binary-ironic-pxe</a> current-tripleo 2f1eb1da3fa4 2 days ago 909 MB<br><a href="http://docker.io/tripleotrain/centos-binary-heat-api" target="_blank">docker.io/tripleotrain/centos-binary-heat-api</a> current-tripleo b425da0e0a89 2 days ago 947 MB<br><a href="http://docker.io/tripleotrain/centos-binary-ironic-api" target="_blank">docker.io/tripleotrain/centos-binary-ironic-api</a> current-tripleo d0b670006bc6 2 days ago 903 MB<br><a href="http://docker.io/tripleotrain/centos-binary-swift-proxy-server" target="_blank">docker.io/tripleotrain/centos-binary-swift-proxy-server</a> current-tripleo 73432aea0d63 2 days ago 895 MB<br><a href="http://docker.io/tripleotrain/centos-binary-neutron-server" target="_blank">docker.io/tripleotrain/centos-binary-neutron-server</a> current-tripleo d7b8f19cc5ed 2 days ago 1.1 GB<br><a href="http://docker.io/tripleotrain/centos-binary-keystone" target="_blank">docker.io/tripleotrain/centos-binary-keystone</a> current-tripleo 8352bb3fd528 2 days ago 905 MB<br><a href="http://docker.io/tripleotrain/centos-binary-zaqar-wsgi" target="_blank">docker.io/tripleotrain/centos-binary-zaqar-wsgi</a> current-tripleo 49a7f0066616 2 days ago 894 MB<br><a href="http://docker.io/tripleotrain/centos-binary-placement-api" target="_blank">docker.io/tripleotrain/centos-binary-placement-api</a> current-tripleo 096ce1da63d3 2 days ago 1 GB<br><a href="http://docker.io/tripleotrain/centos-binary-ironic-inspector" target="_blank">docker.io/tripleotrain/centos-binary-ironic-inspector</a> current-tripleo 4505c408a230 2 days ago 817 MB<br><a href="http://docker.io/tripleotrain/centos-binary-rabbitmq" target="_blank">docker.io/tripleotrain/centos-binary-rabbitmq</a> current-tripleo bee62aacf8fb 2 days ago 700 MB<br><a href="http://docker.io/tripleotrain/centos-binary-haproxy" target="_blank">docker.io/tripleotrain/centos-binary-haproxy</a> current-tripleo 4b11e3d9c95f 2 days ago 692 MB<br><a href="http://docker.io/tripleotrain/centos-binary-mariadb" target="_blank">docker.io/tripleotrain/centos-binary-mariadb</a> current-tripleo 16cc78bc1e94 2 days ago 845 MB<br><a href="http://docker.io/tripleotrain/centos-binary-keepalived" target="_blank">docker.io/tripleotrain/centos-binary-keepalived</a> current-tripleo 67de7d2af948 2 days ago 568 MB<br><a href="http://docker.io/tripleotrain/centos-binary-memcached" target="_blank">docker.io/tripleotrain/centos-binary-memcached</a> current-tripleo a1019d76359c 2 days ago 561 MB<br><a href="http://docker.io/tripleotrain/centos-binary-iscsid" target="_blank">docker.io/tripleotrain/centos-binary-iscsid</a> current-tripleo c62bc10064c2 2 days ago 527 MB<br><a href="http://docker.io/tripleotrain/centos-binary-cron" target="_blank">docker.io/tripleotrain/centos-binary-cron</a> current-tripleo be0199eb5b89 2 days ago 522 MB<br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 28 Apr 2020 at 20:10, Alex Schultz <<a href="mailto:aschultz@redhat.com" target="_blank">aschultz@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, Apr 28, 2020 at 11:57 AM Ruslanas Gžibovskis <<a href="mailto:ruslanas@lpic.lt" target="_blank">ruslanas@lpic.lt</a>> wrote:<br>
><br>
> Hi all,<br>
><br>
> I am running a fresh install of rdo train, on centos7<br>
> I almost a week I am facing error at this step:<br>
> TASK [Run container-puppet tasks (generate config) during step 1]<br>
><br>
> So I have ansible.log attached, I cannot find anything, where it is failing.<br>
> According to some understanding in ansible, it fails if it finds stderr output.<br>
> I cannot find error/fail or smth, I see Notices and Warnings, but I believe it is not stderr?<br>
><br>
> I see containers running and removed after some time.<br>
> (as it should be I think)...<br>
><br>
> Could you help me, where to dig?<br>
><br>
<br>
2020-04-27 22:27:46,147 p=132230 u=root | TASK [Start containers for<br>
step 1 using paunch]<br>
*****************************************************************************************************************************<br>
2020-04-27 22:27:46,148 p=132230 u=root | Monday 27 April 2020<br>
22:27:46 +0200 (0:00:00.137) 0:04:44.326 **********?<br>
2020-04-27 22:27:46,816 p=132230 u=root | ok: [remote-u]<br>
2020-04-27 22:27:46,914 p=132230 u=root | TASK [Debug output for<br>
task: Start containers for step 1]<br>
*******************************************************************************************************************<br>
2020-04-27 22:27:46,915 p=132230 u=root | Monday 27 April 2020<br>
22:27:46 +0200 (0:00:00.767) 0:04:45.093 **********?<br>
2020-04-27 22:27:46,977 p=132230 u=root | fatal: [remote-u]: FAILED! => {<br>
"failed_when_result": true,?<br>
"outputs.stdout_lines | default([]) | union(outputs.stderr_lines |<br>
default([]))": []<br>
<br>
Check /var/log/paunch.log. It probably has additional information as<br>
to why the containers didn't start. You might also check the output<br>
of 'sudo podman ps -a' to see if any containers exited with errors.<br>
<br>
> --<br>
> Ruslanas Gžibovskis<br>
> +370 6030 7030<br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div>Ruslanas Gžibovskis<br>+370 6030 7030<br></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>Ruslanas Gžibovskis<br>+370 6030 7030<br></div></div></div>