Hi, Attaching the stdout for reference. With regards, Swogat Pradhan On Thu, May 16, 2024 at 6:31 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi, I am trying to run a stack update and am executing openstack overcloud deploy, but after reaching step 1 and prefetching the container images the deployment fails with following error:
s 2024-05-16 19:58:54.138764 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-cinder-backup:latest 2024-05-16 19:58:54.143338 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:18.464385 | 43.62s An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'config' s/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee st2024-05-16 19:58:55.487682 | 525400ff-fd40-2d4d-b69c-000000010b27 | FATAL | Manage Puppet containers (generate config) for step 1 | overcloud-novacompute-3 | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <module>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 391, in <module>\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 387, in main\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 132, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 370, in run\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 337, in check_failures\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 586, in result_iterator\n yield fs.pop().result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 425, in result\n return self.__get_result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} 2024-05-16 19:58:55.488510 | 525400ff-fd40-2d4d-b69c-000000010b27 | TIMING | Manage Puppet containers (generate config) for step 1 | overcloud-novacompute-3 | 0:23:19.809572 | 82.91s 2024-05-16 19:58:55.859707 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-cinder-volume:latest 2024-05-16 19:58:55.864370 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:20.185418 | 45.34s 2024-05-16 19:58:57.659959 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-iscsid:latest 2024-05-16 19:58:57.664429 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:21.985477 | 47.14s 2024-05-16 19:59:00.211286 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-manila-share:latest 2024-05-16 19:59:00.216236 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:24.537284 | 49.69s 2024-05-16 19:59:02.113037 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-aodh-evaluator:latest 2024-05-16 19:59:02.117498 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:26.438544 | 51.59s 2024-05-16 19:59:03.853383 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-aodh-listener:latest 2024-05-16 19:59:03.858232 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:28.179287 | 53.33s 2024-05-16 19:59:05.665169 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-aodh-notifier:latest 2024-05-16 19:59:05.670233 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:29.991278 | 55.14s An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'config' /futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\2024-05-16 19:59:06.648301 | 525400ff-fd40-2d4d-b69c-000000010fc3 | FATAL | Manage Puppet containers (generate config) for step 1 | overcloud-controller-no-ceph-3 | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <module>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 391, in <module>\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 387, in main\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 132, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 370, in run\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 337, in check_failures\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 586, in result_iterator\n yield fs.pop().result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 425, in result\n return self.__get_result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
I do not have any idea why this is caused and how to fix it. Can someone please help?
With regards, Swogat Pradhan