[Wallaby] Deployment getting failed Randomly

Lokendra Rathour lokendrarathour at gmail.com
Fri Aug 26 06:25:18 UTC 2022


Hi Team,
we were trying to deploy OpenStack wallaby, and we see that 4 out of 5 runs
deployment is getting failed for mentioned below reasons:
*Error:*

2022-08-25 17:34:29.036371 | 5254004d-021e-d4db-067d-000000007b1a |       TASK
| Create identity internal endpoint
2022-08-25 17:34:31.176105 | 5254004d-021e-d4db-067d-000000007b1a |      FATAL
| Create identity internal endpoint | undercloud | error={"changed": false,
"extra_data": {"data": null, "details": "The request you have made requires
authentication.", "response": "{\"error\":{\"code\":401,\"message\":\"The
request you have made requires authentication.\",\"title\":\"Unauthorized\"}
}\n"}, "msg": "Failed to list services: Client Error for url:
overcloud-public.myhsc.com
<https://overcloud-public.myhsc.com:13000/v3/services> :13000/v3/services
<https://overcloud-public.myhsc.com:13000/v3/services>, The request you
have made requires authentication."}


Debug logs:

pending results....
The full traceback is:
  File
"/tmp/ansible_openstack.cloud.endpoint_payload_qhlqb_qw/ansible_openstack.cloud.endpoint_payload.zip/ansible_collections/open
         stack/cloud/plugins/module_utils/openstack.py", line 407, in
__call__
    results = self.run()
  File
"/tmp/ansible_openstack.cloud.endpoint_payload_qhlqb_qw/ansible_openstack.cloud.endpoint_payload.zip/ansible_collections/open
         stack/cloud/plugins/modules/endpoint.py", line 150, in run
  File "/usr/lib/python3.6/site-packages/openstack/cloud/_identity.py",
line 537, in get_service
    return _utils._get_entity(self, 'service', name_or_id, filters)
  File "/usr/lib/python3.6/site-packages/openstack/cloud/_utils.py", line
197, in _get_entity
    entities = search(name_or_id, filters, **kwargs)
  File "/usr/lib/python3.6/site-packages/openstack/cloud/_identity.py",
line 517, in search_services
    services = self.list_services()
  File "/usr/lib/python3.6/site-packages/openstack/cloud/_identity.py",
line 501, in list_services
    error_message="Failed to list services")
  File "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line
395, in get
    return self.request(url, 'GET', **kwargs)
  File "/usr/lib/python3.6/site-packages/openstack/proxy.py", line 668, in
request
    return _json_response(response, error_message=error_message)
  File "/usr/lib/python3.6/site-packages/openstack/proxy.py", line 646, in
_json_response
    exceptions.raise_from_response(response, error_message=error_message)
  File "/usr/lib/python3.6/site-packages/openstack/exceptions.py", line
238, in raise_from_response
    http_status=http_status, request_id=request_id
2022-08-25 19:38:03.201899 | 5254004d-021e-7578-cd65-000000007ad6 |
 FATAL | Create identity internal endpoint | undercloud | er          ror={
    "changed": false,
    "extra_data": {
        "data": null,
        "details": "The request you have made requires authentication.",
        "response": "{\"error\":{\"code\":401,\"message\":\"The request you
have made requires authentication.\",\"title\":\"Unautho
 rized\"}}\n"
    },
    "invocation": {
        "module_args": {
            "api_timeout": null,
            "auth": null,
            "auth_type": null,
            "availability_zone": null,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "enabled": true,
            "endpoint_interface": "internal",
            "interface": "public",
            "region": "regionOne",
            "region_name": null,
            "service": "keystone",
            "state": "present",
            "timeout": 180,
            "url": "http://[fd00:fd00:fd00:2000::368]:5000",
            "validate_certs": null,
            "wait": true
        }
    },
    "msg": "Failed to list services: Client Error for url:
https://overcloud-public.myhsc.com:13000/v3/services, The request you have
made requires authentication."
}

DeployCommand that was used:

 openstack overcloud deploy --templates \
    -r /home/stack/templates/roles_data.yaml \
    -n /home/stack/templates/custom_network_data.yaml \
    -e
/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml
\
    -e
/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml
\
    -e
/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml
\
    -e
/usr/share/openstack-tripleo-heat-templates/environments/services/ptp.yaml \
    -e
/usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-dns.yaml
\
    -e
/usr/share/openstack-tripleo-heat-templates/environments/ssl/inject-trust-anchor-hiera.yaml
\
    -e
/usr/share/openstack-tripleo-heat-templates/environments/external-ceph.yaml
\
    -e
/usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml
\
    -e /home/stack/templates/overcloud-baremetal-deployed.yaml \
    -e /home/stack/templates/networks-deployed-environment.yaml \
    -e /home/stack/templates/vip-deployed-environment.yaml \
    -e /home/stack/templates/environment.yaml \
    -e /home/stack/templates/ironic-config.yaml \
    -e /home/stack/templates/enable-tls.yaml \
    -e /home/stack/templates/cloudname.yaml \
    -e /home/stack/templates/my-additional-ceph-settings.yaml \
    -e /home/stack/containers-prepare-parameter.yaml

[Issue]
we are seeing the deployment is working perfectly once and the same setting
is not working perfectly in the second run.
the failure rate is high.

what can be the reasons behind this?

*Note:*
Before this, we were deploying with DNS and SSL and it was perfectly
working fine in multiple reruns.
But after SSL we have seen this random failure.


-- 
~ Lokendra
skype: lokendrarathour
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220826/0ab2d183/attachment-0001.htm>


More information about the openstack-discuss mailing list