Hi Team,

we were trying to deploy OpenStack wallaby, and we see that 4 out of 5 runs deployment is getting failed for mentioned below reasons:

Error:

2022-08-25 17:34:29.036371 | 5254004d-021e-d4db-067d-000000007b1a | TASK | Create identity internal endpoint
2022-08-25 17:34:31.176105 | 5254004d-021e-d4db-067d-000000007b1a | FATAL | Create identity internal endpoint | undercloud | error={"changed": false, "extra_data": {"data": null, "details": "The request you have made requires authentication.", "response": "{\"error\":{\"code\":401,\"message\":\"The request you have made requires authentication.\",\"title\":\"Unauthorized\"}}\n"}, "msg": "Failed to list services: Client Error for url: overcloud-public.myhsc.com :13000/v3/services, The request you have made requires authentication."}

Debug logs:

pending results....
The full traceback is:
File "/tmp/ansible_openstack.cloud.endpoint_payload_qhlqb_qw/ansible_openstack.cloud.endpoint_payload.zip/ansible_collections/open stack/cloud/plugins/module_utils/openstack.py", line 407, in __call__
results = self.run()
File "/tmp/ansible_openstack.cloud.endpoint_payload_qhlqb_qw/ansible_openstack.cloud.endpoint_payload.zip/ansible_collections/open stack/cloud/plugins/modules/endpoint.py", line 150, in run
File "/usr/lib/python3.6/site-packages/openstack/cloud/_identity.py", line 537, in get_service
return _utils._get_entity(self, 'service', name_or_id, filters)
File "/usr/lib/python3.6/site-packages/openstack/cloud/_utils.py", line 197, in _get_entity
entities = search(name_or_id, filters, **kwargs)
File "/usr/lib/python3.6/site-packages/openstack/cloud/_identity.py", line 517, in search_services
services = self.list_services()
File "/usr/lib/python3.6/site-packages/openstack/cloud/_identity.py", line 501, in list_services
error_message="Failed to list services")
File "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in get
return self.request(url, 'GET', **kwargs)
File "/usr/lib/python3.6/site-packages/openstack/proxy.py", line 668, in request
return _json_response(response, error_message=error_message)
File "/usr/lib/python3.6/site-packages/openstack/proxy.py", line 646, in _json_response
exceptions.raise_from_response(response, error_message=error_message)
File "/usr/lib/python3.6/site-packages/openstack/exceptions.py", line 238, in raise_from_response
http_status=http_status, request_id=request_id
2022-08-25 19:38:03.201899 | 5254004d-021e-7578-cd65-000000007ad6 | FATAL | Create identity internal endpoint | undercloud | er ror={
"changed": false,
"extra_data": {
"data": null,
"details": "The request you have made requires authentication.",
"response": "{\"error\":{\"code\":401,\"message\":\"The request you have made requires authentication.\",\"title\":\"Unautho rized\"}}\n"
},
"invocation": {
"module_args": {
"api_timeout": null,
"auth": null,
"auth_type": null,
"availability_zone": null,
"ca_cert": null,
"client_cert": null,
"client_key": null,
"enabled": true,
"endpoint_interface": "internal",
"interface": "public",
"region": "regionOne",
"region_name": null,
"service": "keystone",
"state": "present",
"timeout": 180,
"url": "http://[fd00:fd00:fd00:2000::368]:5000",
"validate_certs": null,
"wait": true
}
},
"msg": "Failed to list services: Client Error for url: https://overcloud-public.myhsc.com:13000/v3/services, The request you have made requires authentication."
}

DeployCommand that was used:

openstack overcloud deploy --templates \
-r /home/stack/templates/roles_data.yaml \
-n /home/stack/templates/custom_network_data.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/ptp.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-dns.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/inject-trust-anchor-hiera.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/external-ceph.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \
-e /home/stack/templates/overcloud-baremetal-deployed.yaml \
-e /home/stack/templates/networks-deployed-environment.yaml \
-e /home/stack/templates/vip-deployed-environment.yaml \
-e /home/stack/templates/environment.yaml \
-e /home/stack/templates/ironic-config.yaml \
-e /home/stack/templates/enable-tls.yaml \
-e /home/stack/templates/cloudname.yaml \
-e /home/stack/templates/my-additional-ceph-settings.yaml \
-e /home/stack/containers-prepare-parameter.yaml

[Issue]

we are seeing the deployment is working perfectly once and the same setting is not working perfectly in the second run.

the failure rate is high.

what can be the reasons behind this?

Note:

Before this, we were deploying with DNS and SSL and it was perfectly working fine in multiple reruns.

But after SSL we have seen this random failure.

~ Lokendra

skype: lokendrarathour