Antelope Barbican/Keystone: failure to delete secrets (Keystone TrustNotFound)
Hi,
I've been validating Yoga -> Antelope on our preprod cloud. It was working successfully but "suddenly" (i.e. without a kwown/identified change!) when deleting Magnum clusters, the deletion of Barbican secrets (after stack deletion) fail. In the Magnum log, I find the following error:
----
Failed to delete trust: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a4400250-e8d5-46ec-a28a-dcb97a682512)
(traceback skipped for readability)
2024-05-29 10:43:09.750 2078 INFO magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-173... from Barbican. 2024-05-29 10:43:09.750 2078 INFO barbicanclient.base [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Calculated Containers uuid ref: containers/edd8487a-6cf1-4f4c-9f53-173dfae7a8b2 2024-05-29 10:43:09.760 2078 ERROR barbicanclient.client [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] 4xx Client error: Not Found: Secrets container not found. 2024-05-29 10:43:09.761 2078 ERROR magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Error recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-173...: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. ----
In Keystone log, I find with the same timestamp the following error that seems related to the one above:
----
2024-05-29 10:43:09.690 1713 WARNING keystone.api._shared.authentication [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Could not find trust: cf4b1b6082594025a1ff9e48df383566.: keystone.exception.TrustNotFound: Could not find trust: cf4b1b6082594025a1ff9e48df383566. 2024-05-29 10:43:09.693 1713 WARNING keystone.server.flask.application [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Authorization failed. The request you have made requires authentication. from 134.158.77.23: keystone.exception.Unauthorized: The request you have made requires authentication. 2024-05-29 10:43:09.746 1723 ERROR keystone.server.flask.application [None req-8c76f304-c2a3-418c-8372-02c706ed356e 44d8cb3192b3485790ababe29991491a - 5b67a8641dd041089e018ceea9884eac - 5b67a8641dd041089e018ceea9884eac -] Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.: keystone.exception.UserNotFound: Could not find user: 527ec32509d849e8a49b41c3dc4c4c32. ----
It is not clear for me if the ERROR is releated to the 2 warnings as the request is not the same but it seems to happen every time this way.
In Barbican logs, there is not much things related except a 401 status for the request corresponding ot the TrustNotFound.
I'm looking for advices on how to troubleshoot this. In particular, when the trust should be created and whether I should find somewhere a matching error at the time of the trust creation. It seems the error is only affecting deletion but I may be wrong.
for the record, when starting Magnum there is an error complaining about /etc/magnum/keystone_auth_default_policy.json being not found but ignored it so far. And in Keystone an error seems to occur at Magnum cluster creation time:
---
2024-05-29 10:17:17.530 1711 WARNING py.warnings [None req-f484cc35-3e54-4d7f-8688-e6442c90344f a22b1199c74f4a819a49f3ac037a5e44 1dd67113c8bb4ef0a31dfca53335ffbd - - defaul t default] /usr/lib/python3.9/site-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is re quired ----
Thanks in advance for any hint. Best regards,
Michel
Sorry for a mistake in the previous message. The Magnum message about the missing policy file appears at cluster creation time, not as service startup time. But again, not sure it is harmful...
Michel
Le 29/05/2024 à 14:36, Michel Jouvin a écrit :
Hi,
I've been validating Yoga -> Antelope on our preprod cloud. It was working successfully but "suddenly" (i.e. without a kwown/identified change!) when deleting Magnum clusters, the deletion of Barbican secrets (after stack deletion) fail. In the Magnum log, I find the following error:
Failed to delete trust: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a4400250-e8d5-46ec-a28a-dcb97a682512)
(traceback skipped for readability)
2024-05-29 10:43:09.750 2078 INFO magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-173... from Barbican. 2024-05-29 10:43:09.750 2078 INFO barbicanclient.base [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Calculated Containers uuid ref: containers/edd8487a-6cf1-4f4c-9f53-173dfae7a8b2 2024-05-29 10:43:09.760 2078 ERROR barbicanclient.client [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] 4xx Client error: Not Found: Secrets container not found. 2024-05-29 10:43:09.761 2078 ERROR magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Error recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-173...: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found.
In Keystone log, I find with the same timestamp the following error that seems related to the one above:
2024-05-29 10:43:09.690 1713 WARNING keystone.api._shared.authentication [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Could not find trust: cf4b1b6082594025a1ff9e48df383566.: keystone.exception.TrustNotFound: Could not find trust: cf4b1b6082594025a1ff9e48df383566. 2024-05-29 10:43:09.693 1713 WARNING keystone.server.flask.application [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Authorization failed. The request you have made requires authentication. from 134.158.77.23: keystone.exception.Unauthorized: The request you have made requires authentication. 2024-05-29 10:43:09.746 1723 ERROR keystone.server.flask.application [None req-8c76f304-c2a3-418c-8372-02c706ed356e 44d8cb3192b3485790ababe29991491a - 5b67a8641dd041089e018ceea9884eac - 5b67a8641dd041089e018ceea9884eac -] Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.: keystone.exception.UserNotFound: Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.
It is not clear for me if the ERROR is releated to the 2 warnings as the request is not the same but it seems to happen every time this way.
In Barbican logs, there is not much things related except a 401 status for the request corresponding ot the TrustNotFound.
I'm looking for advices on how to troubleshoot this. In particular, when the trust should be created and whether I should find somewhere a matching error at the time of the trust creation. It seems the error is only affecting deletion but I may be wrong.
for the record, when starting Magnum there is an error complaining about /etc/magnum/keystone_auth_default_policy.json being not found but ignored it so far. And in Keystone an error seems to occur at Magnum cluster creation time:
2024-05-29 10:17:17.530 1711 WARNING py.warnings [None req-f484cc35-3e54-4d7f-8688-e6442c90344f a22b1199c74f4a819a49f3ac037a5e44 1dd67113c8bb4ef0a31dfca53335ffbd - - defaul t default] /usr/lib/python3.9/site-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is re quired
Thanks in advance for any hint. Best regards,
Michel
On 29/5/2024 10:36 pm, Michel Jouvin wrote:
In Keystone log, I find with the same timestamp the following error that seems related to the one above:
2024-05-29 10:43:09.690 1713 WARNING keystone.api._shared.authentication [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Could not find trust: cf4b1b6082594025a1ff9e48df383566.: keystone.exception.TrustNotFound: Could not find trust: cf4b1b6082594025a1ff9e48df383566. 2024-05-29 10:43:09.693 1713 WARNING keystone.server.flask.application [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Authorization failed. The request you have made requires authentication. from 134.158.77.23: keystone.exception.Unauthorized: The request you have made requires authentication. 2024-05-29 10:43:09.746 1723 ERROR keystone.server.flask.application [None req-8c76f304-c2a3-418c-8372-02c706ed356e 44d8cb3192b3485790ababe29991491a - 5b67a8641dd041089e018ceea9884eac - 5b67a8641dd041089e018ceea9884eac -] Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.: keystone.exception.UserNotFound: Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.
Wild guess, was the trust deleted before the secret? If so, how are you creating and deleting these secrets?
Are you able to trace where the calls are happening? Maybe some dependency is missing in the deletion pathway.
Regards, Jake
Hi Jake,
I had more urgent issues to fix and have not looked more to this one, except that today I checked the problem is already present in Yoga on our production cluster so it is not linked to the Antelope upgrade, conversely to what I thought.
This error happens when deleting Magnum clusters and the clusters were also created by Magnum. I have no clue what is done in which order, I don't do anything special as far as I can tell.
The full Antelope traceback corresponding to the problem is below (the Yoga one looks very similar with different paths due to different Python versions).
Best regards,
Michel
--------------------------------------------
2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Failed to delete trust: keystoneauth1.exceptions.htt p.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-9cce561e-1b7e-4b30-84ae-bddbb2a3b01a) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone Traceback (most recent call last): 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/magnum/common/keystone.py", line 246, in delete_trust 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone client.trusts.delete(cluster.trust_id) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneclient/v3/contrib/trusts.py", line 102, in delete 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return super(TrustManager, self).delete(trust_id=base.getid(trust)) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneclient/base.py", line 86, in func 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return f(*args, **new_kwargs) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneclient/base.py", line 479, in delete 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return self._delete( 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneclient/base.py", line 246, in _delete 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone resp, body = self.client.delete(url, **kwargs) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/adapter.py", line 410, in delete 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return self.request(url, 'DELETE', **kwargs) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/adapter.py", line 554, in request 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone resp = super(LegacyJsonAdapter, self).request(*args, **kwargs) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/adapter.py", line 257, in request 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return self.session.request(url, method, **kwargs) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 790, in request 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone auth_headers = self.get_auth_headers(auth) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 1201, in get_auth_headers 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return auth.get_headers(self, **kwargs) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/plugin.py", line 106, in get_headers 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone token = self.get_token(session) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/base.py", line 88, in get_token 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return self.get_access(session).auth_token 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/base.py", line 134, in get_access 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone self.auth_ref = self.get_auth_ref(session) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone resp = session.post(token_url, json=body, headers=headers, 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 1149, in post 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone return self.request(url, 'POST', **kwargs) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 986, in request 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone raise exceptions.from_response(resp, method, url) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-9cce561e-1b7e-4b30-84ae-bddbb2a3b01a) 2024-05-31 08:45:51.920 280026 ERROR magnum.common.keystone 2024-05-31 08:45:51.955 280026 INFO magnum.common.cert_manager.barbican_cert_manager [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/8b27c465-19b1-48c0-8018-001... from Barbican. 2024-05-31 08:45:51.956 280026 INFO barbicanclient.base [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Calculated Containers uuid ref: containers/8b27c465-19b1-48c0-8018-0018bb4be077 2024-05-31 08:45:51.965 280026 ERROR barbicanclient.client [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] 4xx Client error: Not Found: Secrets container not found. 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Error recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/8b27c465-19b1-48c0-8018-001...: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager Traceback (most recent call last): 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/magnum/common/cert_manager/barbican_cert_manager.py", line 198, in delete_cert 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager certificate_container = connection.containers.get(cert_ref) 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/v1/containers.py", line 543, in get 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager response = self._api.get(uuid_ref) 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 70, in get 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager return super(_HTTPClient, self).get(*args, **kwargs).json() 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/keystoneauth1/adapter.py", line 395, in get 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager return self.request(url, 'GET', **kwargs) 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 63, in request 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager self._check_status_code(resp) 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 105, in _check_status_code 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager raise exceptions.HTTPClientError( 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. 2024-05-31 08:45:51.966 280026 ERROR magnum.common.cert_manager.barbican_cert_manager 2024-05-31 08:45:51.967 280026 WARNING magnum.conductor.handlers.common.cert_manager [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Deleting certs is failed for Cluster 18e51e7c-0a31-4b4c-a4fd-8142ac9f877d: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. 2024-05-31 08:45:51.967 280026 INFO magnum.common.cert_manager.barbican_cert_manager [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/e90f62f5-6df9-4d51-b3dd-61e... from Barbican. 2024-05-31 08:45:51.967 280026 INFO barbicanclient.base [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Calculated Containers uuid ref: containers/e90f62f5-6df9-4d51-b3dd-61e6289812de 2024-05-31 08:45:51.976 280026 ERROR barbicanclient.client [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] 4xx Client error: Not Found: Secrets container not found. 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Error recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/e90f62f5-6df9-4d51-b3dd-61e...: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager Traceback (most recent call last): 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/magnum/common/cert_manager/barbican_cert_manager.py", line 198, in delete_cert 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager certificate_container = connection.containers.get(cert_ref) 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/v1/containers.py", line 543, in get 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager response = self._api.get(uuid_ref) 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 70, in get 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager return super(_HTTPClient, self).get(*args, **kwargs).json() 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/keystoneauth1/adapter.py", line 395, in get 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager return self.request(url, 'GET', **kwargs) 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 63, in request 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager self._check_status_code(resp) 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 105, in _check_status_code 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager raise exceptions.HTTPClientError( 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. 2024-05-31 08:45:51.977 280026 ERROR magnum.common.cert_manager.barbican_cert_manager 2024-05-31 08:45:51.978 280026 WARNING magnum.conductor.handlers.common.cert_manager [None req-d9125739-f2e0-404f-953e-fe461578f2fd - - - - - -] Deleting certs is failed for Cluster 18e51e7c-0a31-4b4c-a4fd-8142ac9f877d: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found.
Le 05/06/2024 à 15:44, Jake Yip a écrit :
On 29/5/2024 10:36 pm, Michel Jouvin wrote:
In Keystone log, I find with the same timestamp the following error that seems related to the one above:
2024-05-29 10:43:09.690 1713 WARNING keystone.api._shared.authentication [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Could not find trust: cf4b1b6082594025a1ff9e48df383566.: keystone.exception.TrustNotFound: Could not find trust: cf4b1b6082594025a1ff9e48df383566. 2024-05-29 10:43:09.693 1713 WARNING keystone.server.flask.application [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Authorization failed. The request you have made requires authentication. from 134.158.77.23: keystone.exception.Unauthorized: The request you have made requires authentication. 2024-05-29 10:43:09.746 1723 ERROR keystone.server.flask.application [None req-8c76f304-c2a3-418c-8372-02c706ed356e 44d8cb3192b3485790ababe29991491a - 5b67a8641dd041089e018ceea9884eac - 5b67a8641dd041089e018ceea9884eac -] Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.: keystone.exception.UserNotFound: Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.
Wild guess, was the trust deleted before the secret? If so, how are you creating and deleting these secrets?
Are you able to trace where the calls are happening? Maybe some dependency is missing in the deletion pathway.
Regards, Jake
participants (2)
-
Jake Yip
-
Michel Jouvin