Hello, i have a Kolla-Ansible AIO stack i added SWIFT to after the initial installation. CLI management of containers work on CLI, but Horizon is troubled: Clicking "container" throws "Error: Unable to fetch the policy details", and no horizon based container management is possible therefore. In an effort to find the root problem, i stumbled upon the following, which i assume to be related: it looks to me like there are issues with keystone authentication of SWIFT client (3.10.1) when attempting to fetch capabilities on OS Victoria Kolla-Ansible, but *only* when querying capabilities: <EXAMPLES> ## 1 (swift-container-server)[root@testhost ~]# swift --os-auth-url http://10.10.10.82:35357/v3 --auth-version 3 --os-project-name admin --os-project-domain-name Default --os-username admin --os-user-domain-name Default --os-password T-STRING-s6 stat Account: AUTH_4334f282aa3a498aac95d2cf9aa9fa91 Containers: 1 Objects: 2 Bytes: 0 Containers in policy "policy-0": 1 Objects in policy "policy-0": 2 Bytes in policy "policy-0": 0 Content-Type: text/plain; charset=utf-8 X-Timestamp: 1616964437.15820 Accept-Ranges: bytes X-Account-Project-Domain-Id: default Vary: Accept X-Trans-Id: tx2638af9bd98646ce92dfe-00606342a1 X-Openstack-Request-Id: tx2638af9bd98646ce92dfe-00606342a1 Transfer-Encoding: chunked ## 2 (swift-container-server)[root@testhost ~]# swift --os-auth-url http://10.10.10.82:35357/v3 --auth-version 3 --os-project-name admin --os-project-domain-name Default --os-username admin --os-user-domain-name Default --os-password T-STRING-s6 list mycontainer ## 3 (swift-container-server)[root@testhost ~]# swift --os-auth-url http://10.10.10.82:35357/v3 --auth-version 3 --os-project-name admin --os-project-domain-name Default --os-username admin --os-user-domain-name Default --os-password T-STRING-s6 capabilities Capabilities GET failed: http://10.10.10.82:8080/info 401 Unauthorized [first 60 chars of response] b'{"error": {"code": 401, "title": "Unauthorized", "message": ' Failed Transaction ID: txeb57fccc09e54e3fa2fa5-00606342af </EXAMPLES> Am i on the wrong track? Best Stefan Here is the entire debug output of the failing transaction: (swift-container-server)[root@testhost ~]# swift --os-auth-url http://10.10.10.82:35357/v3 --auth-version 3 --os-project-name admin --os-project-domain-name Default --os-username admin --os-user-domain-name Default --os-password T-STRING-s6 capabilities --debug DEBUG:keystoneclient.auth.identity.v3.base:Making authentication request to http://10.10.10.82:35357/v3/auth/tokens DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 10.10.10.82:35357 DEBUG:urllib3.connectionpool:http://10.10.10.82:35357 "POST /v3/auth/tokens HTTP/1.1" 201 7299 DEBUG:keystoneclient.auth.identity.v3.base:{"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "12940206c0a1464f96a47fe74d19c893", "name": "admin", "password_expires_at": null}, "audit_ids": ["WjAnZyn5TaGufxZGqnvW3Q"], "expires_at": "2021-03-31T14:22:13.000000Z", "issued_at": "2021-03-30T14:22:13.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "4334f282aa3a498aac95d2cf9aa9fa91", "name": "admin"}, "is_domain": false, "roles": [{"id": "b740e2667f034d02b6af2114ec06e20a", "name": "admin"}, {"id": "ceb68a7214c74a969cd83b9146e1fbda", "name": "heat_stack_owner"}, {"id": "6b2ae10793c34fb287dd994a8dd2e8a1", "name": "member"}, {"id": "edb76a75b2fc4af79bf09e38e07be5f1", "name": "reader"}], "catalog": [{"endpoints": [{"id": "05e0b6b67f094b64b1caa9cf78dd34b2", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8780", "region": "RegionOne"}, {"id": "9575e60e3a034ff282bbeda2287b3d01", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8780", "region": "RegionOne"}, {"id": "f5fb2fffde5d4b88b230397560ac6952", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8780", "region": "RegionOne"}], "id": "16edbecc93484568a2f5b46db9909293", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "09275022c3de4405adb6bb25071a49ba", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:9292", "region": "RegionOne"}, {"id": "3a6fa026c7c9422f8cb013492f5ba0e0", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:9292", "region": "RegionOne"}, {"id": "a4fa0d097eed4c3b97ff4ed0f27096d6", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:9292", "region": "RegionOne"}], "id": "2a71f9f773734edb8ea6b711c844b334", "type": "image", "name": "glance"}, {"endpoints": [{"id": "a20fbce4737540ce91039175c7409811", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8774/v2.1", "region": "RegionOne"}, {"id": "b16bad7fdf1342329a59e231cf56851d", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8774/v2.1", "region": "RegionOne"}, {"id": "ce27e7239706458c80e42a68a13383b2", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8774/v2.1", "region": "RegionOne"}], "id": "2c56db642dde46a2ab2f766dbd0ba9c9", "type": "compute", "name": "nova"}, {"endpoints": [{"id": "8c183a6cd8244b2cb1630a662c3a292e", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8004/v1/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "958487dfae194f69b5f56f0b6e901163", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8004/v1/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "a8e0db3b2e1c49a99ed97ec18a64ece6", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8004/v1/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}], "id": "5428e2837ccf413886140c0603867ee3", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "2e9a6a2fc78340b7bdc18041dec79a86", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8774/v2/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "894ab56c6c7140ebaac8e7ec502e9b03", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8774/v2/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "8e86543bfeb547d78cf707100d0ef7f0", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8774/v2/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}], "id": "5639441ae48441ed9c8163fc537f0c1a", "type": "compute_legacy", "name": "nova_legacy"}, {"endpoints": [{"id": "32092c9ac4634ece9d89edc2b146af54", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:9696", "region": "RegionOne"}, {"id": "dad19d0e0d8e420caa5b1276d82917e4", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:9696", "region": "RegionOne"}, {"id": "f085e59dae874005b8e6fb1df55db5d5", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:9696", "region": "RegionOne"}], "id": "56e63ef1e3b24fbca6a2efb7394eb77b", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "0ccb4189837f482aac1ee67a0f5b69b7", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8000/v1", "region": "RegionOne"}, {"id": "be4fab7e4ae04786817702817ea09c34", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8000/v1", "region": "RegionOne"}, {"id": "eb8ccdc41ab94aaa9567e29e728e9109", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8000/v1", "region": "RegionOne"}], "id": "83cee8b745d74b1d9a3bc1e37d3e21bf", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "917c36b07fae4b278a2f63d3886933c3", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:5000", "region": "RegionOne"}, {"id": "9d40ecfea4a04327b64e5751ed8043ce", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:5000", "region": "RegionOne"}, {"id": "d4941b09a2cb40648f3b019bb1653120", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:35357", "region": "RegionOne"}], "id": "94dce13398fd42e4805ae1a7d9976e78", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "10297840af2b4370a2f63908aea3ff9c", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8776/v2/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "a1a617f8996d439b8de091a4f15a88ae", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8776/v2/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "a930062cb2da497bbeb727c9fec6bba0", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8776/v2/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}], "id": "d7f1cd74b2294126ac939b7d705369a3", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"id": "1bb18181af9b4dd28bbf16491adb16fe", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8080/v1/AUTH_4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "686b82631d4d40b1b62ed1a7cb8fb1cf", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8080/v1", "region": "RegionOne"}, {"id": "e0cc80d07c6246668ca9cce9d736af62", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8080/v1/AUTH_4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}], "id": "eabcc4d34b3c4ce499025816e2302f34", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "4c2b1dbe8d3544c9add475d63b0492d4", "interface": "public", "region_id": "RegionOne", "url": "http://10.10.10.82:8776/v3/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "77634a04b3644d0eb2abab41f3325a1c", "interface": "internal", "region_id": "RegionOne", "url": "http://10.10.10.82:8776/v3/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}, {"id": "7f11d1c28644413e8cfe5ff19b91ebd2", "interface": "admin", "region_id": "RegionOne", "url": "http://10.10.10.82:8776/v3/4334f282aa3a498aac95d2cf9aa9fa91", "region": "RegionOne"}], "id": "fda50e1a62564367bca385a64088334d", "type": "volumev3", "name": "cinderv3"}]}} DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 10.10.10.82:8080 DEBUG:urllib3.connectionpool:http://10.10.10.82:8080 "GET /info HTTP/1.1" 401 114 INFO:swiftclient:REQ: curl -i http://10.10.10.82:8080/info -X GET -H "Accept-Encoding: gzip" INFO:swiftclient:RESP STATUS: 401 Unauthorized INFO:swiftclient:RESP HEADERS: {'Content-Type': 'application/json', 'Content-Length': '114', 'WWW-Authenticate': 'Keystone uri="http://10.10.10.82:5000"', 'X-Trans-Id': 'txac041bfff30342729c2dd-0060633415', 'X-Openstack-Request-Id': 'txac041bfff30342729c2dd-0060633415', 'Date': 'Tue, 30 Mar 2021 14:22:13 GMT'} INFO:swiftclient:RESP BODY: b'{"error": {"code": 401, "title": "Unauthorized", "message": "The request you have made requires authentication."}}' Capabilities GET failed: http://10.10.10.82:8080/info 401 Unauthorized [first 60 chars of response] b'{"error": {"code": 401, "title": "Unauthorized", "message": ' Failed Transaction ID: txac041bfff30342729c2dd-0060633415