Re: [glance][openstack-ansible] Snapshots disappear during saving
Oh, I have a guess what this might actually be. During snapshot upload process user token that is used for the upload might get expired. If that's the case, following changes in user_variables might help to resolve the issue:
glance_glance_api_conf_overrides: keystone_authtoken: service_token_roles_required: True service_token_roles: service
I found out that after inserting the above the problems now occurs every time I try to take a snapshot of an instance. Below are my logs: For swift: Mar 29 12:50:37 infra1-swift-proxy-container-27169fa7 proxy-server[2049]: Could not autocreate account '/AUTH_024cc551782f41e395d3c9f13582ef7d' (txn: tx340e7873c3 ce42ab9808e-006061cd0f) Mar 29 12:50:37 infra1-swift-proxy-container-27169fa7 proxy-server[2049]: 192.168.110.106 192.168.110.211 29/Mar/2021/12/50/37 PUT /v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_images HTTP/1.0 50 3 - python-swiftclient-3.10.1 gAAAAABgYczvODdX... - 118 - tx340e7873c3ce42ab9808e-006061cd0f - 14.3464 - - 1617022223.371025085 1617022237.717381477 - Mar 29 12:50:40 infra1-swift-proxy-container-27169fa7 proxy-server[2053]: ERROR with Account server 10.0.3.212:6002/os-objects re: Trying to HEAD /v1/AUTH_024cc551782f41e395d 3c9f13582ef7d: Host unreachable (txn: tx44e98ea667f04751bf22e-006061cd1f) Mar 29 12:50:40 infra1-swift-proxy-container-27169fa7 proxy-server[2053]: Account HEAD returning 503 for [] (txn: tx44e98ea667f04751bf22e-006061cd1f) Mar 29 12:50:40 infra1-swift-proxy-container-27169fa7 proxy-server[2053]: - - 29/Mar/2021/12/50/40 HEAD /v1/AUTH_024cc551782f41e395d3c9f13582ef7d%3Fformat%3Djson HTTP/1.0 503 - Swift - - - - tx44e98 ea667f04751bf22e-006061cd1f - 1.0482 RL - 1617022239.738376379 1617022240.786619186 - and for glance: Mar 29 12:50:56 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:50:56.155 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP STATUS: 503 Service Unavailable Mar 29 12:50:56 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:50:56.156 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP HEADERS: {'content-type': 'text/html; charset=UTF-8', 'content-length': '118', 'x-trans-id': 'tx44e98ea667f04751bf22e-006061cd1f', 'x-openstack-request-id': 'tx44e98ea667f04751bf22e-006061cd1f', 'date': 'Mon, 29 Mar 2021 12:50:56 GMT', 'connection': 'close'} Mar 29 12:50:56 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:50:56.157 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP BODY: b'<html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>' Mar 29 12:51:15 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:15.544 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] REQ: curl -i http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... -X PUT -H "X-Auth-Token: gAAAAABgYczvODdX..." -H "Content-Length: 0" Mar 29 12:51:15 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:15.545 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP STATUS: 503 Service Unavailable Mar 29 12:51:15 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:15.545 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP HEADERS: {'content-type': 'text/html; charset=UTF-8', 'content-length': '118', 'x-trans-id': 'tx01d5fb0cb5cb4f6dbff12-006061cd34', 'x-openstack-request-id': 'tx01d5fb0cb5cb4f6dbff12-006061cd34', 'date': 'Mon, 29 Mar 2021 12:51:15 GMT', 'connection': 'close'} Mar 29 12:51:15 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:15.545 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP BODY: b'<html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>' Mar 29 12:51:42 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:42.007 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] REQ: curl -i http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... -X PUT -H "X-Auth-Token: gAAAAABgYczvODdX..." -H "Content-Length: 0" Mar 29 12:51:42 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:42.008 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP STATUS: 503 Service Unavailable Mar 29 12:51:42 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:42.009 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP HEADERS: {'content-type': 'text/html; charset=UTF-8', 'content-length': '118', 'x-trans-id': 'tx9cab391639a344f4aa548-006061cd4b', 'x-openstack-request-id': 'tx9cab391639a344f4aa548-006061cd4b', 'date': 'Mon, 29 Mar 2021 12:51:42 GMT', 'connection': 'close'} Mar 29 12:51:42 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:51:42.009 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP BODY: b'<html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>' Mar 29 12:52:16 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:52:16.472 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] REQ: curl -i http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... -X PUT -H "X-Auth-Token: gAAAAABgYczvODdX..." -H "Content-Length: 0" Mar 29 12:52:16 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:52:16.473 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP STATUS: 503 Service Unavailable Mar 29 12:52:16 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:52:16.473 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP HEADERS: {'content-type': 'text/html; charset=UTF-8', 'content-length': '118', 'x-trans-id': 'tx926ae448b316485d93a68-006061cd6e', 'x-openstack-request-id': 'tx926ae448b316485d93a68-006061cd6e', 'date': 'Mon, 29 Mar 2021 12:52:16 GMT', 'connection': 'close'} Mar 29 12:52:16 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:52:16.474 2403 INFO swiftclient [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] RESP BODY: b'<html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>' Mar 29 12:52:16 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:52:16.476 2403 ERROR glance.api.v2.image_data [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Failed to upload image data due to internal error: glance_store.exceptions.BackendException: Failed to add container to Swift. Mar 29 12:52:16 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Caught error: Failed to add container to Swift. Got error from Swift: Container PUT failed: http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... 503 Service Unavailable [first 60 chars of response] b'<html><h1>Service Unavailable</h1><p>The server is currently'.: glance_store.exceptions.BackendException: Failed to add container to Swift. Got error from Swift: Container PUT failed: http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... 503 Service Unavailable [first 60 chars of response] b'<html><h1>Service Unavailable</h1><p>The server is currently'. 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi Traceback (most recent call last): 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/_drivers/swift/store.py", line 1154, in _create_container_if_missing 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi connection.head_container(container) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/swiftclient/client.py", line 1870, in head_container 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi return self._retry(None, head_container, container, headers=headers) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/swiftclient/client.py", line 1801, in _retry 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi rv = func(self.url, self.token, *args, 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/swiftclient/client.py", line 1097, in head_container 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi raise ClientException.from_response( 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi swiftclient.exceptions.ClientException: Container HEAD failed: http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... 404 Not Found 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi During handling of the above exception, another exception occurred: 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi Traceback (most recent call last): 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/_drivers/swift/store.py", line 1162, in _create_container_if_missing 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi connection.put_container(container) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/swiftclient/client.py", line 1890, in put_container 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi return self._retry(None, put_container, container, headers=headers, 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/swiftclient/client.py", line 1801, in _retry 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi rv = func(self.url, self.token, *args, 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/swiftclient/client.py", line 1144, in put_container 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi raise ClientException.from_response(resp, 'Container PUT failed', body) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi swiftclient.exceptions.ClientException: Container PUT failed: http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... 503 Service Unavailable [first 60 chars of response] b'<html><h1>Service Unavailable</h1><p>The server is currently' 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi During handling of the above exception, another exception occurred: 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi Traceback (most recent call last): 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/common/wsgi.py", line 1347, in __call__ 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/common/wsgi.py", line 1391, in dispatch 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi return method(*args, **kwargs) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/common/utils.py", line 416, in wrapped 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/api/v2/image_data.py", line 298, in upload 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self._restore(image_repo, image) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self.force_reraise() 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/six.py", line 703, in reraise 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi raise value 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/api/v2/image_data.py", line 163, in upload 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/domain/proxy.py", line 208, in set_data 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self.base.set_data(data, size, backend=backend, set_active=set_active) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/notifier.py", line 501, in set_data 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self.force_reraise() 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/six.py", line 703, in reraise 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi raise value 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/notifier.py", line 447, in set_data 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/api/policy.py", line 198, in set_data 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/quota/__init__.py", line 318, in set_data 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/location.py", line 567, in set_data 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance/location.py", line 458, in _upload_to_store 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/multi_backend.py", line 398, in add_with_multihash 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/multi_backend.py", line 480, in store_add_to_backend_with_multihash 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/driver.py", line 279, in add_adapter 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/capabilities.py", line 176, in op_checker 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/_drivers/swift/store.py", line 942, in add 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi self._create_container_if_missing(location.container, 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi File "/openstack/venvs/glance-22.1.0/lib/python3.8/site-packages/glance_store/_drivers/swift/store.py", line 1167, in _create_container_if_missing 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi raise glance_store.BackendException(msg) 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi glance_store.exceptions.BackendException: Failed to add container to Swift. 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi Got error from Swift: Container PUT failed: http://192.168.110.211:8080/v1/AUTH_024cc551782f41e395d3c9f13582ef7d/glance_... 503 Service Unavailable [first 60 chars of response] b'<html><h1>Service Unavailable</h1><p>The server is currently'. 2021-03-29 12:52:16.552 2403 ERROR glance.common.wsgi Mar 29 12:52:16 infra1-glance-container-99614ac2 uwsgi[2403]: Mon Mar 29 12:52:16 2021 - uwsgi_response_writev_headers_and_body_do(): Connection reset by peer [core/writer.c line 306] during PUT /v2/images/15af8603-2428-4a9d-9202-eb7250cfea46/file (192.168.110.213) Mar 29 12:52:16 infra1-glance-container-99614ac2 glance-wsgi-api[2403]: 2021-03-29 12:52:16.565 2403 CRITICAL glance [req-d19ce0e3-2577-482e-ae09-9ea3c63aabe2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Unhandled error: OSError: write error 2021-03-29 12:52:16.565 2403 ERROR glance OSError: write error 2021-03-29 12:52:16.565 2403 ERROR glance Kind regards, Oliver
So according to the issue, you get 503 while trying to reach 10.0.3.212:6002/os-objects, which is swift_account_port. Are there any logs specificly for swift-account? Also I guess some adjustments are required for swift as well for this mechanism to work. Eventually I believe the original issue you saw might be related to this doc: https://docs.openstack.org/keystone/latest/admin/manage-services.html#config... Might be that swift also needs applying changes to accept service tokens... 29.03.2021, 16:06, "Oliver Wenz" <oliver.wenz@dhbw-mannheim.de>:
Oh, I have a guess what this might actually be. During snapshot upload process user token that is used for the upload might get expired. If that's the case, following changes in user_variables might help to resolve the issue:
glance_glance_api_conf_overrides: keystone_authtoken: service_token_roles_required: True service_token_roles: service
I found out that after inserting the above the problems now occurs every time I try to take a snapshot of an instance. Below are my logs:
-- Kind Regards, Dmitriy Rabotyagov
participants (2)
-
Dmitriy Rabotyagov
-
Oliver Wenz