March 2023 - openstack-discuss - lists.openstack.org

[Openstack Trove] Fails to create DB container - "context deadline exceeded", even though images can be pulled manually
by ddorra＠t-online.de 05 Mar '23

05 Mar '23

Hi, my Trove service installed into a Openstack Victoria fails in starting DB instances: 2023-03-05 12:06:17.172 1015 INFO trove.guestagent.datastore.mysql_common.service [-] Starting docker container, image: mysql:5.7.29 2023-03-05 12:06:17.174 1015 WARNING trove.guestagent.utils.docker [-] Failed to get container database: docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service [-] Failed to start mysql: docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") It sounds that it's unable to connect to Docker in order to download the image. However, after logging in to the DB instance I am able to pull docker images manually, or even a cloudinit action can successfully pull images during instance creation... root@t1:~# docker images REPOSITORY TAG IMAGE ID CREATED SIZE mysql 5.7.29 5d9483f9a7b2 2 years ago 455MB root@t1:~# <mailto:root@t1:~#> The cloudinit I need to adjust the default GW in oder to reach internet. I hope that is early enough in the boot process and doesn't hamper the guestagent. Any hint what I can do? ============= Trove cloudinit root@voscontrol:~# cat /etc/trove/cloudinit/mysql.cloudinit #cloud-config write_files: - content: | ... path: /etc/ssh/sshd_config - content: | ... path: /root/.bash_aliases runcmd: - [ ip, route, delete, default, via, 10.0.0.79 ] - [ ip, route, add, default, via, 10.0.0.62 ] - [ service, ssh, restart ] - [ docker, pull, "mysql:5.7.29" ] root@voscontrol:~# ================ guest config root@t1:/etc/trove# cat ./conf.d/guest_info.conf [DEFAULT] guest_id=291070bc-671d-437a-a68c-9cee840e614c datastore_manager=mysql datastore_version=5.7.29 tenant_id=606b291caeab4dc2bb1072e2e43b082a instance_rpc_encr_key=EEKenGHSe4exOOU8a0ivV2aR8FwOr2yw root@t1:/etc/trove# cat ./conf.d/trove-guestagent.conf [DEFAULT] log_file = trove-guestagent.log log_dir = /var/log/trove/ ignore_users = os_admin control_exchange = trove #transport_url = rabbit://openstack:pass123@192.168.100.79:5672/ transport_url = rabbit://openstack:pass123@10.0.0.79:5672/ rpc_backend = rabbit command_process_timeout = 60 use_syslog = False debug = True [service_credentials] #auth_url = http://192.168.100.79:5000/v3 auth_url = http://10.0.0.79:5000/v3 region_name = RegionOne project_name = service password = pass123 project_domain_name = Default user_domain_name = Default username = trove root@t1:/etc/trove# ================ /var/log/trove/trove-guestagent.log ... 2023-03-05 12:06:17.162 1015 DEBUG oslo_concurrency.processutils [-] CMD "sudo mkdir -p /var/run/mysqld" returned: 0 in 0.009s execute /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 2023-03-05 12:06:17.162 1015 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo chown -R 1001:1001 /var/run/mysqld execute /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 2023-03-05 12:06:17.171 1015 DEBUG oslo_concurrency.processutils [-] CMD "sudo chown -R 1001:1001 /var/run/mysqld" returned: 0 in 0.009s execute /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 2023-03-05 12:06:17.172 1015 INFO trove.guestagent.datastore.mysql_common.service [-] Starting docker container, image: mysql:5.7.29 2023-03-05 12:06:17.174 1015 WARNING trove.guestagent.utils.docker [-] Failed to get container database: docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service [-] Failed to start mysql: docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service response.raise_for_status() 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/database/json 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 58, in start_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service container = client.containers.get(name) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 887, in get 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service resp = self.client.api.inspect_container(container_id) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service return f(self, resource_id, *args, **kwargs) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 771, in inspect_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._get(self._url("/containers/{0}/json", container)), True 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._raise_for_status(response) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service response.raise_for_status() 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create?name=database 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 810, in run 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service detach=detach, **kwargs) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 868, in create 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service resp = self.client.api.create_container(**create_kwargs) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 430, in create_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service return self.create_container_from_config(config, name) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 441, in create_container_from_config 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service return self._result(res, True) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._raise_for_status(response) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service docker.errors.ImageNotFound: 404 Client Error: Not Found ("No such image: mysql:5.7.29") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service response.raise_for_status() 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=5.7.29&fromImage=mysql 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 612, in start_db 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service command=command 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 73, in start_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service command=command 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 812, in run 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self.client.images.pull(image, platform=platform) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/images.py", line 445, in pull 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service repository, tag=tag, stream=True, **kwargs 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/image.py", line 415, in pull 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._raise_for_status(response) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager [-] Failed to prepare datastore: Failed to start mysql: trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager response.raise_for_status() 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/database/json 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 58, in start_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager container = client.containers.get(name) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 887, in get 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager resp = self.client.api.inspect_container(container_id) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager return f(self, resource_id, *args, **kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 771, in inspect_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._get(self._url("/containers/{0}/json", container)), True 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._raise_for_status(response) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager response.raise_for_status() 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create?name=database 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 810, in run 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager detach=detach, **kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 868, in create 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager resp = self.client.api.create_container(**create_kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 430, in create_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager return self.create_container_from_config(config, name) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 441, in create_container_from_config 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager return self._result(res, True) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._raise_for_status(response) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager docker.errors.ImageNotFound: 404 Client Error: Not Found ("No such image: mysql:5.7.29") 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager response.raise_for_status() 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=5.7.29&fromImage=mysql 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 612, in start_db 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager command=command 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 73, in start_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager command=command 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 812, in run 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self.client.images.pull(image, platform=platform) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/images.py", line 445, in pull 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager repository, tag=tag, stream=True, **kwargs 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/image.py", line 415, in pull 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._raise_for_status(response) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py", line 223, in _prepare 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager cluster_config, snapshot, ds_version=ds_version) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager result = f(*args, **kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/manager.py", line 96, in do_prepare 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self.app.start_db(ds_version=ds_version, command=command) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 620, in start_db 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise exception.TroveError(_("Failed to start mysql")) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.549 1015 INFO trove.guestagent.datastore.manager [-] Ending datastore prepare for 'mysql'. 2023-03-05 12:06:32.550 1015 INFO trove.guestagent.datastore.service [-] Set final status to failed to spawn. 2023-03-05 12:06:32.550 1015 DEBUG trove.guestagent.datastore.service [-] Casting set_status message to conductor (status is 'failed to spawn'). set_status /opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/service.py:144 2023-03-05 12:06:32.550 1015 DEBUG trove.conductor.api [-] Making async call to cast heartbeat for instance: 291070bc-671d-437a-a68c-9cee840e614c heartbeat /opt/guest-agent-venv/lib/python3.6/site-packages/trove/conductor/api.py:73 2023-03-05 12:06:32.553 1015 DEBUG trove.guestagent.datastore.service [-] Successfully cast set_status. set_status /opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/service.py:151 2023-03-05 12:06:32.553 1015 DEBUG trove.conductor.api [-] Making async call to cast error notification notify_exc_info /opt/guest-agent-venv/lib/python3.6/site-packages/trove/conductor/api.py:115 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server [-] Exception during message handling: trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server response.raise_for_status() 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/database/json 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 58, in start_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server container = client.containers.get(name) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 887, in get 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server resp = self.client.api.inspect_container(container_id) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return f(self, resource_id, *args, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 771, in inspect_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._get(self._url("/containers/{0}/json", container)), True 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._raise_for_status(response) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server response.raise_for_status() 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create?name=database 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 810, in run 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server detach=detach, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 868, in create 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server resp = self.client.api.create_container(**create_kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 430, in create_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return self.create_container_from_config(config, name) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 441, in create_container_from_config 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return self._result(res, True) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._raise_for_status(response) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server docker.errors.ImageNotFound: 404 Client Error: Not Found ("No such image: mysql:5.7.29") 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server response.raise_for_status() 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=5.7.29&fromImage=mysql 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 612, in start_db 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server command=command 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 73, in start_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server command=command 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 812, in run 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self.client.images.pull(image, platform=platform) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/images.py", line 445, in pull 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server repository, tag=tag, stream=True, **kwargs 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/image.py", line 415, in pull 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._raise_for_status(response) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py", line 207, in prepare 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server ds_version=ds_version) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py", line 223, in _prepare 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server cluster_config, snapshot, ds_version=ds_version) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/manager.py", line 96, in do_prepare 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self.app.start_db(ds_version=ds_version, command=command) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 620, in start_db 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise exception.TroveError(_("Failed to start mysql")) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:58.303 1015 DEBUG trove.guestagent.datastore.manager [-] Getting file system stats for '/var/lib/mysql' get_filesystem_stats /opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py:368 2023-03-05 12:07:16.911 1015 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.update_status run_periodic_tasks /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_service/periodic_task.py:211 2023-03-05 12:07:16.911 1015 INFO trove.guestagent.datastore.manager [-] Database service is not installed, skip status check 2023-03-05 12:08:16.918 1015 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.update_status run_periodic_tasks /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_service/periodic_task.py:211 2023-03-05 12:08:16.919 1015 INFO trove.guestagent.datastore.manager [-] Database service is not installed, skip status check root@t1:/var/log/trove#

1 0

[ironic] The future of x/ironic-staging-drivers
by Dmitry Tantsur 03 Mar '23

03 Mar '23

Hi folks! I have been maintaining $subj together with Riccardo and a few occasional volunteers for many years. Now that our priorities have changed, it is not maintained any more. We haven't even created stable/zed, and I'm afraid to check the CI status. We're looking for volunteers to maintain the repository long-term. If none are found, the project will be deprecated and frozen in its current state. Please speak up if you care. Dmitry -- Red Hat GmbH <https://www.redhat.com/de/global/dach>, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,Managing Directors: Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross

1 0

[release] Release countdown for week R-2, March 6-10
by Thierry Carrez 03 Mar '23

03 Mar '23

Development Focus ----------------- At this point we should have release candidates (RC1 or recent intermediary release) for almost all the deliverables. Teams should be working on any release-critical bugs that would require another RC or intermediary release before the final release. Actions ------- Early in the week, the release team will be proposing stable/2023.1 branch creation for all deliverables that have not branched yet, using the latest available 2023.1 Antelope release as the branch point. If your team is ready to go for creating that branch, please let us know by leaving a +1 on these patches. If you would like to wait for another release before branching, you can -1 the patch and update it later in the week with the new release you would like to use. By the end of the week the release team will merge those patches though, unless an exception is granted. Once stable/2023.1 branches are created, if a release-critical bug is detected, you will need to fix the issue in the master branch first, then backport the fix to the stable/2023.1 branch before releasing out of the stable/2023.1 branch. After all of the cycle-with-rc projects have branched we will branch devstack, grenade, and the requirements repos. This will effectively open them up for Bobcat development, though the focus should still be on finishing up Antelope until the final 2023.1 release. For projects with translations, watch for any translation patches coming through and merge them quickly. A new release should be produced so that translations are included in the final 2023.1 Antelope release. Finally, now is a good time to finalize release notes. In particular, consider adding any relevant "prelude" content. Release notes are targeted for the downstream consumers of your project, so it would be great to include any useful information for those that are going to pick up and use or deploy the 2023.1 Antelope version of your project. Upcoming Deadlines & Dates -------------------------- Final RC deadline: March 17 (end of R-1 week) Final 2023.1 Antelope release: March 22 Virtual PTG: March 27-31 -- Thierry Carrez (ttx)

1 0

[neutron] Drivers meeting cancelled
by Rodolfo Alonso Hernandez 03 Mar '23

03 Mar '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting is cancelled. Have a nice weekend. *PS: do not forget to add your topics to the PTG agenda [2]**. PTG is coming!* [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers [2]https://etherpad.opendev.org/p/neutron-bobcat-ptg

1 0

[kolla] [train] [cinder] Volume multiattach exposed to non-admin users via API
by Albert Braden 03 Mar '23

03 Mar '23

According to this document [1] multiattach volumes can only be setup if explicitly allowed by creating a “multiattach” volume type. “Starting from the Queens release the ability to attach a volume to multiple hosts/servers requires that the volume is of a special type that includes an extra-spec capability setting of multiattach=<is> True… Creating a new volume type is an admin-only operation by default. One of our customers appears to have used TerraForm to create a volume with the multiattach flag set and it worked, and that volume has multiple attachments. When I look here [2] it appears that the default is: #"volume:multiattach": "rule:xena_system_admin_or_project_member" So it looks like, by default, any project member can create a multiattach volume. What am I missing? [1]: https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html [2]: https://docs.openstack.org/cinder/latest/configuration/block-storage/sample…

5 10

[governance][heat][tacker] Moving governance of tosca-parser(and heat-translator ?) to Tacker
by Takashi Kajinami 02 Mar '23

02 Mar '23

Hello, Currently tosca-parser is part of heat's governance, but the core reviewers of this repository does not contain any active heat cores while we see multiple Tacker cores in this group. Considering the fact the project is mainly maintained by Tacker cores, I'm wondering if we can migrate this repository to Tacker's governance. Most of the current heat cores are not quite familiar with the codes in this repository, and if Tacker team is not interested in maintaining this repository then I'd propose retiring this. Similarly, we have heat-translator project which has both heat cores and tacker cores as its core reviewers. IIUC this is tightly related to the work in tosca-parser, I'm wondering it makes more sense to move this project to Tacker, because the requirement is mostly made from Tacker side rather than Heat side. [1] https://review.opendev.org/admin/groups/1f7855baf3cf14fedf72e443eef18d844bc… [2] https://review.opendev.org/admin/groups/66028971dcbb58add6f0e7c17ac72643c48… Thank you, Takashi

3 8

[infra][ironic][tact-sig] Intent to grant control of x/virtualpdu to OpenStack community
by Jeremy Stanley 02 Mar '23

02 Mar '23

The OpenStack Ironic project relies on VirtualPDU, which is no longer actively developed. Ironic contributors reached out to the VirtualPDU maintainers with an offer to officially assume maintenance responsibilities in order to avoid forking it, and received a response from one (Mathieu Mitchell) who indicated support for that plan. Unfortunately, communication died off shortly thereafter, and the Ironic team has been unable to raise any of the current maintainers to actually add their access to the x/virtualpdu project in OpenDev's Gerrit. We don't have a documented process since this is the first time it's really come up, but I'm officially announcing that I intend to use my administrative permissions as an OpenDev sysadmin to add membership of an OpenStack Ironic team representative to the following Gerrit groups if no objections are raised before Wednesday, March 8: * virtualpdu-core * virtualpdu-release This will effectively grant full control of the x/virtualpdu repository to OpenStack Ironic contributors. Please follow up to the service-discuss(a)lists.opendev.org or openstack-discuss(a)lists.openstack.org mailing list with any concerns as soon as possible, or you can feel free to reach out to me directly by email or in IRC (fungi in the #opendev channel on the OFTC network). -- Jeremy Stanley

3 3

(OpenStack-Upgrade)
by Adivya Singh 02 Mar '23

02 Mar '23

Hi Team, I am planning to upgrade my Current Environment, The Upgrade procedure is available in OpenStack Site and Forums. But i am looking fwd to roll back Plan , Other then have a Local backup copy of galera Database Regards Adivya Singh

5 9

[OPENSTACKSDK] - Missing feature or bad reader?
by Gaël THEROND 02 Mar '23

02 Mar '23

> Hi everyone, > > I'm currently adding a new module on ansible-collections-openstack, > however, I'm having a hard time finding the appropriate function for my > module. > > Within ansible-collections-openstack, we have a compute_services_info > module that list services using the conn.compute.services() function that, > to my knowledge, come from the openstacksdk.compute.v2.service.py module. > > Our ansible module replicate what you get with: > openstack --os-cloud compute service list command. > or > openstack --os-cloud volume service list command. > (If I'm not wrong, it seems, openstack client is leveraging osc-lib for > that and not the SDK). > > My problem is, I want to add another similar module on our collection, > (volume_services_info) that would do the same but for volumes services: > > Unfortunately, either I'm not looking at the right place, or any volume > endpoint (v2/v3) within the openstack sdk is implementing the appropriate > service module. >From what I see block_storage in SDK is currently missing implementation for >service management (it is an admin-only and such APIs tend to be of lower prio >in SDK). >Artem All right, thanks a lot for the confirmation Artem, gonna see it we could add that pretty quickly on SDK as cinder already provide the appropriate /os-service API endpoint.

1 0

[OPENSTACKSDK] - Missing feature or bad reader?
by Gaël THEROND 02 Mar '23

02 Mar '23

Hi everyone, I'm currently adding a new module on ansible-collections-openstack, however, I'm having a hard time finding the appropriate function for my module. Within ansible-collections-openstack, we have a compute_services_info module that list services using the conn.compute.services() function that, to my knowledge, come from the openstacksdk.compute.v2.service.py module. Our ansible module replicate what you get with: openstack --os-cloud compute service list command. or openstack --os-cloud volume service list command. (If I'm not wrong, it seems, openstack client is leveraging osc-lib for that and not the SDK). My problem is, I want to add another similar module on our collection, (volume_services_info) that would do the same but for volumes services: Unfortunately, either I'm not looking at the right place, or any volume endpoint (v2/v3) within the openstack sdk is implementing the appropriate service module. Did I miss something or is that class simply missing? Thanks everyone!

2 1