March 2023 - openstack-discuss - lists.openstack.org

[neutron] CI meeting Tuesday 7.03.2023 cancelled
by Slawek Kaplonski 06 Mar '23

06 Mar '23

Hi, Due to some internal event which I have tomorrow in the same time as our CI meeting I will not be able to run the CI meeting. I don't see any really serious issues in our CI this week and after discussing that with Rodolfo we decided to cancel this week's CI meeting. See You on the meeting next week. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[cinder] Error when creating backups from iscsi volume
by Rishat Azizov 06 Mar '23

06 Mar '23

Hello! We have an error with creating backups from iscsi volume. Usually, this happens with large backups over 100GB. 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server [req-f6619913-6f96-4226-8d75-2da3fca722f1 23de1b92e7674cf59486f07ac75b886b a7585b47d1f143e9839c49b4e3bbe1b4 - - -] Exception during message handling: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Command: multipath -f 3624a93705842cfae35d7483200015ec6 Exit code: 1 Stdout: '' Stderr: 'Feb 16 00:22:45 | 3624a93705842cfae35d7483200015ec6 is not a multipath device\n' 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/utils.py", line 890, in wrapper 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 410, in create_backup 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server volume_utils.update_backup_error(backup, str(err)) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server raise self.value 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 399, in create_backup 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server updates = self._run_backup(context, backup, volume) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 493, in _run_backup 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server ignore_errors=True) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 1066, in _detach_device 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server force=force, ignore_errors=ignore_errors) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 141, in trace_logging_wrapper 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 360, in inner 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 880, in disconnect_volume 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server is_disconnect_call=True) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 942, in _cleanup_connection 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server self._linuxscsi.flush_multipath_device(multipath_name) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/linuxscsi.py", line 382, in flush_multipath_device 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server root_helper=self._root_helper) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/executor.py", line 52, in _execute 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server result = self.__execute(*args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/privileged/rootwrap.py", line 172, in execute 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return execute_root(*cmd, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 247, in _wrap 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return self.channel.remote_call(name, args, kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 224, in remote_call 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server raise exc_type(*result[2]) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Command: multipath -f 3624a93705842cfae35d7483200015ec6 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Exit code: 1 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Stdout: '' 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Stderr: 'Feb 16 00:22:45 | 3624a93705842cfae35d7483200015ec6 is not a multipath device\n' 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Could you please help with this error?

2 1

[neutron] Bug deputy report (week 09, starting on Feb-27-2022)
by Lajos Katona 06 Mar '23

06 Mar '23

Hi neutrinos, Here are the bugs reported between February 27 and March 05. *First the unassigned bugs:* * https://bugs.launchpad.net/neutron/+bug/2009043 neutron-l3-agent restart some random ha routers get wrong state *High prio bugs* * https://bugs.launchpad.net/neutron/+bug/2008712 Security group rule deleted by cascade (because its remote group had been deleted) is not deleted in the backend * https://bugs.launchpad.net/neutron/+bug/2008947 Investigate test_restart_rpc_on_sighup_multiple_workers failure * https://bugs.launchpad.net/neutron/+bug/2009055 Performance issue when creating lots of ports * https://bugs.launchpad.net/neutron/+bug/2009215 [OVS] Error during OVS agent start *Medium or lower:* * https://bugs.launchpad.net/neutron/+bug/2008858 Call the api and do not return for a long time * https://bugs.launchpad.net/neutron/+bug/2008912 "_validate_create_network_callback" failing with 'NoneType' object has no attribute 'qos_policy_id' * https://bugs.launchpad.net/neutron/+bug/2008943 OVN DB Sync utility cannot find NB DB Port Group * https://bugs.launchpad.net/neutron/+bug/2009053 OVN: default stateless SG blocks metadata traffic * https://bugs.launchpad.net/neutron/+bug/2009221 [OVS] Custom ethertype traffic is not coming into the VM *One incomplete:* * https://bugs.launchpad.net/neutron/+bug/2008808 Duplicate packet when ping to external with out floating ip *Already merged / released bugs:* * https://bugs.launchpad.net/neutron/+bug/2008695 Remove any LB HM references from the external_ids upon deleting an HM * https://bugs.launchpad.net/neutron/+bug/2008767 [sqlalchemy-20][vnpaas] SQL execution without transaction in progress Have a bugless week :-) Lajos (lajoskatona)

1 0

[kolla] Weekly meeting on 8th March cancelled
by Michał Nasiadka 06 Mar '23

06 Mar '23

Hola Koalas, Weekly meeting on Wed this week is cancelled - I’m off on vacation. See you next week! Best regards, Michal

1 0

[Openstack Trove] Fails to create DB container - "context deadline exceeded", even though images can be pulled manually
by ddorra＠t-online.de 05 Mar '23

05 Mar '23

Hi, my Trove service installed into a Openstack Victoria fails in starting DB instances: 2023-03-05 12:06:17.172 1015 INFO trove.guestagent.datastore.mysql_common.service [-] Starting docker container, image: mysql:5.7.29 2023-03-05 12:06:17.174 1015 WARNING trove.guestagent.utils.docker [-] Failed to get container database: docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service [-] Failed to start mysql: docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") It sounds that it's unable to connect to Docker in order to download the image. However, after logging in to the DB instance I am able to pull docker images manually, or even a cloudinit action can successfully pull images during instance creation... root@t1:~# docker images REPOSITORY TAG IMAGE ID CREATED SIZE mysql 5.7.29 5d9483f9a7b2 2 years ago 455MB root@t1:~# <mailto:root@t1:~#> The cloudinit I need to adjust the default GW in oder to reach internet. I hope that is early enough in the boot process and doesn't hamper the guestagent. Any hint what I can do? ============= Trove cloudinit root@voscontrol:~# cat /etc/trove/cloudinit/mysql.cloudinit #cloud-config write_files: - content: | ... path: /etc/ssh/sshd_config - content: | ... path: /root/.bash_aliases runcmd: - [ ip, route, delete, default, via, 10.0.0.79 ] - [ ip, route, add, default, via, 10.0.0.62 ] - [ service, ssh, restart ] - [ docker, pull, "mysql:5.7.29" ] root@voscontrol:~# ================ guest config root@t1:/etc/trove# cat ./conf.d/guest_info.conf [DEFAULT] guest_id=291070bc-671d-437a-a68c-9cee840e614c datastore_manager=mysql datastore_version=5.7.29 tenant_id=606b291caeab4dc2bb1072e2e43b082a instance_rpc_encr_key=EEKenGHSe4exOOU8a0ivV2aR8FwOr2yw root@t1:/etc/trove# cat ./conf.d/trove-guestagent.conf [DEFAULT] log_file = trove-guestagent.log log_dir = /var/log/trove/ ignore_users = os_admin control_exchange = trove #transport_url = rabbit://openstack:pass123@192.168.100.79:5672/ transport_url = rabbit://openstack:pass123@10.0.0.79:5672/ rpc_backend = rabbit command_process_timeout = 60 use_syslog = False debug = True [service_credentials] #auth_url = http://192.168.100.79:5000/v3 auth_url = http://10.0.0.79:5000/v3 region_name = RegionOne project_name = service password = pass123 project_domain_name = Default user_domain_name = Default username = trove root@t1:/etc/trove# ================ /var/log/trove/trove-guestagent.log ... 2023-03-05 12:06:17.162 1015 DEBUG oslo_concurrency.processutils [-] CMD "sudo mkdir -p /var/run/mysqld" returned: 0 in 0.009s execute /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 2023-03-05 12:06:17.162 1015 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sudo chown -R 1001:1001 /var/run/mysqld execute /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 2023-03-05 12:06:17.171 1015 DEBUG oslo_concurrency.processutils [-] CMD "sudo chown -R 1001:1001 /var/run/mysqld" returned: 0 in 0.009s execute /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 2023-03-05 12:06:17.172 1015 INFO trove.guestagent.datastore.mysql_common.service [-] Starting docker container, image: mysql:5.7.29 2023-03-05 12:06:17.174 1015 WARNING trove.guestagent.utils.docker [-] Failed to get container database: docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service [-] Failed to start mysql: docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service response.raise_for_status() 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/database/json 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 58, in start_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service container = client.containers.get(name) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 887, in get 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service resp = self.client.api.inspect_container(container_id) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service return f(self, resource_id, *args, **kwargs) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 771, in inspect_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._get(self._url("/containers/{0}/json", container)), True 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._raise_for_status(response) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service response.raise_for_status() 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create?name=database 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 810, in run 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service detach=detach, **kwargs) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 868, in create 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service resp = self.client.api.create_container(**create_kwargs) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 430, in create_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service return self.create_container_from_config(config, name) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 441, in create_container_from_config 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service return self._result(res, True) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._raise_for_status(response) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service docker.errors.ImageNotFound: 404 Client Error: Not Found ("No such image: mysql:5.7.29") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service response.raise_for_status() 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=5.7.29&fromImage=mysql 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service Traceback (most recent call last): 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 612, in start_db 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service command=command 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 73, in start_container 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service command=command 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 812, in run 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self.client.images.pull(image, platform=platform) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/images.py", line 445, in pull 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service repository, tag=tag, stream=True, **kwargs 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/image.py", line 415, in pull 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service self._raise_for_status(response) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.540 1015 ERROR trove.guestagent.datastore.mysql_common.service 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager [-] Failed to prepare datastore: Failed to start mysql: trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager response.raise_for_status() 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/database/json 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 58, in start_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager container = client.containers.get(name) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 887, in get 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager resp = self.client.api.inspect_container(container_id) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager return f(self, resource_id, *args, **kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 771, in inspect_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._get(self._url("/containers/{0}/json", container)), True 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._raise_for_status(response) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager response.raise_for_status() 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create?name=database 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 810, in run 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager detach=detach, **kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 868, in create 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager resp = self.client.api.create_container(**create_kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 430, in create_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager return self.create_container_from_config(config, name) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 441, in create_container_from_config 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager return self._result(res, True) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._raise_for_status(response) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager docker.errors.ImageNotFound: 404 Client Error: Not Found ("No such image: mysql:5.7.29") 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager response.raise_for_status() 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=5.7.29&fromImage=mysql 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 612, in start_db 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager command=command 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 73, in start_container 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager command=command 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 812, in run 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self.client.images.pull(image, platform=platform) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/images.py", line 445, in pull 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager repository, tag=tag, stream=True, **kwargs 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/image.py", line 415, in pull 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self._raise_for_status(response) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager Traceback (most recent call last): 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py", line 223, in _prepare 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager cluster_config, snapshot, ds_version=ds_version) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager result = f(*args, **kwargs) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/manager.py", line 96, in do_prepare 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager self.app.start_db(ds_version=ds_version, command=command) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 620, in start_db 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager raise exception.TroveError(_("Failed to start mysql")) 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.546 1015 ERROR trove.guestagent.datastore.manager 2023-03-05 12:06:32.549 1015 INFO trove.guestagent.datastore.manager [-] Ending datastore prepare for 'mysql'. 2023-03-05 12:06:32.550 1015 INFO trove.guestagent.datastore.service [-] Set final status to failed to spawn. 2023-03-05 12:06:32.550 1015 DEBUG trove.guestagent.datastore.service [-] Casting set_status message to conductor (status is 'failed to spawn'). set_status /opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/service.py:144 2023-03-05 12:06:32.550 1015 DEBUG trove.conductor.api [-] Making async call to cast heartbeat for instance: 291070bc-671d-437a-a68c-9cee840e614c heartbeat /opt/guest-agent-venv/lib/python3.6/site-packages/trove/conductor/api.py:73 2023-03-05 12:06:32.553 1015 DEBUG trove.guestagent.datastore.service [-] Successfully cast set_status. set_status /opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/service.py:151 2023-03-05 12:06:32.553 1015 DEBUG trove.conductor.api [-] Making async call to cast error notification notify_exc_info /opt/guest-agent-venv/lib/python3.6/site-packages/trove/conductor/api.py:115 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server [-] Exception during message handling: trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server response.raise_for_status() 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/database/json 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 58, in start_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server container = client.containers.get(name) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 887, in get 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server resp = self.client.api.inspect_container(container_id) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return f(self, resource_id, *args, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 771, in inspect_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._get(self._url("/containers/{0}/json", container)), True 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._raise_for_status(response) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server docker.errors.NotFound: 404 Client Error: Not Found ("No such container: database") 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server response.raise_for_status() 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create?name=database 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 810, in run 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server detach=detach, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 868, in create 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server resp = self.client.api.create_container(**create_kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 430, in create_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return self.create_container_from_config(config, name) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/container.py", line 441, in create_container_from_config 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return self._result(res, True) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 265, in _result 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._raise_for_status(response) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server docker.errors.ImageNotFound: 404 Client Error: Not Found ("No such image: mysql:5.7.29") 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 259, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server response.raise_for_status() 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise HTTPError(http_error_msg, response=self) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=5.7.29&fromImage=mysql 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 612, in start_db 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server command=command 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/utils/docker.py", line 73, in start_container 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server command=command 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/containers.py", line 812, in run 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self.client.images.pull(image, platform=platform) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/models/images.py", line 445, in pull 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server repository, tag=tag, stream=True, **kwargs 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/image.py", line 415, in pull 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self._raise_for_status(response) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise create_api_error_from_http_exception(e) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise cls(e, response=response, explanation=explanation) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server docker.errors.APIError: 500 Server Error: Internal Server Error ("Get "https://registry-1.docker.io/v2/": context deadline exceeded") 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py", line 207, in prepare 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server ds_version=ds_version) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py", line 223, in _prepare 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server cluster_config, snapshot, ds_version=ds_version) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/manager.py", line 96, in do_prepare 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server self.app.start_db(ds_version=ds_version, command=command) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 620, in start_db 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server raise exception.TroveError(_("Failed to start mysql")) 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server trove.common.exception.TroveError: Failed to start mysql 2023-03-05 12:06:32.557 1015 ERROR oslo_messaging.rpc.server 2023-03-05 12:06:58.303 1015 DEBUG trove.guestagent.datastore.manager [-] Getting file system stats for '/var/lib/mysql' get_filesystem_stats /opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py:368 2023-03-05 12:07:16.911 1015 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.update_status run_periodic_tasks /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_service/periodic_task.py:211 2023-03-05 12:07:16.911 1015 INFO trove.guestagent.datastore.manager [-] Database service is not installed, skip status check 2023-03-05 12:08:16.918 1015 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.update_status run_periodic_tasks /opt/guest-agent-venv/lib/python3.6/site-packages/oslo_service/periodic_task.py:211 2023-03-05 12:08:16.919 1015 INFO trove.guestagent.datastore.manager [-] Database service is not installed, skip status check root@t1:/var/log/trove#

1 0

[ironic] The future of x/ironic-staging-drivers
by Dmitry Tantsur 03 Mar '23

03 Mar '23

Hi folks! I have been maintaining $subj together with Riccardo and a few occasional volunteers for many years. Now that our priorities have changed, it is not maintained any more. We haven't even created stable/zed, and I'm afraid to check the CI status. We're looking for volunteers to maintain the repository long-term. If none are found, the project will be deprecated and frozen in its current state. Please speak up if you care. Dmitry -- Red Hat GmbH <https://www.redhat.com/de/global/dach>, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,Managing Directors: Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross

1 0

[release] Release countdown for week R-2, March 6-10
by Thierry Carrez 03 Mar '23

03 Mar '23

Development Focus ----------------- At this point we should have release candidates (RC1 or recent intermediary release) for almost all the deliverables. Teams should be working on any release-critical bugs that would require another RC or intermediary release before the final release. Actions ------- Early in the week, the release team will be proposing stable/2023.1 branch creation for all deliverables that have not branched yet, using the latest available 2023.1 Antelope release as the branch point. If your team is ready to go for creating that branch, please let us know by leaving a +1 on these patches. If you would like to wait for another release before branching, you can -1 the patch and update it later in the week with the new release you would like to use. By the end of the week the release team will merge those patches though, unless an exception is granted. Once stable/2023.1 branches are created, if a release-critical bug is detected, you will need to fix the issue in the master branch first, then backport the fix to the stable/2023.1 branch before releasing out of the stable/2023.1 branch. After all of the cycle-with-rc projects have branched we will branch devstack, grenade, and the requirements repos. This will effectively open them up for Bobcat development, though the focus should still be on finishing up Antelope until the final 2023.1 release. For projects with translations, watch for any translation patches coming through and merge them quickly. A new release should be produced so that translations are included in the final 2023.1 Antelope release. Finally, now is a good time to finalize release notes. In particular, consider adding any relevant "prelude" content. Release notes are targeted for the downstream consumers of your project, so it would be great to include any useful information for those that are going to pick up and use or deploy the 2023.1 Antelope version of your project. Upcoming Deadlines & Dates -------------------------- Final RC deadline: March 17 (end of R-1 week) Final 2023.1 Antelope release: March 22 Virtual PTG: March 27-31 -- Thierry Carrez (ttx)

1 0

[neutron] Drivers meeting cancelled
by Rodolfo Alonso Hernandez 03 Mar '23

03 Mar '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting is cancelled. Have a nice weekend. *PS: do not forget to add your topics to the PTG agenda [2]**. PTG is coming!* [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers [2]https://etherpad.opendev.org/p/neutron-bobcat-ptg

1 0

[kolla] [train] [cinder] Volume multiattach exposed to non-admin users via API
by Albert Braden 03 Mar '23

03 Mar '23

According to this document [1] multiattach volumes can only be setup if explicitly allowed by creating a “multiattach” volume type. “Starting from the Queens release the ability to attach a volume to multiple hosts/servers requires that the volume is of a special type that includes an extra-spec capability setting of multiattach=<is> True… Creating a new volume type is an admin-only operation by default. One of our customers appears to have used TerraForm to create a volume with the multiattach flag set and it worked, and that volume has multiple attachments. When I look here [2] it appears that the default is: #"volume:multiattach": "rule:xena_system_admin_or_project_member" So it looks like, by default, any project member can create a multiattach volume. What am I missing? [1]: https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html [2]: https://docs.openstack.org/cinder/latest/configuration/block-storage/sample…

5 10

[governance][heat][tacker] Moving governance of tosca-parser(and heat-translator ?) to Tacker
by Takashi Kajinami 02 Mar '23

02 Mar '23

Hello, Currently tosca-parser is part of heat's governance, but the core reviewers of this repository does not contain any active heat cores while we see multiple Tacker cores in this group. Considering the fact the project is mainly maintained by Tacker cores, I'm wondering if we can migrate this repository to Tacker's governance. Most of the current heat cores are not quite familiar with the codes in this repository, and if Tacker team is not interested in maintaining this repository then I'd propose retiring this. Similarly, we have heat-translator project which has both heat cores and tacker cores as its core reviewers. IIUC this is tightly related to the work in tosca-parser, I'm wondering it makes more sense to move this project to Tacker, because the requirement is mostly made from Tacker side rather than Heat side. [1] https://review.opendev.org/admin/groups/1f7855baf3cf14fedf72e443eef18d844bc… [2] https://review.opendev.org/admin/groups/66028971dcbb58add6f0e7c17ac72643c48… Thank you, Takashi

3 8