Nova-conductor is having trouble with AMQP authentication
Airamek
vegkeppnemairamek at gmail.com
Fri Jul 8 01:28:15 UTC 2022
Hello everyone!
I've installed OpenStack Ussuri on my home servers (one controller and
one compute node, both running OpenSuse Leap 15.3) based on the
instructions in the
docs(https://docs.openstack.org/install-guide/openstack-services.html#minimal-deployment-for-ussuri)
Everything works until I launch an instance in Horizon. The instance
gets stuck on "Scheduling". Looking trough the logs I've came across
nova-conductor.log(on the controller), which contained the following
error log(full log in the attachments):
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server
[req-eb225ffd-9b12-4e80-987b-34f0317082c6
741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6
9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654
4596c7f3b97740b3adfa8f82b6240654] Exception during message handling:
amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was
refused using authentication mechanism AMQPLAIN. For details see the
broker logfile.
I'm 100% sure my RabbitMQ username and password are set correctly in
nova.conf. I've included my nova.conf too, with the passwords removed.
I would be really thankful, if someone could point me in the right
direction!
P.S: Please excuse me for any grammatical or spelling mistakes, English
is not my first language.
-------------- next part --------------
2022-07-08 00:44:03.933 24662 INFO oslo_service.service [req-8b628570-bee1-4744-9c41-cac8ea76917d - - - - -] Caught SIGTERM, stopping children
2022-07-08 00:44:03.945 24662 INFO oslo_service.service [req-8b628570-bee1-4744-9c41-cac8ea76917d - - - - -] Waiting on 2 children to exit
2022-07-08 00:44:03.949 24662 INFO oslo_service.service [req-8b628570-bee1-4744-9c41-cac8ea76917d - - - - -] Child 24716 killed by signal 15
2022-07-08 00:44:03.950 24662 INFO oslo_service.service [req-8b628570-bee1-4744-9c41-cac8ea76917d - - - - -] Child 24717 killed by signal 15
2022-07-08 00:44:09.847 25830 INFO oslo_service.service [req-610a6fd5-5a14-47b6-b7a2-f80490141c47 - - - - -] Starting 2 workers
2022-07-08 00:44:09.865 25885 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 00:44:09.873 25830 WARNING oslo_config.cfg [req-610a6fd5-5a14-47b6-b7a2-f80490141c47 - - - - -] Deprecated: Option "auth_strategy" from group "api" is deprecated for removal (
The only non-default choice, ``noauth2``, is for internal development and
testing purposes only and should not be used in deployments. This option and
its middleware, NoAuthMiddleware[V2_18], will be removed in a future release.
). Its value may be silently ignored in the future.
2022-07-08 00:44:09.877 25886 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 00:44:09.886 25830 WARNING oslo_config.cfg [req-610a6fd5-5a14-47b6-b7a2-f80490141c47 - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2022-07-08 00:48:08.510 25886 ERROR stevedore.extension [req-eb225ffd-9b12-4e80-987b-34f0317082c6 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Could not load 'oslo_cache.etcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw'
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server [req-eb225ffd-9b12-4e80-987b-34f0317082c6 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Exception during message handling: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 1655, in schedule_and_build_instances
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server accel_uuids=accel_uuids)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 1448, in build_and_run_instance
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server cctxt.cast(ctxt, 'build_and_run_instance', **kwargs)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 152, in cast
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server transport_options=self.transport_options)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in send
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 616, in _send
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server with self._get_connection(rpc_common.PURPOSE_SEND) as conn:
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 570, in _get_connection
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server purpose=purpose)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/common.py", line 423, in __init__
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server self.connection = connection_pool.get()
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 109, in get
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server return self.create()
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 146, in create
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server return self.connection_cls(self.conf, self.url, purpose)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 618, in __init__
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server self.ensure_connection()
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 735, in ensure_connection
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server self.connection.ensure_connection(errback=on_error)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 389, in ensure_connection
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server self._ensure_connection(*args, **kwargs)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 445, in _ensure_connection
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server callback, timeout=timeout
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 874, in _connection_factory
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server self._connection = self._establish_connection()
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 809, in _establish_connection
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server conn = self.transport.establish_connection()
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server conn.connect()
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 320, in connect
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server self.drain_events(timeout=self.connect_timeout)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 508, in drain_events
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout):
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 514, in blocking_read
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server callback(channel, method_sig, buf, None)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 521, in on_inbound_method
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server method_sig, payload, content,
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/abstract_channel.py", line 145, in dispatch_method
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server listener(*args)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 651, in _on_close
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server (class_id, method_id), ConnectionError)
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 00:48:08.846 25886 ERROR oslo_messaging.rpc.server
2022-07-08 00:59:06.907 25830 INFO oslo_service.service [req-610a6fd5-5a14-47b6-b7a2-f80490141c47 - - - - -] Caught SIGTERM, stopping children
2022-07-08 00:59:06.912 25830 INFO oslo_service.service [req-610a6fd5-5a14-47b6-b7a2-f80490141c47 - - - - -] Waiting on 2 children to exit
2022-07-08 00:59:06.924 25830 INFO oslo_service.service [req-610a6fd5-5a14-47b6-b7a2-f80490141c47 - - - - -] Child 25886 killed by signal 15
2022-07-08 00:59:06.926 25830 INFO oslo_service.service [req-610a6fd5-5a14-47b6-b7a2-f80490141c47 - - - - -] Child 25885 killed by signal 15
2022-07-08 00:59:13.233 32454 INFO oslo_service.service [req-f9c2be9a-2d76-4fcc-bc38-04c312f276ba - - - - -] Starting 2 workers
2022-07-08 00:59:13.250 32521 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 00:59:13.300 32522 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 00:59:13.296 32454 WARNING oslo_config.cfg [req-f9c2be9a-2d76-4fcc-bc38-04c312f276ba - - - - -] Deprecated: Option "auth_strategy" from group "api" is deprecated for removal (
The only non-default choice, ``noauth2``, is for internal development and
testing purposes only and should not be used in deployments. This option and
its middleware, NoAuthMiddleware[V2_18], will be removed in a future release.
). Its value may be silently ignored in the future.
2022-07-08 00:59:13.337 32454 WARNING oslo_config.cfg [req-f9c2be9a-2d76-4fcc-bc38-04c312f276ba - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2022-07-08 01:00:04.052 32521 ERROR stevedore.extension [req-57288225-27ca-413d-8b4e-1e4227b32748 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Could not load 'oslo_cache.etcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw'
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server [req-57288225-27ca-413d-8b4e-1e4227b32748 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Exception during message handling: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 1655, in schedule_and_build_instances
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server accel_uuids=accel_uuids)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 1448, in build_and_run_instance
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server cctxt.cast(ctxt, 'build_and_run_instance', **kwargs)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 152, in cast
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server transport_options=self.transport_options)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in send
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 616, in _send
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server with self._get_connection(rpc_common.PURPOSE_SEND) as conn:
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 570, in _get_connection
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server purpose=purpose)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/common.py", line 423, in __init__
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server self.connection = connection_pool.get()
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 109, in get
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server return self.create()
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 146, in create
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server return self.connection_cls(self.conf, self.url, purpose)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 618, in __init__
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server self.ensure_connection()
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 735, in ensure_connection
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server self.connection.ensure_connection(errback=on_error)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 389, in ensure_connection
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server self._ensure_connection(*args, **kwargs)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 445, in _ensure_connection
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server callback, timeout=timeout
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 874, in _connection_factory
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server self._connection = self._establish_connection()
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 809, in _establish_connection
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server conn = self.transport.establish_connection()
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server conn.connect()
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 320, in connect
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server self.drain_events(timeout=self.connect_timeout)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 508, in drain_events
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout):
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 514, in blocking_read
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server callback(channel, method_sig, buf, None)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 521, in on_inbound_method
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server method_sig, payload, content,
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/abstract_channel.py", line 145, in dispatch_method
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server listener(*args)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 651, in _on_close
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server (class_id, method_id), ConnectionError)
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:00:04.270 32521 ERROR oslo_messaging.rpc.server
2022-07-08 01:04:53.610 32454 INFO oslo_service.service [req-f9c2be9a-2d76-4fcc-bc38-04c312f276ba - - - - -] Caught SIGTERM, stopping children
2022-07-08 01:04:53.616 32454 INFO oslo_service.service [req-f9c2be9a-2d76-4fcc-bc38-04c312f276ba - - - - -] Waiting on 2 children to exit
2022-07-08 01:04:53.627 32454 INFO oslo_service.service [req-f9c2be9a-2d76-4fcc-bc38-04c312f276ba - - - - -] Child 32521 killed by signal 15
2022-07-08 01:04:53.630 32454 INFO oslo_service.service [req-f9c2be9a-2d76-4fcc-bc38-04c312f276ba - - - - -] Child 32522 killed by signal 15
2022-07-08 01:04:59.323 2700 INFO oslo_service.service [req-65ffbd46-f9be-4b7c-a6b2-144e9932e98d - - - - -] Starting 2 workers
2022-07-08 01:04:59.338 2766 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:04:59.350 2700 WARNING oslo_config.cfg [req-65ffbd46-f9be-4b7c-a6b2-144e9932e98d - - - - -] Deprecated: Option "auth_strategy" from group "api" is deprecated for removal (
The only non-default choice, ``noauth2``, is for internal development and
testing purposes only and should not be used in deployments. This option and
its middleware, NoAuthMiddleware[V2_18], will be removed in a future release.
). Its value may be silently ignored in the future.
2022-07-08 01:04:59.356 2767 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:04:59.365 2700 WARNING oslo_config.cfg [req-65ffbd46-f9be-4b7c-a6b2-144e9932e98d - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2022-07-08 01:05:33.816 2767 ERROR stevedore.extension [req-a6193109-e4b0-44b5-a02a-bfa50509dca2 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Could not load 'oslo_cache.etcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw'
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server [req-a6193109-e4b0-44b5-a02a-bfa50509dca2 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Exception during message handling: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 1655, in schedule_and_build_instances
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server accel_uuids=accel_uuids)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 1448, in build_and_run_instance
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server cctxt.cast(ctxt, 'build_and_run_instance', **kwargs)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 152, in cast
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server transport_options=self.transport_options)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in send
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 616, in _send
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server with self._get_connection(rpc_common.PURPOSE_SEND) as conn:
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 570, in _get_connection
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server purpose=purpose)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/common.py", line 423, in __init__
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server self.connection = connection_pool.get()
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 109, in get
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server return self.create()
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 146, in create
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server return self.connection_cls(self.conf, self.url, purpose)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 618, in __init__
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server self.ensure_connection()
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 735, in ensure_connection
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server self.connection.ensure_connection(errback=on_error)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 389, in ensure_connection
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server self._ensure_connection(*args, **kwargs)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 445, in _ensure_connection
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server callback, timeout=timeout
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 874, in _connection_factory
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server self._connection = self._establish_connection()
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 809, in _establish_connection
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server conn = self.transport.establish_connection()
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server conn.connect()
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 320, in connect
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server self.drain_events(timeout=self.connect_timeout)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 508, in drain_events
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout):
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 514, in blocking_read
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server callback(channel, method_sig, buf, None)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 521, in on_inbound_method
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server method_sig, payload, content,
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/abstract_channel.py", line 145, in dispatch_method
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server listener(*args)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 651, in _on_close
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server (class_id, method_id), ConnectionError)
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:05:34.004 2767 ERROR oslo_messaging.rpc.server
2022-07-08 01:11:17.263 2700 INFO oslo_service.service [req-65ffbd46-f9be-4b7c-a6b2-144e9932e98d - - - - -] Caught SIGTERM, stopping children
2022-07-08 01:11:17.273 2700 INFO oslo_service.service [req-65ffbd46-f9be-4b7c-a6b2-144e9932e98d - - - - -] Waiting on 2 children to exit
2022-07-08 01:11:17.274 2700 INFO oslo_service.service [req-65ffbd46-f9be-4b7c-a6b2-144e9932e98d - - - - -] Child 2766 killed by signal 15
2022-07-08 01:11:17.278 2700 INFO oslo_service.service [req-65ffbd46-f9be-4b7c-a6b2-144e9932e98d - - - - -] Child 2767 killed by signal 15
2022-07-08 01:11:24.003 5546 INFO oslo_service.service [req-656245cc-7034-49ee-8fa5-31339ef2b978 - - - - -] Starting 2 workers
2022-07-08 01:11:24.019 5582 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:11:24.028 5546 WARNING oslo_config.cfg [req-656245cc-7034-49ee-8fa5-31339ef2b978 - - - - -] Deprecated: Option "auth_strategy" from group "api" is deprecated for removal (
The only non-default choice, ``noauth2``, is for internal development and
testing purposes only and should not be used in deployments. This option and
its middleware, NoAuthMiddleware[V2_18], will be removed in a future release.
). Its value may be silently ignored in the future.
2022-07-08 01:11:24.036 5583 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:11:24.038 5546 WARNING oslo_config.cfg [req-656245cc-7034-49ee-8fa5-31339ef2b978 - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2022-07-08 01:11:49.934 5583 ERROR stevedore.extension [req-9559f892-d837-4c51-98a0-cb378efc1372 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Could not load 'oslo_cache.etcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw'
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server [req-9559f892-d837-4c51-98a0-cb378efc1372 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Exception during message handling: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 1655, in schedule_and_build_instances
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server accel_uuids=accel_uuids)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 1448, in build_and_run_instance
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server cctxt.cast(ctxt, 'build_and_run_instance', **kwargs)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 152, in cast
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server transport_options=self.transport_options)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in send
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 616, in _send
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server with self._get_connection(rpc_common.PURPOSE_SEND) as conn:
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 570, in _get_connection
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server purpose=purpose)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/common.py", line 423, in __init__
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server self.connection = connection_pool.get()
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 109, in get
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server return self.create()
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 146, in create
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server return self.connection_cls(self.conf, self.url, purpose)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 618, in __init__
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server self.ensure_connection()
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 735, in ensure_connection
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server self.connection.ensure_connection(errback=on_error)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 389, in ensure_connection
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server self._ensure_connection(*args, **kwargs)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 445, in _ensure_connection
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server callback, timeout=timeout
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 874, in _connection_factory
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server self._connection = self._establish_connection()
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 809, in _establish_connection
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server conn = self.transport.establish_connection()
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server conn.connect()
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 320, in connect
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server self.drain_events(timeout=self.connect_timeout)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 508, in drain_events
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout):
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 514, in blocking_read
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server callback(channel, method_sig, buf, None)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 521, in on_inbound_method
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server method_sig, payload, content,
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/abstract_channel.py", line 145, in dispatch_method
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server listener(*args)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 651, in _on_close
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server (class_id, method_id), ConnectionError)
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:11:50.143 5583 ERROR oslo_messaging.rpc.server
2022-07-08 01:17:11.819 5546 INFO oslo_service.service [req-656245cc-7034-49ee-8fa5-31339ef2b978 - - - - -] Caught SIGTERM, stopping children
2022-07-08 01:17:11.836 5546 INFO oslo_service.service [req-656245cc-7034-49ee-8fa5-31339ef2b978 - - - - -] Waiting on 2 children to exit
2022-07-08 01:17:11.869 5546 INFO oslo_service.service [req-656245cc-7034-49ee-8fa5-31339ef2b978 - - - - -] Child 5582 killed by signal 15
2022-07-08 01:17:11.870 5546 INFO oslo_service.service [req-656245cc-7034-49ee-8fa5-31339ef2b978 - - - - -] Child 5583 killed by signal 15
2022-07-08 01:18:47.201 8868 INFO oslo_service.service [req-bd8defb3-624b-4008-ae9e-372bad93b72f - - - - -] Starting 2 workers
2022-07-08 01:18:47.243 9104 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:18:47.275 9103 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:18:47.325 8868 WARNING oslo_config.cfg [req-bd8defb3-624b-4008-ae9e-372bad93b72f - - - - -] Deprecated: Option "auth_strategy" from group "api" is deprecated for removal (
The only non-default choice, ``noauth2``, is for internal development and
testing purposes only and should not be used in deployments. This option and
its middleware, NoAuthMiddleware[V2_18], will be removed in a future release.
). Its value may be silently ignored in the future.
2022-07-08 01:18:47.390 8868 WARNING oslo_config.cfg [req-bd8defb3-624b-4008-ae9e-372bad93b72f - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2022-07-08 01:18:47.804 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 2.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:18:47.877 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 2.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:18:49.837 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 4.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:18:49.906 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 4.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:18:53.861 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 6.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:18:53.939 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 6.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:18:59.879 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 8.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:18:59.957 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 8.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:07.902 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 10.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:07.977 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 10.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:17.925 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 12.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:18.000 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 12.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:29.952 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 14.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:30.025 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 14.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:43.980 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 16.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:19:44.053 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 16.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:20:00.010 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 18.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:20:00.082 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 18.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:20:18.044 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 20.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:20:18.114 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 20.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:20:38.079 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 22.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:20:38.148 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 22.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:21:00.115 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 24.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:21:00.185 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 24.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:21:24.156 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 26.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:21:24.222 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 26.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:21:50.196 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 28.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:21:50.261 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 28.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:22:18.239 9104 ERROR oslo.messaging._drivers.impl_rabbit [req-9990be72-77ed-4924-9ecd-7c8ea83cfecb - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 30.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:22:18.304 9103 ERROR oslo.messaging._drivers.impl_rabbit [req-90000cde-6ac5-40dc-b825-9078b35379b3 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 30.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:22:26.391 8868 INFO oslo_service.service [req-bd8defb3-624b-4008-ae9e-372bad93b72f - - - - -] Caught SIGTERM, stopping children
2022-07-08 01:22:26.518 8868 INFO oslo_service.service [req-bd8defb3-624b-4008-ae9e-372bad93b72f - - - - -] Waiting on 2 children to exit
2022-07-08 01:22:26.523 8868 INFO oslo_service.service [req-bd8defb3-624b-4008-ae9e-372bad93b72f - - - - -] Child 9104 killed by signal 15
2022-07-08 01:22:26.589 8868 INFO oslo_service.service [req-bd8defb3-624b-4008-ae9e-372bad93b72f - - - - -] Child 9103 killed by signal 15
2022-07-08 01:25:10.902 2697 INFO oslo_service.service [req-9449b74b-519f-4206-af91-a28562c646ea - - - - -] Starting 2 workers
2022-07-08 01:25:10.937 3134 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:25:10.939 3135 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 01:25:10.934 2697 WARNING oslo_config.cfg [req-9449b74b-519f-4206-af91-a28562c646ea - - - - -] Deprecated: Option "auth_strategy" from group "api" is deprecated for removal (
The only non-default choice, ``noauth2``, is for internal development and
testing purposes only and should not be used in deployments. This option and
its middleware, NoAuthMiddleware[V2_18], will be removed in a future release.
). Its value may be silently ignored in the future.
2022-07-08 01:25:10.955 2697 WARNING oslo_config.cfg [req-9449b74b-519f-4206-af91-a28562c646ea - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2022-07-08 01:25:11.160 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 2.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:11.179 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 2.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:13.175 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 4.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:13.195 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 4.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:17.193 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 6.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:17.210 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 6.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:23.211 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 8.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:23.227 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 8.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:31.232 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 10.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:31.248 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 10.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:41.254 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 12.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:41.270 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 12.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:53.280 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 14.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:25:53.295 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 14.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:26:07.309 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 16.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:26:07.322 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 16.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:26:23.339 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 18.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:26:23.351 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 18.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:26:41.370 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 20.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:26:41.382 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 20.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:27:01.402 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 22.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:27:01.414 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 22.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:27:23.437 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 24.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:27:23.449 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 24.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:27:47.473 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 26.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:27:47.485 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 26.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:28:13.511 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 28.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:28:13.521 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 28.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:28:41.556 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 30.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:28:41.563 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 30.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:29:11.601 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:29:11.606 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:29:43.654 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:29:43.654 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:30:15.700 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:30:15.700 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:30:47.755 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:30:47.756 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:31:19.804 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:31:19.804 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:31:51.855 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:31:51.855 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:32:23.901 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:32:23.902 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:32:55.958 3134 ERROR oslo.messaging._drivers.impl_rabbit [req-58e45156-9a88-424f-8a34-ae9c6df58a08 - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:32:55.958 3135 ERROR oslo.messaging._drivers.impl_rabbit [req-402af6d8-0524-46c3-be35-9fedcdb67a5a - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 32.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 01:39:42.378 3135 ERROR stevedore.extension [req-d1dd37be-fbd5-40a1-82d6-cc794be38f3d 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Could not load 'oslo_cache.etcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw'
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server [req-d1dd37be-fbd5-40a1-82d6-cc794be38f3d 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Exception during message handling: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 1655, in schedule_and_build_instances
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server accel_uuids=accel_uuids)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 1448, in build_and_run_instance
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server cctxt.cast(ctxt, 'build_and_run_instance', **kwargs)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 152, in cast
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server transport_options=self.transport_options)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in send
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 616, in _send
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server with self._get_connection(rpc_common.PURPOSE_SEND) as conn:
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 570, in _get_connection
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server purpose=purpose)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/common.py", line 423, in __init__
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server self.connection = connection_pool.get()
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 109, in get
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server return self.create()
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 146, in create
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server return self.connection_cls(self.conf, self.url, purpose)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 618, in __init__
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server self.ensure_connection()
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 735, in ensure_connection
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server self.connection.ensure_connection(errback=on_error)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 389, in ensure_connection
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server self._ensure_connection(*args, **kwargs)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 445, in _ensure_connection
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server callback, timeout=timeout
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 874, in _connection_factory
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server self._connection = self._establish_connection()
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 809, in _establish_connection
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server conn = self.transport.establish_connection()
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server conn.connect()
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 320, in connect
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server self.drain_events(timeout=self.connect_timeout)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 508, in drain_events
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout):
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 514, in blocking_read
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server callback(channel, method_sig, buf, None)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 521, in on_inbound_method
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server method_sig, payload, content,
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/abstract_channel.py", line 145, in dispatch_method
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server listener(*args)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 651, in _on_close
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server (class_id, method_id), ConnectionError)
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 01:39:42.618 3135 ERROR oslo_messaging.rpc.server
2022-07-08 02:03:03.123 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [844ee0e5-c6c4-46ca-ad87-d0151cd6ac9b] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: (0, 0): (320) CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'
2022-07-08 02:03:03.137 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer
2022-07-08 02:03:03.140 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer
2022-07-08 02:03:04.190 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [844ee0e5-c6c4-46ca-ad87-d0151cd6ac9b] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:04.199 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
2022-07-08 02:03:04.229 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
2022-07-08 02:03:05.213 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:05.245 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:06.210 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [844ee0e5-c6c4-46ca-ad87-d0151cd6ac9b] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:07.229 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
2022-07-08 02:03:07.263 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
2022-07-08 02:03:07.598 3134 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer
2022-07-08 02:03:07.624 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] Connection failed: [Errno 111] ECONNREFUSED (retrying in 2.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:08.242 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:08.279 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:09.641 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] Connection failed: [Errno 111] ECONNREFUSED (retrying in 4.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:10.226 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [844ee0e5-c6c4-46ca-ad87-d0151cd6ac9b] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 6 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:10.257 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:10.301 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:13.486 3135 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer
2022-07-08 02:03:13.501 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] Connection failed: [Errno 111] ECONNREFUSED (retrying in 2.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:13.657 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] Connection failed: [Errno 111] ECONNREFUSED (retrying in 6.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:14.273 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
2022-07-08 02:03:14.317 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
2022-07-08 02:03:15.287 3134 ERROR oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:15.331 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:15.518 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] Connection failed: [Errno 111] ECONNREFUSED (retrying in 4.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:16.247 3135 ERROR oslo.messaging._drivers.impl_rabbit [-] [844ee0e5-c6c4-46ca-ad87-d0151cd6ac9b] AMQP server on laena.internal.teamorange.hu:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 8 seconds.: ConnectionRefusedError: [Errno 111] ECONNREFUSED
2022-07-08 02:03:17.354 3134 INFO oslo.messaging._drivers.impl_rabbit [-] [72484087-8b6c-4cfb-b0c4-52ef7f7fa16c] Reconnected to AMQP server on laena.internal.teamorange.hu:5672 via [amqp] client with port 46022.
2022-07-08 02:03:17.418 3135 INFO oslo.messaging._drivers.impl_rabbit [-] [6b7eedb1-5128-450b-accc-0f78a93b563e] Reconnected to AMQP server on laena.internal.teamorange.hu:5672 via [amqp] client with port 46026.
2022-07-08 02:03:24.285 3135 INFO oslo.messaging._drivers.impl_rabbit [-] [844ee0e5-c6c4-46ca-ad87-d0151cd6ac9b] Reconnected to AMQP server on laena.internal.teamorange.hu:5672 via [amqp] client with port 46054.
2022-07-08 02:03:57.591 3134 ERROR stevedore.extension [req-da7fd51a-06a4-420b-9987-17c5ceb0047f 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Could not load 'oslo_cache.etcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw'
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server [req-da7fd51a-06a4-420b-9987-17c5ceb0047f 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Exception during message handling: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 1655, in schedule_and_build_instances
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server accel_uuids=accel_uuids)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 1448, in build_and_run_instance
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server cctxt.cast(ctxt, 'build_and_run_instance', **kwargs)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 152, in cast
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server transport_options=self.transport_options)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in send
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 616, in _send
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server with self._get_connection(rpc_common.PURPOSE_SEND) as conn:
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 570, in _get_connection
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server purpose=purpose)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/common.py", line 423, in __init__
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server self.connection = connection_pool.get()
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 109, in get
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server return self.create()
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 146, in create
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server return self.connection_cls(self.conf, self.url, purpose)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 618, in __init__
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server self.ensure_connection()
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 735, in ensure_connection
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server self.connection.ensure_connection(errback=on_error)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 389, in ensure_connection
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server self._ensure_connection(*args, **kwargs)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 445, in _ensure_connection
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server callback, timeout=timeout
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 874, in _connection_factory
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server self._connection = self._establish_connection()
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 809, in _establish_connection
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server conn = self.transport.establish_connection()
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server conn.connect()
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 320, in connect
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server self.drain_events(timeout=self.connect_timeout)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 508, in drain_events
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout):
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 514, in blocking_read
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server callback(channel, method_sig, buf, None)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 521, in on_inbound_method
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server method_sig, payload, content,
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/abstract_channel.py", line 145, in dispatch_method
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server listener(*args)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 651, in _on_close
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server (class_id, method_id), ConnectionError)
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 02:03:57.799 3134 ERROR oslo_messaging.rpc.server
2022-07-08 02:05:17.158 2697 INFO oslo_service.service [req-9449b74b-519f-4206-af91-a28562c646ea - - - - -] Caught SIGTERM, stopping children
2022-07-08 02:05:17.264 2697 INFO oslo_service.service [req-9449b74b-519f-4206-af91-a28562c646ea - - - - -] Waiting on 2 children to exit
2022-07-08 02:05:17.267 2697 INFO oslo_service.service [req-9449b74b-519f-4206-af91-a28562c646ea - - - - -] Child 3134 killed by signal 15
2022-07-08 02:05:17.268 2697 INFO oslo_service.service [req-9449b74b-519f-4206-af91-a28562c646ea - - - - -] Child 3135 killed by signal 15
2022-07-08 02:05:30.091 19820 INFO oslo_service.service [req-aa2efa96-8b64-4db9-a74b-ddb22587846f - - - - -] Starting 2 workers
2022-07-08 02:05:30.113 19911 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 02:05:30.137 19820 WARNING oslo_config.cfg [req-aa2efa96-8b64-4db9-a74b-ddb22587846f - - - - -] Deprecated: Option "auth_strategy" from group "api" is deprecated for removal (
The only non-default choice, ``noauth2``, is for internal development and
testing purposes only and should not be used in deployments. This option and
its middleware, NoAuthMiddleware[V2_18], will be removed in a future release.
). Its value may be silently ignored in the future.
2022-07-08 02:05:30.165 19912 INFO nova.service [-] Starting conductor node (version 21.2.2-21.2.2~dev12-lp152.1.34)
2022-07-08 02:05:30.173 19820 WARNING oslo_config.cfg [req-aa2efa96-8b64-4db9-a74b-ddb22587846f - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal (
Support for image service configuration via standard keystoneauth1 Adapter
options was added in the 17.0.0 Queens release. The api_servers option was
retained temporarily to allow consumers time to cut over to a real load
balancing solution.
). Its value may be silently ignored in the future.
2022-07-08 02:06:19.381 19911 ERROR stevedore.extension [req-41a6d1e9-1ffd-496f-bf84-ddad3226456d 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Could not load 'oslo_cache.etcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw'
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server [req-41a6d1e9-1ffd-496f-bf84-ddad3226456d 741cb0b6280cb6fbedf1d8c6df4fc854b3e177e331441e019b961839089154d6 9b41f3e228984401a7642f94cd47dc73 - 4596c7f3b97740b3adfa8f82b6240654 4596c7f3b97740b3adfa8f82b6240654] Exception during message handling: amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 1655, in schedule_and_build_instances
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server accel_uuids=accel_uuids)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/rpcapi.py", line 1448, in build_and_run_instance
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server cctxt.cast(ctxt, 'build_and_run_instance', **kwargs)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 152, in cast
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server transport_options=self.transport_options)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 654, in send
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server transport_options=transport_options)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 616, in _send
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server with self._get_connection(rpc_common.PURPOSE_SEND) as conn:
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 570, in _get_connection
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server purpose=purpose)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/common.py", line 423, in __init__
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server self.connection = connection_pool.get()
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 109, in get
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server return self.create()
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/pool.py", line 146, in create
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server return self.connection_cls(self.conf, self.url, purpose)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 618, in __init__
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server self.ensure_connection()
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 735, in ensure_connection
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server self.connection.ensure_connection(errback=on_error)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 389, in ensure_connection
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server self._ensure_connection(*args, **kwargs)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 445, in _ensure_connection
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server callback, timeout=timeout
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 874, in _connection_factory
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server self._connection = self._establish_connection()
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 809, in _establish_connection
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server conn = self.transport.establish_connection()
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server conn.connect()
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 320, in connect
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server self.drain_events(timeout=self.connect_timeout)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 508, in drain_events
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout):
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 514, in blocking_read
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server callback(channel, method_sig, buf, None)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 521, in on_inbound_method
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server method_sig, payload, content,
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/abstract_channel.py", line 145, in dispatch_method
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server listener(*args)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/amqp/connection.py", line 651, in _on_close
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server (class_id, method_id), ConnectionError)
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server amqp.exceptions.AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.
2022-07-08 02:06:19.617 19911 ERROR oslo_messaging.rpc.server
-------------- next part --------------
[DEFAULT]
enabled_apis = osapi_compute,metadata
transport_url = rabbit://openstack:[ActualPasswordWouldBeHere]@laena.internal.teamorange.hu//
my_ip = 10.66.33.14
rootwrap_config=/etc/nova/rootwrap.conf
[api_database]
connection = mysql+pymysql://nova:[ActualPasswordWouldBeHere]@laena.internal.teamorange.hu/nova_api
[database]
connection = mysql+pymysql://nova:[ActualPasswordWouldBeHere]@laena.internal.teamorange.hu/nova
[api]
auth_strategy = keystone
[keystone_authtoken]
www_authenticate_uri = http://laena.internal.teamorange.hu:5000/
auth_url = http://laena.internal.teamorange.hu:5000/
memcached_servers = controller:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = [ActualPasswordWouldBeHere]
[vnc]
enabled = true
server_listen = $my_ip
server_proxyclient_address = $my_ip
[glance]
api_servers = http://laena.internal.teamorange.hu:9292
[oslo_concurrency]
lock_path = /var/run/nova
[placement]
region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://laena.internal.teamorange.hu:5000/v3
username = placement
password = [ActualPasswordWouldBeHere]
[neutron]
auth_url = http://laena.internal.teamorange.hu:5000
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = neutron
password = [ActualPasswordWouldBeHere]
service_metadata_proxy = true
metadata_proxy_shared_secret = [ActualPasswordWouldBeHere]
[cinder]
os_region_name = RegionOne
[scheduler]
discover_hosts_in_cells_interval = 300
More information about the openstack-discuss
mailing list