Folks,

I am deploying OpenStack manually and have completed minimal development of the Ussuri.

My controller node can find my compute node and confirm there are compute hosts in the database with the instruction:

” openstack compute service list --service nova-compute”

But when I want to create an instance on the compute node, the status of the compute node just remains “build”.

And I try to look for faults from the “/var/log/nova/nova-compute.log” of the compute node:

2021-04-03 00:59:43.379 1432 INFO os_vif [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Successfully unplugged vif VIFBridge(active=True,address=fa:16:3e:55:a5:65,bridge_name='brq3169e77c-99',has_traffic_filtering=True,id=28248ef5-6ad6-44bf-b2ce-3fa7ac2371ef,network=Network(3169e77c-9945-454f-9562-6e9a55e1adce),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap28248ef5-6a')

2021-04-03 00:59:43.380 1432 INFO nova.virt.libvirt.driver [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 3de9dac7-6704-4df9-bd81-cfdadf3ba59a] Deletion of /var/lib/nova/instances/3de9dac7-6704-4df9-bd81-cfdadf3ba59a_del complete

2021-04-03 00:59:43.426 1432 INFO nova.compute.manager [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 3de9dac7-6704-4df9-bd81-cfdadf3ba59a] Took 0.06 seconds to destroy the instance on the hypervisor.

2021-04-03 00:59:43.980 1432 INFO nova.compute.manager [-] [instance: 3de9dac7-6704-4df9-bd81-cfdadf3ba59a] Took 0.55 seconds to deallocate network for instance.

2021-04-03 00:59:44.139 1432 INFO nova.scheduler.client.report [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Deleted allocation for instance 3de9dac7-6704-4df9-bd81-cfdadf3ba59a

2021-04-03 01:00:00.763 1432 INFO nova.compute.claims [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] Claim successful on node compute1

2021-04-03 01:00:00.991 1432 INFO nova.virt.libvirt.driver [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] Creating image

2021-04-03 01:00:01.421 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/nova/nova-dist.conf', '--config-file', '/etc/nova/nova.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmpqsoglfcp/privsep.sock']

2021-04-03 01:00:02.056 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Spawned new privsep daemon via rootwrap

2021-04-03 01:00:01.986 1861 INFO oslo.privsep.daemon [-] privsep daemon starting

2021-04-03 01:00:01.989 1861 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0

2021-04-03 01:00:01.991 1861 INFO oslo.privsep.daemon [-] privsep daemon running as pid 1861

2021-04-03 01:00:02.716 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/nova/nova-dist.conf', '--config-file', '/etc/nova/nova.conf', '--privsep_context', 'vif_plug_linux_bridge.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmp50wp4hs_/privsep.sock']

2021-04-03 01:00:03.404 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Spawned new privsep daemon via rootwrap

2021-04-03 01:00:03.324 1890 INFO oslo.privsep.daemon [-] privsep daemon starting

2021-04-03 01:00:03.326 1890 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0

2021-04-03 01:00:03.328 1890 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_NET_ADMIN/CAP_NET_ADMIN/none

2021-04-03 01:00:03.328 1890 INFO oslo.privsep.daemon [-] privsep daemon running as pid 1890

2021-04-03 01:00:03.620 1432 INFO os_vif [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Successfully plugged vif VIFBridge(active=False,address=fa:16:3e:e6:b4:89,bridge_name='brq3169e77c-99',has_traffic_filtering=True,id=c18f70ac-31ac-40eb-90f0-72c4e559f712,network=Network(3169e77c-9945-454f-9562-6e9a55e1adce),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tapc18f70ac-31')

2021-04-03 01:00:04.177 1432 INFO nova.compute.manager [-] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] VM 已开始 (Lifecycle Event)

2021-04-03 01:00:04.202 1432 INFO nova.compute.manager [req-f136de80-ad6d-497a-b190-4181d739c8ba - - - - -] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] VM 已暂停 (Lifecycle Event)

2021-04-03 01:00:04.246 1432 INFO nova.compute.manager [req-f136de80-ad6d-497a-b190-4181d739c8ba - - - - -] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] During sync_power_state the instance has a pending task (spawning). Skip.

2021-04-03 01:00:36.581 1432 INFO nova.compute.manager [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Updating bandwidth usage cache

2021-04-03 01:00:51.700 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 2.0 seconds): socket.timeout: timed out

2021-04-03 01:01:08.908 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 4.0 seconds): socket.timeout: timed out

2021-04-03 01:01:13.358 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 1 seconds.: socket.timeout: timed out

2021-04-03 01:01:27.960 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 6.0 seconds): socket.timeout: timed out

2021-04-03 01:01:37.983 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 2 seconds.: socket.timeout: timed out

2021-04-03 01:01:55.240 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 4 seconds.: socket.timeout: timed out

2021-04-03 01:02:05.262 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 8.0 seconds): socket.timeout: timed out

2021-04-03 01:02:05.262 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed

2021-04-03 01:02:05.263 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed

2021-04-03 01:02:21.319 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>

2021-04-03 01:02:31.335 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>

2021-04-03 01:02:41.351 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 6 seconds.: socket.timeout: timed out

2021-04-03 01:02:56.387 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: timed out. Trying again in 2 seconds.: socket.timeout: timed out

2021-04-03 01:03:06.406 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: timed out. Trying again in 2 seconds.: socket.timeout: timed out

2021-04-03 01:03:16.417 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 10.0 seconds): socket.timeout: timed out

2021-04-03 01:03:31.457 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 8 seconds.: socket.timeout: timed out

2021-04-03 01:03:41.475 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: timed out. Trying again in 4 seconds.: socket.timeout: timed out

2021-04-03 01:03:51.495 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: timed out. Trying again in 4 seconds.: socket.timeout: timed out

2021-04-03 01:04:09.540 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: timed out. Trying again in 6 seconds.: socket.timeout: timed out

2021-04-03 01:04:19.558 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: timed out. Trying again in 6 seconds.: socket.timeout: timed out

2021-04-03 01:04:29.565 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 12.0 seconds): socket.timeout: timed out

2021-04-03 01:04:39.570 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 10 seconds.: socket.timeout: timed out

2021-04-03 01:04:59.594 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>

2021-04-03 01:05:09.607 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>

2021-04-03 01:05:09.610 1432 WARNING nova.virt.libvirt.driver [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] Timeout waiting for [('network-vif-plugged', 'c18f70ac-31ac-40eb-90f0-72c4e559f712')] for instance with vm_state building and task_state spawning.: eventlet.timeout.Timeout: 300 seconds

And then controller node couldn’t find the compute node

I don’t know why these happen. I have checked the port of 5672 is open on controller node and Rabbitmq is running.

Please help me.