Is the cloud running on a single server? How did you deploy it?
Your description indicates that instances are unable to reach their DHCP servers. They don't get an IP address and consequently can't be ping'ed.
Start with openstack network agent list to see if there is an obvious problem with the DHCP agent(s). Also check if /dnsmasq/ processes (which normally implement the DHCP service) are running; there should be one per Neutron network. Also check the relevant log files and the /dnsmasq /configuration and lease files.
If there is no problem at that level, I would guess network connectivity between instances and DHCP servers is broken. That is harder to troubleshoot, as there are many moving parts and many implementation options.
On 2022/10/21 5:51 AM, Albert Shih wrote:
Hi,
I've a small openstack installation running Victoria on Ubuntu 20 LTS.
After a update (not upgrade) of Ubuntu, I loose all network on all my instance.
On a instance console I got :
[[0;32m OK [0m] Started [0;1;39mifup for ens3[0m. Starting [0;1;39mRaise network interfaces[0m... [[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (1min 4s / 6min 2s) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (1min 4s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (1min 5s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (1min 5s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Raise ne���ork interfaces (1min 6s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m] A start job is running for Raise ne���ork interfaces (1min 6s / 6min 2s) M[K[ [0;31m*[0m] A start job is running for Raise ne���ork interfaces (1min 7s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m] A start job is running for Raise ne���ork interfaces (1min 7s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Raise ne���ork interfaces (1min 8s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (1min 8s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (1min 9s / 6min 2s) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (1min 9s / 6min 2s) M[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���rk interfaces (1min 10s / 6min 2s) M[K[[0m[0;31m* [0m] A start job is running for Raise ne���rk interfaces (1min 10s / 6min 2s) M[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���rk interfaces (1min 11s / 6min 2s) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���rk interfaces (1min 11s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���rk interfaces (1min 12s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���rk interfaces (1min 12s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Raise ne���rk interfaces (1min 13s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m] A start job is running for Raise ne���rk interfaces (1min 13s / 6min 2s) M[K[ [0;31m*[0m] A start job is running for Raise ne���rk interfaces (1min 14s / 6min 2s) ...... ...... M[K[ [0;31m*[0;1;31m*[0m] A start job is running for Raise ne���rk interfaces (2min 59s / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Raise network interfaces (3min / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise network interfaces (3min / 6min 2s) M[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (3min 1s / 6min 2s) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (3min 1s / 6min 2s) M[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (3min 2s / 6min 2s) M[K[[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (3min 2s / 6min 2s) M[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (3min 3s / 6min 2s) M[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Raise ne���ork interfaces (3min 3s / 6min 2s) M[K[[0;32m OK [0m] Finished [0;1;39mRaise network interfaces[0m. [K[[0;32m OK [0m] Reached target [0;1;39mNetwork[0m. Starting [0;1;39mInitial cloud-ini��� (metadata service crawler)[0m... [ 186.709234] cloud-init[514]: Cloud-init v. 20.4.1 running 'init' at Thu, 20 Oct 2022 20:12:41 +0000. Up 186.69 seconds. [ 186.727021] cloud-init[514]: ci-info: ++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++ [ 186.728928] cloud-init[514]: ci-info: +--------+------+------------------------------+-----------+-------+-------------------+ [ 186.730709] cloud-init[514]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address | [ 186.732694] cloud-init[514]: ci-info: +--------+------+------------------------------+-----------+-------+-------------------+ [ 186.734765] cloud-init[514]: ci-info: | ens3 | True | fe80::f816:3eff:fec3:2e6c/64 | . | link | fa:16:3e:c3:2e:6c | [ 186.736681] cloud-init[514]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . | [ 186.738473] cloud-init[514]: ci-info: | lo | True | ::1/128 | . | host | . | [ 186.740246] cloud-init[514]: ci-info: +--------+------+------------------------------+-----------+-------+-------------------+ [ 186.741723] cloud-init[514]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++ [ 186.742306] cloud-init[514]: ci-info: +-------+-------------+---------+-----------+-------+ [ 186.742881] cloud-init[514]: ci-info: | Route | Destination | Gateway | Interface | Flags | [ 186.743460] cloud-init[514]: ci-info: +-------+-------------+---------+-----------+-------+ [ 186.744075] cloud-init[514]: ci-info: | 1 | fe80::/64 | :: | ens3 | U | [ 186.744681] cloud-init[514]: ci-info: | 3 | local | :: | ens3 | U | [ 186.745255] cloud-init[514]: ci-info: | 4 | multicast | :: | ens3 | U | [ 186.745833] cloud-init[514]: ci-info: +-------+-------------+---------+-----------+-------+
in the log of neutron-linuxbridge-agent.log it seem everything should work :
2022-10-20 20:17:54.967 59611 INFO neutron.plugins.ml2.drivers.agent._common_agent [req-9e6b23d3-1e0e-4c48-8f0e-30d761233188 - - - - -] Port tap4f68bcf1-4d updated. Details: {'device': 'tap4f68bcf1-4d', 'network_id': '9223f9ff-7ab0-4268-9b7f-3b5966625c65', 'port_id': '4f68bcf1-4d32-4ad3-a0df-0415f7170c5d', 'mac_address': 'fa:16:3e:c3:2e:6c', 'admin_state_up': True, 'network_type': 'flat', 'segmentation_id': None, 'physical_network': 'provider', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '12cdcc06-bbf0-48f2-b303-bdfab460c696', 'ip_address': '145.238.137.52'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False}
But I'm unable to ping any instance.
Any idea ?
Regards