[neutron] metadata-agents work but are not alive
Hi *, last week I upgraded my lab environment from Caracal to Epoxy. It's a package based deployment, so no kolla or ansible involved, but we use pacemaker for HA. The distro was upgraded from 22.04 to 24.04 as well. The upgrade of Openstack itself went quite smoothly, no issues at all. But for some reason, the neutron-metadata-agents don't update their status in the neutron db: MariaDB [(none)]> select id,`binary`,admin_state_up,heartbeat_timestamp,resources_synced from neutron.agents; +--------------------------------------+---------------------------+----------------+---------------------+------------------+ | id | binary | admin_state_up | heartbeat_timestamp | resources_synced | +--------------------------------------+---------------------------+----------------+---------------------+------------------+ | 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:43 | NULL | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:59 | NULL | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | neutron-openvswitch-agent | 1 | 2025-07-23 07:30:00 | NULL | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:48 | NULL | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | neutron-l3-agent | 1 | 2025-07-23 07:30:00 | NULL | | a52fb631-8fc3-4707-9fca-06165c56b481 | neutron-metadata-agent | 1 | 2025-07-17 09:37:14 | NULL | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | neutron-l3-agent | 1 | 2025-07-23 07:29:53 | NULL | | e8f8f679-f881-445b-b207-4beee679f6d3 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:53 | NULL | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | neutron-metadata-agent | 1 | 2025-07-17 09:50:40 | NULL | +--------------------------------------+---------------------------+----------------+---------------------+------------------+ So they're also shown as not alive: # openstack network agent list Could not load 'message_list': module 'zaqarclient.queues.v2.cli' has no attribute 'OldListMessages' Could not load 'message_post': module 'zaqarclient.queues.v2.cli' has no attribute 'OldPostMessages' +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+ | 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | Open vSwitch agent | controller02 | None | :-) | UP | neutron-openvswitch-agent | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | DHCP agent | controller01 | nova | :-) | UP | neutron-dhcp-agent | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | Open vSwitch agent | controller01 | None | :-) | UP | neutron-openvswitch-agent | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | DHCP agent | controller02 | nova | :-) | UP | neutron-dhcp-agent | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | L3 agent | controller02 | nova | :-) | UP | neutron-l3-agent | | a52fb631-8fc3-4707-9fca-06165c56b481 | Metadata agent | controller01 | None | XXX | UP | neutron-metadata-agent | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | L3 agent | controller01 | nova | :-) | UP | neutron-l3-agent | | e8f8f679-f881-445b-b207-4beee679f6d3 | Open vSwitch agent | compute002 | None | :-) | UP | neutron-openvswitch-agent | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | Metadata agent | controller02 | None | XXX | UP | neutron-metadata-agent | +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+ I turned on debug logs, nothing pops up in the logs, and the daemons work. Launching a new VM produces expected output in the agent log file, but I'll spare you the debug logs. The instance can reach the metadata service: $ curl http://169.254.169.254/latest/meta-data/instance-id i-0000000f RabbitMQ accepts the connections as well: 2025-07-18 19:33:27.773935+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672) has a client-provided name: neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1 2025-07-18 19:33:27.775762+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672 - neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1): user 'openstack' authenticated and granted access to vhost 'openstack' So I'm wondering what I might be missing here. I'd appreciate any pointers! Thanks! Eugen
Hi Eugen, Looks you are missing the fixes for https://bugs.launchpad.net/neutron/+bug/2112492 On Wed, Jul 23, 2025 at 1:14 PM Eugen Block <eblock@nde.ag> wrote:
Hi *,
last week I upgraded my lab environment from Caracal to Epoxy. It's a package based deployment, so no kolla or ansible involved, but we use pacemaker for HA. The distro was upgraded from 22.04 to 24.04 as well.
The upgrade of Openstack itself went quite smoothly, no issues at all. But for some reason, the neutron-metadata-agents don't update their status in the neutron db:
MariaDB [(none)]> select id,`binary`,admin_state_up,heartbeat_timestamp,resources_synced from neutron.agents;
+--------------------------------------+---------------------------+----------------+---------------------+------------------+ | id | binary | admin_state_up | heartbeat_timestamp | resources_synced |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+ | 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:43 | NULL | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:59 | NULL | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | neutron-openvswitch-agent | 1 | 2025-07-23 07:30:00 | NULL | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:48 | NULL | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | neutron-l3-agent | 1 | 2025-07-23 07:30:00 | NULL | | a52fb631-8fc3-4707-9fca-06165c56b481 | neutron-metadata-agent | 1 | 2025-07-17 09:37:14 | NULL | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | neutron-l3-agent | 1 | 2025-07-23 07:29:53 | NULL | | e8f8f679-f881-445b-b207-4beee679f6d3 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:53 | NULL | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | neutron-metadata-agent | 1 | 2025-07-17 09:50:40 | NULL |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
So they're also shown as not alive:
# openstack network agent list Could not load 'message_list': module 'zaqarclient.queues.v2.cli' has no attribute 'OldListMessages' Could not load 'message_post': module 'zaqarclient.queues.v2.cli' has no attribute 'OldPostMessages'
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+ | 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | Open vSwitch agent | controller02 | None | :-) | UP | neutron-openvswitch-agent | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | DHCP agent | controller01 | nova | :-) | UP | neutron-dhcp-agent | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | Open vSwitch agent | controller01 | None | :-) | UP | neutron-openvswitch-agent | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | DHCP agent | controller02 | nova | :-) | UP | neutron-dhcp-agent | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | L3 agent | controller02 | nova | :-) | UP | neutron-l3-agent | | a52fb631-8fc3-4707-9fca-06165c56b481 | Metadata agent | controller01 | None | XXX | UP | neutron-metadata-agent | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | L3 agent | controller01 | nova | :-) | UP | neutron-l3-agent | | e8f8f679-f881-445b-b207-4beee679f6d3 | Open vSwitch agent | compute002 | None | :-) | UP | neutron-openvswitch-agent | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | Metadata agent | controller02 | None | XXX | UP | neutron-metadata-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
I turned on debug logs, nothing pops up in the logs, and the daemons work. Launching a new VM produces expected output in the agent log file, but I'll spare you the debug logs. The instance can reach the metadata service:
$ curl http://169.254.169.254/latest/meta-data/instance-id i-0000000f
RabbitMQ accepts the connections as well:
2025-07-18 19:33:27.773935+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672) has a client-provided name: neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1 2025-07-18 19:33:27.775762+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672 - neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1): user 'openstack' authenticated and granted access to vhost 'openstack'
So I'm wondering what I might be missing here. I'd appreciate any pointers!
Thanks! Eugen
-- Thanks and Regards Yatin Karel
Thanks a lot for your quick response! Though I am a bit confused which changes exactly I need. I tried to apply all those changes mentioned manually, but now the metadata agent doesn't start: ImportError: cannot import name 'loopingcall' from 'neutron.common' (/usr/lib/python3/dist-packages/neutron/common/__init__.py) I compared the other neutron.common imports and see that there are corresponding files for them, like ipv6_utils.py or utils.py, but no loopingcall as required according to: https://review.opendev.org/c/openstack/neutron/+/952561/5/neutron/agent/meta... Zitat von Yatin Karel <ykarel@redhat.com>:
Hi Eugen,
Looks you are missing the fixes for https://bugs.launchpad.net/neutron/+bug/2112492
On Wed, Jul 23, 2025 at 1:14 PM Eugen Block <eblock@nde.ag> wrote:
Hi *,
last week I upgraded my lab environment from Caracal to Epoxy. It's a package based deployment, so no kolla or ansible involved, but we use pacemaker for HA. The distro was upgraded from 22.04 to 24.04 as well.
The upgrade of Openstack itself went quite smoothly, no issues at all. But for some reason, the neutron-metadata-agents don't update their status in the neutron db:
MariaDB [(none)]> select id,`binary`,admin_state_up,heartbeat_timestamp,resources_synced from neutron.agents;
+--------------------------------------+---------------------------+----------------+---------------------+------------------+ | id | binary | admin_state_up | heartbeat_timestamp | resources_synced |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+ | 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:43 | NULL | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:59 | NULL | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | neutron-openvswitch-agent | 1 | 2025-07-23 07:30:00 | NULL | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:48 | NULL | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | neutron-l3-agent | 1 | 2025-07-23 07:30:00 | NULL | | a52fb631-8fc3-4707-9fca-06165c56b481 | neutron-metadata-agent | 1 | 2025-07-17 09:37:14 | NULL | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | neutron-l3-agent | 1 | 2025-07-23 07:29:53 | NULL | | e8f8f679-f881-445b-b207-4beee679f6d3 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:53 | NULL | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | neutron-metadata-agent | 1 | 2025-07-17 09:50:40 | NULL |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
So they're also shown as not alive:
# openstack network agent list Could not load 'message_list': module 'zaqarclient.queues.v2.cli' has no attribute 'OldListMessages' Could not load 'message_post': module 'zaqarclient.queues.v2.cli' has no attribute 'OldPostMessages'
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+ | 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | Open vSwitch agent | controller02 | None | :-) | UP | neutron-openvswitch-agent | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | DHCP agent | controller01 | nova | :-) | UP | neutron-dhcp-agent | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | Open vSwitch agent | controller01 | None | :-) | UP | neutron-openvswitch-agent | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | DHCP agent | controller02 | nova | :-) | UP | neutron-dhcp-agent | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | L3 agent | controller02 | nova | :-) | UP | neutron-l3-agent | | a52fb631-8fc3-4707-9fca-06165c56b481 | Metadata agent | controller01 | None | XXX | UP | neutron-metadata-agent | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | L3 agent | controller01 | nova | :-) | UP | neutron-l3-agent | | e8f8f679-f881-445b-b207-4beee679f6d3 | Open vSwitch agent | compute002 | None | :-) | UP | neutron-openvswitch-agent | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | Metadata agent | controller02 | None | XXX | UP | neutron-metadata-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
I turned on debug logs, nothing pops up in the logs, and the daemons work. Launching a new VM produces expected output in the agent log file, but I'll spare you the debug logs. The instance can reach the metadata service:
$ curl http://169.254.169.254/latest/meta-data/instance-id i-0000000f
RabbitMQ accepts the connections as well:
2025-07-18 19:33:27.773935+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672) has a client-provided name: neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1 2025-07-18 19:33:27.775762+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672 - neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1): user 'openstack' authenticated and granted access to vhost 'openstack'
So I'm wondering what I might be missing here. I'd appreciate any pointers!
Thanks! Eugen
-- Thanks and Regards Yatin Karel
Hi Eugen, On Wed, Jul 23, 2025 at 2:14 PM Eugen Block <eblock@nde.ag> wrote:
Thanks a lot for your quick response! Though I am a bit confused which changes exactly I need. I tried to apply all those changes mentioned manually, but now the metadata agent doesn't start:
ImportError: cannot import name 'loopingcall' from 'neutron.common' (/usr/lib/python3/dist-packages/neutron/common/__init__.py)
I compared the other neutron.common imports and see that there are corresponding files for them, like ipv6_utils.py or utils.py, but no loopingcall as required according to:
https://review.opendev.org/c/openstack/neutron/+/952561/5/neutron/agent/meta...
You need to pull two patches:- https://review.opendev.org/c/openstack/neutron/+/952561 https://review.opendev.org/c/openstack/neutron/+/953064
For the loopingcall error you missed pulling the patch fully, that module was added in the above patch itself https://review.opendev.org/c/openstack/neutron/+/952561/5/neutron/common/loo...
Zitat von Yatin Karel <ykarel@redhat.com>:
Hi Eugen,
Looks you are missing the fixes for https://bugs.launchpad.net/neutron/+bug/2112492
On Wed, Jul 23, 2025 at 1:14 PM Eugen Block <eblock@nde.ag> wrote:
Hi *,
last week I upgraded my lab environment from Caracal to Epoxy. It's a package based deployment, so no kolla or ansible involved, but we use pacemaker for HA. The distro was upgraded from 22.04 to 24.04 as well.
The upgrade of Openstack itself went quite smoothly, no issues at all. But for some reason, the neutron-metadata-agents don't update their status in the neutron db:
MariaDB [(none)]> select id,`binary`,admin_state_up,heartbeat_timestamp,resources_synced from neutron.agents;
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
| id | binary | admin_state_up | heartbeat_timestamp | resources_synced |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
| 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:43 | NULL | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:59 | NULL | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | neutron-openvswitch-agent | 1 | 2025-07-23 07:30:00 | NULL | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:48 | NULL | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | neutron-l3-agent | 1 | 2025-07-23 07:30:00 | NULL | | a52fb631-8fc3-4707-9fca-06165c56b481 | neutron-metadata-agent | 1 | 2025-07-17 09:37:14 | NULL | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | neutron-l3-agent | 1 | 2025-07-23 07:29:53 | NULL | | e8f8f679-f881-445b-b207-4beee679f6d3 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:53 | NULL | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | neutron-metadata-agent | 1 | 2025-07-17 09:50:40 | NULL |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
So they're also shown as not alive:
# openstack network agent list Could not load 'message_list': module 'zaqarclient.queues.v2.cli' has no attribute 'OldListMessages' Could not load 'message_post': module 'zaqarclient.queues.v2.cli' has no attribute 'OldPostMessages'
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | Open vSwitch agent | controller02 | None | :-) | UP | neutron-openvswitch-agent | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | DHCP agent | controller01 | nova | :-) | UP | neutron-dhcp-agent | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | Open vSwitch agent | controller01 | None | :-) | UP | neutron-openvswitch-agent | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | DHCP agent | controller02 | nova | :-) | UP | neutron-dhcp-agent | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | L3 agent | controller02 | nova | :-) | UP | neutron-l3-agent | | a52fb631-8fc3-4707-9fca-06165c56b481 | Metadata agent | controller01 | None | XXX | UP | neutron-metadata-agent | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | L3 agent | controller01 | nova | :-) | UP | neutron-l3-agent | | e8f8f679-f881-445b-b207-4beee679f6d3 | Open vSwitch agent | compute002 | None | :-) | UP | neutron-openvswitch-agent | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | Metadata agent | controller02 | None | XXX | UP | neutron-metadata-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
I turned on debug logs, nothing pops up in the logs, and the daemons work. Launching a new VM produces expected output in the agent log file, but I'll spare you the debug logs. The instance can reach the metadata service:
$ curl http://169.254.169.254/latest/meta-data/instance-id i-0000000f
RabbitMQ accepts the connections as well:
2025-07-18 19:33:27.773935+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672) has a client-provided name: neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1 2025-07-18 19:33:27.775762+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672 - neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1): user 'openstack' authenticated and granted access to vhost 'openstack'
So I'm wondering what I might be missing here. I'd appreciate any
pointers!
Thanks! Eugen
-- Thanks and Regards Yatin Karel
-- Thanks and Regards Yatin Karel
Awesome, that fixed it. Thanks a lot for your quick help! Zitat von Yatin Karel <ykarel@redhat.com>:
Hi Eugen,
On Wed, Jul 23, 2025 at 2:14 PM Eugen Block <eblock@nde.ag> wrote:
Thanks a lot for your quick response! Though I am a bit confused which changes exactly I need. I tried to apply all those changes mentioned manually, but now the metadata agent doesn't start:
ImportError: cannot import name 'loopingcall' from 'neutron.common' (/usr/lib/python3/dist-packages/neutron/common/__init__.py)
I compared the other neutron.common imports and see that there are corresponding files for them, like ipv6_utils.py or utils.py, but no loopingcall as required according to:
https://review.opendev.org/c/openstack/neutron/+/952561/5/neutron/agent/meta...
You need to pull two patches:- https://review.opendev.org/c/openstack/neutron/+/952561 https://review.opendev.org/c/openstack/neutron/+/953064
For the loopingcall error you missed pulling the patch fully, that module was added in the above patch itself https://review.opendev.org/c/openstack/neutron/+/952561/5/neutron/common/loo...
Zitat von Yatin Karel <ykarel@redhat.com>:
Hi Eugen,
Looks you are missing the fixes for https://bugs.launchpad.net/neutron/+bug/2112492
On Wed, Jul 23, 2025 at 1:14 PM Eugen Block <eblock@nde.ag> wrote:
Hi *,
last week I upgraded my lab environment from Caracal to Epoxy. It's a package based deployment, so no kolla or ansible involved, but we use pacemaker for HA. The distro was upgraded from 22.04 to 24.04 as well.
The upgrade of Openstack itself went quite smoothly, no issues at all. But for some reason, the neutron-metadata-agents don't update their status in the neutron db:
MariaDB [(none)]> select id,`binary`,admin_state_up,heartbeat_timestamp,resources_synced from neutron.agents;
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
| id | binary | admin_state_up | heartbeat_timestamp | resources_synced |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
| 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:43 | NULL | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:59 | NULL | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | neutron-openvswitch-agent | 1 | 2025-07-23 07:30:00 | NULL | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | neutron-dhcp-agent | 1 | 2025-07-23 07:29:48 | NULL | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | neutron-l3-agent | 1 | 2025-07-23 07:30:00 | NULL | | a52fb631-8fc3-4707-9fca-06165c56b481 | neutron-metadata-agent | 1 | 2025-07-17 09:37:14 | NULL | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | neutron-l3-agent | 1 | 2025-07-23 07:29:53 | NULL | | e8f8f679-f881-445b-b207-4beee679f6d3 | neutron-openvswitch-agent | 1 | 2025-07-23 07:29:53 | NULL | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | neutron-metadata-agent | 1 | 2025-07-17 09:50:40 | NULL |
+--------------------------------------+---------------------------+----------------+---------------------+------------------+
So they're also shown as not alive:
# openstack network agent list Could not load 'message_list': module 'zaqarclient.queues.v2.cli' has no attribute 'OldListMessages' Could not load 'message_post': module 'zaqarclient.queues.v2.cli' has no attribute 'OldPostMessages'
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 0a5f439e-a31c-4e93-8861-6c4f97d354c0 | Open vSwitch agent | controller02 | None | :-) | UP | neutron-openvswitch-agent | | 0e3cea86-2cc3-455f-8eb4-e14a9897cad3 | DHCP agent | controller01 | nova | :-) | UP | neutron-dhcp-agent | | 19c944fb-bdc1-4aa0-9612-7794f924da3d | Open vSwitch agent | controller01 | None | :-) | UP | neutron-openvswitch-agent | | 20709aa0-2c55-4a18-8f7f-2c65c1bc1297 | DHCP agent | controller02 | nova | :-) | UP | neutron-dhcp-agent | | 4d6c8bff-eb30-46ca-becd-9a8cbb08e9cc | L3 agent | controller02 | nova | :-) | UP | neutron-l3-agent | | a52fb631-8fc3-4707-9fca-06165c56b481 | Metadata agent | controller01 | None | XXX | UP | neutron-metadata-agent | | aa3bd2e4-e3dd-4ad5-bc69-963e68e8567f | L3 agent | controller01 | nova | :-) | UP | neutron-l3-agent | | e8f8f679-f881-445b-b207-4beee679f6d3 | Open vSwitch agent | compute002 | None | :-) | UP | neutron-openvswitch-agent | | f04f347d-f376-4afc-99bc-0b4cd1c4c76a | Metadata agent | controller02 | None | XXX | UP | neutron-metadata-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
I turned on debug logs, nothing pops up in the logs, and the daemons work. Launching a new VM produces expected output in the agent log file, but I'll spare you the debug logs. The instance can reach the metadata service:
$ curl http://169.254.169.254/latest/meta-data/instance-id i-0000000f
RabbitMQ accepts the connections as well:
2025-07-18 19:33:27.773935+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672) has a client-provided name: neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1 2025-07-18 19:33:27.775762+00:00 [info] <0.7940.1> connection <0.7940.1> (192.168.160.8:60856 -> 192.168.160.9:5672 - neutron-metadata-agent:987277:5352e341-42df-4a92-a03f-1a58afab53a1): user 'openstack' authenticated and granted access to vhost 'openstack'
So I'm wondering what I might be missing here. I'd appreciate any
pointers!
Thanks! Eugen
-- Thanks and Regards Yatin Karel
-- Thanks and Regards Yatin Karel
participants (2)
-
Eugen Block
-
Yatin Karel