Dear mailinglist, We have the following behavour, which occured in about 3% of our networks. We have a default configured of 3 dhcp servers per subnet, but in about 3% of all networks, we have unexpectedly 4, from which one is in state "reserved_dhcp_port", also there is NO namespace on that control/network node. Setup: Openstack Rocky, Networkmode legacy # openstack port list --device-owner network:dhcp --network a2d4605d-997e-4807-9250-c0c80af3183e +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ | ID                                   | Name | MAC Address       | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ | 40237ca8-aed6-49ee-xxxx-49acf97797e4 | | fa:16:3e:e1:33:83 | ip_address='172.16.0.4', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | | 6a3898c6-6520-43e7-xxxx-e6b2639f2600 | | fa:16:3e:ff:f0:7b | ip_address='172.16.0.2', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | | 89479d32-881d-45b9-xxxx-1b41ed8fd703 | | fa:16:3e:ef:7d:dc | ip_address='172.16.0.6', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | | eaeb0ef7-48c6-43ea-xxxx-8615ed704e48 | | fa:16:3e:66:7c:96 | ip_address='172.16.0.3', subnet_id='e4380af7-13b4-xxxx-90ff-6b0a2818052b' | ACTIVE | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ full output of one of that port: # openstack port show 89479d32-881d-45b9-xxxx-1b41ed8fd703 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field                   | Value | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up          | UP | | allowed_address_pairs | | | binding_host_id         | ctl5 | | binding_profile | | | binding_vif_details     | datapath_type='system', ovs_hybrid_plug='True', port_filter='True' | | binding_vif_type        | ovs | | binding_vnic_type       | normal | | created_at              | 2020-01-16T11:22:14Z | | data_plane_status       | None | | description | | | device_id               | reserved_dhcp_port | | device_owner            | network:dhcp | | dns_assignment          | None | | dns_domain              | None | | dns_name                | None | | extra_dhcp_opts | | | fixed_ips               | ip_address='172.16.0.6', subnet_id='e4380af7-13b4-40f9-xxxx-xxxx' | | id                      | 89479d32-881d-45b9-xxxx-1b41ed8fd703 | | location                | Munch({'project': Munch({'domain_id': None, 'id': u'a772e4ab888e4fxxxx', 'name': 'admin', 'domain_name': 'Default'}), 'cloud': '', 'region_name': 'ch-zh1', 'zone': None}) | | mac_address             | fa:16:3e:ef:7d:xx | | name | | | network_id              | a2d4605d-997e-4807-xxxx-xxx | | port_security_enabled   | False | | project_id              | 232ecbeb96fd4663xxxx | | propagate_uplink_status | None | | qos_policy_id           | None | | resource_request        | None | | revision_number         | 8 | | security_group_ids | | | status                  | ACTIVE | | tags | | | trunk_details           | None | | updated_at              | 2020-02-14T09:16:57Z | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ctl5:~# ip netns |grep a2d4605d ctl5:~# *Workaround:* I can easily delete those ports manually and everything seems to be ok (openstack delete port <portid>) When I tried figuring out, what happens, when I disable dhcp in an affected subnet, the following happens (different example): The 6f34 is a reserved port (last line) # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-af71-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID                                   | Name | MAC Address       | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 23f9e5d7-75a2-440a-xxxx-d82000286fbe |      | fa:16:3e:8b:e2:a8 | ip_address='192.168.50.3', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | 352bc1a4-e002-440b-xxxx-0383cfb1632b |      | fa:16:3e:29:46:6b | ip_address='192.168.50.5', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | 6ee79608-7834-4015-xxxx-fda3f47b5e59 |      | fa:16:3e:0c:b6:60 | ip_address='192.168.50.4', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | 6f3435f8-23c0-427c-xxxx-fca1a6910aff |      | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ - I disable dhcp (openstack subnet set --no-dhcp <subnetid>), expected to remove all dhcp ports, but the what happens is that all "healthy" dhcp ports will be removed, the reserved port remains # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-af71-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID                                   | Name | MAC Address       | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 6f3435f8-23c0-427c-xxxx-fca1a6910aff |      | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ - when I now reenable dhcp, nothing happens, still the single, reserved port is existing, no new ports are created. Just the state went from ACTIVE to DOWN to ACTIVE # openstack subnet set --dhcp 882b360a-eae1-4736-a276-af752f552440 # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID                                   | Name | MAC Address       | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 6f3435f8-23c0-427c-xxxx-fca1a6910aff |      | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | DOWN   | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ s# openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID                                   | Name | MAC Address       | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 6f3435f8-23c0-427c-xxxx-fca1a6910aff |      | fa:16:3e:2d:35:b6 | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ - after deleting the port manually and disable/enable dhcp, 3 new working ports are created # openstack port delete 6f3435f8-23c0-427c-xxxx-fca1a6910aff # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 # openstack subnet set --no-dhcp 882b360a-eae1-4736-xxxx-af752f552440 # openstack subnet set --dhcp 882b360a-eae1-4736-xxxx-af752f552440 # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 # openstack port list --device-owner network:dhcp --network 8589eb04-228f-4ae1-xxxx-a0bc5335f9c5 +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | ID                                   | Name | MAC Address       | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ | 9fd63011-2b1f-410a-xxxx-40d42779c637 |      | fa:16:3e:15:be:fd | ip_address='192.168.50.4', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | cdd32840-7899-47b2-xxxx-7d36eae515a2 |      | fa:16:3e:4c:40:5c | ip_address='192.168.50.3', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | | dc1eeba2-4afd-4502-xxxx-7370d086e553 |      | fa:16:3e:29:29:5d | ip_address='192.168.50.2', subnet_id='882b360a-eae1-4736-xxxx-af752f552440' | ACTIVE | +--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+ Any ideas where those are coming from? thanks a lot Nicolas --