[kolla-ansible][yoga][neutron][neutron-dhcp]ERRORS.... Need to understand
Hello. I come back to you to try to understand my problem. My installation is based on kolla-ansible, Yoga, Centos-Stream. For some time, without understanding why, neutron_dhcp no longer works on external networks and the networks of my students' projects. This generates many error messages, which are difficult to understand. Could this be a bug in this version? Before, it worked very well, but I have the impression that when the number of network increases (i’m not sure), from a moment, dhcp does not work anymore. What is the way to find the problem? Here are examples of errors that won't stop even if no one creates more instances of them 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent device, namespace, scope, broadcast) 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 272, in _wrap 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent r_call_timeout) 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 215, in remote_call 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent raise exc_type(*result[2]) 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap73cd30e2-35, namespace qdhcp-395f1edb-de6d-4dd8-b31b-69a53780c012. 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent 2023-04-04 20:36:52.871 7 INFO neutron.agent.dhcp.agent [-] Finished network 395f1edb-de6d-4dd8-b31b-69a53780c012 dhcp configuration 2023-04-04 20:36:52.872 7 INFO neutron.agent.dhcp.agent [-] Starting network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for 8db375e0-28d2-4379-9330-397dc9b4f2b1.: neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent Traceback (most recent call last): 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/dhcp/agent.py", line 218, in call_driver 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent rv = getattr(driver, action)(**action_kwargs) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 274, in enable 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent common_utils.wait_until_true(self._enable, timeout=300) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/common/utils.py", line 717, in wait_until_true 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent while not predicate(): 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 286, in _enable 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent interface_name = self.device_manager.setup(self.network) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1779, in setup 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent namespace=network.namespace) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/interface.py", line 152, in init_l3 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent device.addr.add(ip_cidr) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 537, in add 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent add_broadcast) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 823, in add_ip_address 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent device, namespace, scope, broadcast) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 272, in _wrap 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent r_call_timeout) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 215, in remote_call 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent raise exc_type(*result[2]) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent Thanks a lot Franck
Hi, This has almost the same trace as a bug I've been working on, https://bugs.launchpad.net/bugs/1988069 You could confirm by looking in the namespace: $ sudo ip netns exec qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1 ip a If you see any mtu value less than 68 then it is the same issue. Fix is to increase the mtu on network 8db375e0-28d2-4379-9330-397dc9b4f2b1 to at least 68, preferably 1450 or so depending on your overlay. If that's not it please file a new bug with all the above info provided. The other possibilities are a kernel issue, or the pyroute2 library, both of which have caused issues configuring IP addresses in namespaces. -Brian On 4/4/23 2:48 PM, Franck VEDEL wrote:
Hello.
I come back to you to try to understand my problem. My installation is based on kolla-ansible, Yoga, Centos-Stream. For some time, without understanding why, neutron_dhcp no longer works on external networks and the networks of my students' projects. This generates many error messages, which are difficult to understand.
Could this be a bug in this version? Before, it worked very well, but I have the impression that when the number of network increases (i’m not sure), from a moment, dhcp does not work anymore. What is the way to find the problem?
Here are examples of errors that won't stop even if no one creates more instances of them
2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent device, namespace, scope, broadcast) 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 272, in _wrap 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent r_call_timeout) 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 215, in remote_call 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent raise exc_type(*result[2]) 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap73cd30e2-35, namespace qdhcp-395f1edb-de6d-4dd8-b31b-69a53780c012. 2023-04-04 20:36:52.870 7 ERROR neutron.agent.dhcp.agent 2023-04-04 20:36:52.871 7 INFO neutron.agent.dhcp.agent [-] Finished network 395f1edb-de6d-4dd8-b31b-69a53780c012 dhcp configuration 2023-04-04 20:36:52.872 7 INFO neutron.agent.dhcp.agent [-] Starting network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for 8db375e0-28d2-4379-9330-397dc9b4f2b1.: neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent Traceback (most recent call last): 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/dhcp/agent.py", line 218, in call_driver 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent rv = getattr(driver, action)(**action_kwargs) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 274, in enable 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent common_utils.wait_until_true(self._enable, timeout=300) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/common/utils.py", line 717, in wait_until_true 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent while not predicate(): 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 286, in _enable 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent interface_name = self.device_manager.setup(self.network) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1779, in setup 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent namespace=network.namespace) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/interface.py", line 152, in init_l3 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent device.addr.add(ip_cidr) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 537, in add 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent add_broadcast) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 823, in add_ip_address 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent device, namespace, scope, broadcast) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 272, in _wrap 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent r_call_timeout) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 215, in remote_call 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent raise exc_type(*result[2]) 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-04 20:36:53.587 7 ERROR neutron.agent.dhcp.agent
Thanks a lot
Franck
Hi , and thanks a lot for your help. so $ sudo ip netns exec qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1 ip a 3401: tap1c220d25-b9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 68 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:20:ef:bc brd ff:ff:ff:ff:ff:ff inet 169.254.169.254/32 brd 169.254.169.254 scope global tap1c220d25-b9 valid_lft forever preferred_lft forever inet 10.1.42.1/24 brd 10.1.42.255 scope global tap1c220d25-b9 valid_lft forever preferred_lft forever $tail -f /var/log/kolla/neutron/neutron-dhcp-agent.log |grep 8db375e0-28d2-4379-9330-397dc9b4f2b1 2023-04-05 08:36:11.558 7 INFO neutron.agent.dhcp.agent [-] Starting network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration 2023-04-05 08:36:12.532 7 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for 8db375e0-28d2-4379-9330-397dc9b4f2b1.: neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-05 08:36:12.532 7 ERROR neutron.agent.dhcp.agent neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-05 08:36:12.534 7 INFO neutron.agent.dhcp.agent [-] Finished network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration openstack network show 8db375e0-28d2-4379-9330-397dc9b4f2b1 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2023-04-04T11:49:47Z | | description | | | dns_domain | None | | id | 8db375e0-28d2-4379-9330-397dc9b4f2b1 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | None | | is_vlan_transparent | None | | mtu | 68 | | name | NET1 | | port_security_enabled | True | | project_id | 6d6cbcbb6e8d4264bcd13dd8b3d79466 | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 456 | | qos_policy_id | None | | revision_number | 2 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | b72cf65c-1a22-4165-a55b-884bcc9c636a | | tags | | | updated_at | 2023-04-04T11:49:47Z | +---------------------------+———————————————————+ $openstack subnet show b72cf65c-1a22-4165-a55b-884bcc9c636a +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | allocation_pools | 10.1.42.1-10.1.42.253 | | cidr | 10.1.42.0/24 | | created_at | 2023-04-04T11:49:47Z | | description | | | dns_nameservers | | | dns_publish_fixed_ip | None | | enable_dhcp | True | | gateway_ip | 10.1.42.254 | | host_routes | | | id | b72cf65c-1a22-4165-a55b-884bcc9c636a | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | | | network_id | 8db375e0-28d2-4379-9330-397dc9b4f2b1 | | project_id | 6d6cbcbb6e8d4264bcd13dd8b3d79466 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2023-04-04T11:49:47Z | +----------------------+--------------------------------------+ Since yesterday, we have a lot of network (2 per students , 100 students) , so a lot of errors: In the log of dnsmask we can see this: Apr 5 08:55:02 dnsmasq-dhcp[2466928]: read /var/lib/neutron/dhcp/7d33855e-03a8-431b-9cf8-64f8e95b8f93/host Apr 5 08:55:02 dnsmasq-dhcp[2466928]: read /var/lib/neutron/dhcp/7d33855e-03a8-431b-9cf8-64f8e95b8f93/opts Apr 5 08:55:05 dnsmasq-dhcp[609053]: DHCPDISCOVER(tap5af0e639-f3) fa:16:3e:8d:86:b9 no address available Apr 5 08:55:05 dnsmasq-dhcp[4572]: DHCPDISCOVER(tap6fe70972-c9) fa:16:3e:6d:dc:c8 no address available You say: If that's not it please file a new bug with all the above info provided. How can I do that ? Thanks a lot Franck
Le 5 avr. 2023 à 00:00, Brian Haley <haleyb.dev@gmail.com> a écrit :
8db375e0-28d2-4379-9330-397dc9b4f2b1
Hi, On 4/5/23 2:57 AM, Franck VEDEL wrote:
Hi , and thanks a lot for your help.
so $ sudo ip netns exec qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1 ip a 3401: tap1c220d25-b9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 68qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:20:ef:bc brd ff:ff:ff:ff:ff:ff inet 169.254.169.254/32 brd 169.254.169.254 scope global tap1c220d25-b9 valid_lft forever preferred_lft forever inet 10.1.42.1/24 brd 10.1.42.255 scope global tap1c220d25-b9 valid_lft forever preferred_lft forever
Ok, so it is as I thought, someone has set a very small mtu on a network.
$tail -f /var/log/kolla/neutron/neutron-dhcp-agent.log |grep 8db375e0-28d2-4379-9330-397dc9b4f2b1 2023-04-05 08:36:11.558 7 INFO neutron.agent.dhcp.agent [-] Starting network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration 2023-04-05 08:36:12.532 7 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for 8db375e0-28d2-4379-9330-397dc9b4f2b1.: neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-05 08:36:12.532 7 ERROR neutron.agent.dhcp.agent neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-05 08:36:12.534 7 INFO neutron.agent.dhcp.agent [-] Finished network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration
openstack network show 8db375e0-28d2-4379-9330-397dc9b4f2b1 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2023-04-04T11:49:47Z | | description | | | dns_domain | None | | id | 8db375e0-28d2-4379-9330-397dc9b4f2b1 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | None | | is_vlan_transparent | None | | mtu |68 | | name | NET1 | | port_security_enabled | True | | project_id | 6d6cbcbb6e8d4264bcd13dd8b3d79466 | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 456 | | qos_policy_id | None | | revision_number | 2 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets |b72cf65c-1a22-4165-a55b-884bcc9c636a| | tags | | | updated_at | 2023-04-04T11:49:47Z | +---------------------------+———————————————————+
Increasing the mtu on this network to at least 1280 should fix the issue with DHCP, that is the IPv6 minimum mtu and most likely causing the issue. You should check other networks and use the same value, but it is probably 1450 given the overlay is vxlan. Since this is a known issue, there is no need for a new bug, but for your reference you can go to https://bugs.launchpad.net/neutron to search and file a new one. Thanks, -Brian
$openstack subnet showb72cf65c-1a22-4165-a55b-884bcc9c636a +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | allocation_pools | 10.1.42.1-10.1.42.253 | | cidr | 10.1.42.0/24 | | created_at | 2023-04-04T11:49:47Z | | description | | | dns_nameservers | | | dns_publish_fixed_ip | None | | enable_dhcp | True | | gateway_ip | 10.1.42.254 | | host_routes | | | id | b72cf65c-1a22-4165-a55b-884bcc9c636a | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | | | network_id | 8db375e0-28d2-4379-9330-397dc9b4f2b1 | | project_id | 6d6cbcbb6e8d4264bcd13dd8b3d79466 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2023-04-04T11:49:47Z | +----------------------+--------------------------------------+
Since yesterday, we have a lot of network (2 per students , 100 students) , so a lot of errors: In the log of dnsmask we can see this: Apr 5 08:55:02 dnsmasq-dhcp[2466928]: read /var/lib/neutron/dhcp/7d33855e-03a8-431b-9cf8-64f8e95b8f93/host Apr 5 08:55:02 dnsmasq-dhcp[2466928]: read /var/lib/neutron/dhcp/7d33855e-03a8-431b-9cf8-64f8e95b8f93/opts Apr 5 08:55:05 dnsmasq-dhcp[609053]: DHCPDISCOVER(tap5af0e639-f3) fa:16:3e:8d:86:b9 no address available Apr 5 08:55:05 dnsmasq-dhcp[4572]: DHCPDISCOVER(tap6fe70972-c9) fa:16:3e:6d:dc:c8 no address available
You say: If that's not it please file a new bug with all the above info provided. How can I do that ?
Thanks a lot
Franck
Le 5 avr. 2023 à 00:00, Brian Haley <haleyb.dev@gmail.com <mailto:haleyb.dev@gmail.com>> a écrit :
8db375e0-28d2-4379-9330-397dc9b4f2b1
Hi, thanks a lot Brian, and thanks to Sahid (for his help today, in french :) ) Problem is very small MTU. a user (student) created a network with a small mtu (on the day of the final exam) and for the rest of the day, no more dhcp.... The problem is addressed here: https://bugs.launchpad.net/neutron/+bug/1988069 <https://bugs.launchpad.net/neutron/+bug/1988069> Before I have time to update, if it happens again, I will know what to do… Thanks a lot Franck VEDEL Dép. Réseaux Informatiques & Télécoms IUT1 - Univ GRENOBLE Alpes 0476824462 Stages, Alternance, Emploi.
Le 5 avr. 2023 à 15:37, Brian Haley <haleyb.dev@gmail.com> a écrit :
Hi,
On 4/5/23 2:57 AM, Franck VEDEL wrote:
Hi , and thanks a lot for your help. so $ sudo ip netns exec qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1 ip a 3401: tap1c220d25-b9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 68qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:20:ef:bc brd ff:ff:ff:ff:ff:ff inet 169.254.169.254/32 brd 169.254.169.254 scope global tap1c220d25-b9 valid_lft forever preferred_lft forever inet 10.1.42.1/24 brd 10.1.42.255 scope global tap1c220d25-b9 valid_lft forever preferred_lft forever
Ok, so it is as I thought, someone has set a very small mtu on a network.
$tail -f /var/log/kolla/neutron/neutron-dhcp-agent.log |grep 8db375e0-28d2-4379-9330-397dc9b4f2b1 2023-04-05 08:36:11.558 7 INFO neutron.agent.dhcp.agent [-] Starting network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration 2023-04-05 08:36:12.532 7 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for 8db375e0-28d2-4379-9330-397dc9b4f2b1.: neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-05 08:36:12.532 7 ERROR neutron.agent.dhcp.agent neutron.privileged.agent.linux.ip_lib.InvalidArgument: Invalid parameter/value used on interface tap1c220d25-b9, namespace qdhcp-8db375e0-28d2-4379-9330-397dc9b4f2b1. 2023-04-05 08:36:12.534 7 INFO neutron.agent.dhcp.agent [-] Finished network 8db375e0-28d2-4379-9330-397dc9b4f2b1 dhcp configuration openstack network show 8db375e0-28d2-4379-9330-397dc9b4f2b1 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2023-04-04T11:49:47Z | | description | | | dns_domain | None | | id | 8db375e0-28d2-4379-9330-397dc9b4f2b1 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | None | | is_vlan_transparent | None | | mtu |68 | | name | NET1 | | port_security_enabled | True | | project_id | 6d6cbcbb6e8d4264bcd13dd8b3d79466 | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 456 | | qos_policy_id | None | | revision_number | 2 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets |b72cf65c-1a22-4165-a55b-884bcc9c636a| | tags | | | updated_at | 2023-04-04T11:49:47Z | +---------------------------+———————————————————+
Increasing the mtu on this network to at least 1280 should fix the issue with DHCP, that is the IPv6 minimum mtu and most likely causing the issue. You should check other networks and use the same value, but it is probably 1450 given the overlay is vxlan.
Since this is a known issue, there is no need for a new bug, but for your reference you can go to https://bugs.launchpad.net/neutron <https://bugs.launchpad.net/neutron> to search and file a new one.
Thanks,
-Brian
$openstack subnet showb72cf65c-1a22-4165-a55b-884bcc9c636a +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | allocation_pools | 10.1.42.1-10.1.42.253 | | cidr | 10.1.42.0/24 | | created_at | 2023-04-04T11:49:47Z | | description | | | dns_nameservers | | | dns_publish_fixed_ip | None | | enable_dhcp | True | | gateway_ip | 10.1.42.254 | | host_routes | | | id | b72cf65c-1a22-4165-a55b-884bcc9c636a | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | | | network_id | 8db375e0-28d2-4379-9330-397dc9b4f2b1 | | project_id | 6d6cbcbb6e8d4264bcd13dd8b3d79466 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2023-04-04T11:49:47Z | +----------------------+--------------------------------------+ Since yesterday, we have a lot of network (2 per students , 100 students) , so a lot of errors: In the log of dnsmask we can see this: Apr 5 08:55:02 dnsmasq-dhcp[2466928]: read /var/lib/neutron/dhcp/7d33855e-03a8-431b-9cf8-64f8e95b8f93/host Apr 5 08:55:02 dnsmasq-dhcp[2466928]: read /var/lib/neutron/dhcp/7d33855e-03a8-431b-9cf8-64f8e95b8f93/opts Apr 5 08:55:05 dnsmasq-dhcp[609053]: DHCPDISCOVER(tap5af0e639-f3) fa:16:3e:8d:86:b9 no address available Apr 5 08:55:05 dnsmasq-dhcp[4572]: DHCPDISCOVER(tap6fe70972-c9) fa:16:3e:6d:dc:c8 no address available You say: If that's not it please file a new bug with all the above info provided. How can I do that ? Thanks a lot Franck
Le 5 avr. 2023 à 00:00, Brian Haley <haleyb.dev@gmail.com <mailto:haleyb.dev@gmail.com> <mailto:haleyb.dev@gmail.com <mailto:haleyb.dev@gmail.com>>> a écrit :
8db375e0-28d2-4379-9330-397dc9b4f2b1
participants (2)
-
Brian Haley
-
Franck VEDEL