[Openstack] neutron-dhcp-agent not completely populating dnsmasq config

Matt Davis mattd5574 at gmail.com
Fri Jul 17 20:28:08 UTC 2015


I'm running into an issue with an icehouse installation with some hosts not
receiving IP addresses from the dhcp agent.  I've traced it as far as the
host file that dnsmasq uses and found that not all of the addresses are
ending up in the host file.  For example, I created 5 new VMs as follows:

nova list
+--------------------------------------+------+--------+------------+-------------+-----------------------+
| ID                                   | Name | Status | Task State | Power
State | Networks              |
+--------------------------------------+------+--------+------------+-------------+-----------------------+
| ababd9a3-95a3-47f1-b38e-9e54c04372d5 | svr0 | ACTIVE | -          |
Running     | zeroconf=192.168.4.15 |
| 9d640125-e84e-46d8-a686-e0678e35e4d3 | svr1 | ACTIVE | -          |
Running     | zeroconf=192.168.4.16 |
| 4e32e64e-83b9-45e7-935e-01f1af4fc803 | svr2 | ACTIVE | -          |
Running     | zeroconf=192.168.4.17 |
| bd00ece0-7c0d-4c82-a7da-32a89f4ba7b6 | svr3 | ACTIVE | -          |
Running     | zeroconf=192.168.4.18 |
| 1a822bf6-1d3d-42b8-8e6e-d3dadc901b51 | svr4 | ACTIVE | -          |
Running     | zeroconf=192.168.4.19 |
+--------------------------------------+------+--------+------------+-------------+-----------------------+

Two of them received IP addresses (svr1 and svr3) and the others timed out
waiting for a DHCP response.  Syslog shows that dnsmasq sees the requests
but has no record for the MAC addresses for those VMs.  Looking in the host
file used by dnsmasq, I see the two entries for the working VMs and no
entries for the ones that failed:

root at head2:/home/mdavis# cat
/var/lib/neutron/dhcp/f4795c04-8b46-4d87-9bee-484c56e3c62d/host
fa:16:3e:28:71:be,host-192-168-4-1.openstacklocal,192.168.4.1
fa:16:3e:f6:4c:21,host-192-168-4-2.openstacklocal,192.168.4.2
fa:16:3e:aa:bd:82,host-192-168-4-16.openstacklocal,192.168.4.16
fa:16:3e:9d:1c:35,host-192-168-4-18.openstacklocal,192.168.4.18

Looking at the log for neutron-dhcp-agent, I see it building the file but
no mention of my missing addresses:

2015-07-17 20:04:48.324 3877 DEBUG neutron.agent.linux.dhcp
[req-94b3f85c-ca88-43f1-adf6-12a470b99136 None] Building host file:
/var/lib/neutron/dhcp/f4795c04-8b46-4d87-9bee-484c56e3c62d/host
_output_hosts_file
/usr/lib/python2.7/dist-packages/neutron/agent/linux/dhcp.py:504
2015-07-17 20:04:48.324 3877 DEBUG neutron.agent.linux.dhcp
[req-94b3f85c-ca88-43f1-adf6-12a470b99136 None] Adding fa:16:3e:28:71:be :
host-192-168-4-1.openstacklocal : 192.168.4.1 _output_hosts_file
/usr/lib/python2.7/dist-packages/neutron/agent/linux/dhcp.py:512
2015-07-17 20:04:48.325 3877 DEBUG neutron.agent.linux.dhcp
[req-94b3f85c-ca88-43f1-adf6-12a470b99136 None] Adding fa:16:3e:f6:4c:21 :
host-192-168-4-2.openstacklocal : 192.168.4.2 _output_hosts_file
/usr/lib/python2.7/dist-packages/neutron/agent/linux/dhcp.py:512
2015-07-17 20:04:48.325 3877 DEBUG neutron.agent.linux.dhcp
[req-94b3f85c-ca88-43f1-adf6-12a470b99136 None] Adding fa:16:3e:aa:bd:82 :
host-192-168-4-16.openstacklocal : 192.168.4.16 _output_hosts_file
/usr/lib/python2.7/dist-packages/neutron/agent/linux/dhcp.py:512
2015-07-17 20:04:48.326 3877 DEBUG neutron.agent.linux.dhcp
[req-94b3f85c-ca88-43f1-adf6-12a470b99136 None] Adding fa:16:3e:9d:1c:35 :
host-192-168-4-18.openstacklocal : 192.168.4.18 _output_hosts_file
/usr/lib/python2.7/dist-packages/neutron/agent/linux/dhcp.py:512
2015-07-17 20:04:48.327 3877 DEBUG neutron.agent.linux.dhcp
[req-94b3f85c-ca88-43f1-adf6-12a470b99136 None] Done building host file
/var/lib/neutron/dhcp/f4795c04-8b46-4d87-9bee-484c56e3c62d/host
_output_hosts_file
/usr/lib/python2.7/dist-packages/neutron/agent/linux/dhcp.py:526

It looks like _output_hosts_file() is iterating over an incomplete list.

I waited several minutes for the contents of the host file to change.  No
update to the host file.  I then restarted neutron-dhcp-agent the host file
fills in as expected:

cat /var/lib/neutron/dhcp/f4795c04-8b46-4d87-9bee-484c56e3c62d/host
fa:16:3e:df:f8:09,host-192-168-4-17.openstacklocal,192.168.4.17
fa:16:3e:28:71:be,host-192-168-4-1.openstacklocal,192.168.4.1
fa:16:3e:a4:80:9c,host-192-168-4-19.openstacklocal,192.168.4.19
fa:16:3e:ca:6d:06,host-192-168-4-15.openstacklocal,192.168.4.15
fa:16:3e:aa:bd:82,host-192-168-4-16.openstacklocal,192.168.4.16
fa:16:3e:9d:1c:35,host-192-168-4-18.openstacklocal,192.168.4.18
fa:16:3e:f6:4c:21,host-192-168-4-2.openstacklocal,192.168.4.2

I rebooted the VMs and they all received addresses and worked as expected.

What is the event that triggers the dhcp agent to refresh its listing of
ports, and shouldn't it always do this when a new VM is spun up?  This is a
multinode test system, but it's not busy (I am the only user and only those
5 VMs were active for the test) and I don't see any other behavior problems.

Thanks,

-Matt Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150717/f61847ad/attachment.html>


More information about the Openstack mailing list