Open Stack

Hello All,
 
I am having problems with some instances not receiving DCHPACK from my VM's dhcp server after extended periods of time. These instances are running heavy network, I/O, and RAM loads when the DHCPREQUEST does not receive a response. Note that this only happens to a handful of instances, not all instances running the exact same load. I have printed a relevant section of the syslog below.
 
My current environment is a multi-host flat dhcp network with nova-network running on each node. All servers and VM's are running Ubuntu 12.04 and using kvm/libvirt. I am also using large NFS servers to transfer big files to and from instances across the VM network. 
 
In the past, I used Eucalyptus and the same problem would occur. This tends to happen more frequently with more VMs and overall network load. 
 
If anyone has any ideas as to why the dhcp server stops responding to only a few instances occasionally please let me know.
 
 
Thanks,
 
Justin
 
 
-------------------
/var/log/syslog:
 
Oct 12 07:53:36 instance2 dhclient: DHCPREQUEST of 192.168.2.2 on eth0 to 192.168.2.3 port 67
Oct 12 07:53:36 instance2 dhclient: DHCPACK of 192.168.2.2 from 192.168.2.3
Oct 12 07:53:36 instance2 dhclient: bound to 192.168.2.2 -- renewal in 56 seconds.
Oct 12 07:54:32 instance2 dhclient: DHCPREQUEST of 192.168.2.2 on eth0 to 192.168.2.3 port 67
Oct 12 07:55:04  dhclient: last message repeated 4 times
Oct 12 07:55:15 instance2 dhclient: DHCPREQUEST of 192.168.2.2 on eth0 to 192.168.2.3 port 67
Oct 12 07:55:28 instance2 dhclient: DHCPREQUEST of 192.168.2.2 on eth0 to 255.255.255.255 port 67
Oct 12 07:55:37 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3
Oct 12 07:55:40 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
Oct 12 07:55:44 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9
Oct 12 07:55:53 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 18
Oct 12 07:56:11 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 14
Oct 12 07:56:25 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 20
Oct 12 07:56:45 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 17
Oct 12 07:57:02 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 12
Oct 12 07:57:14 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 14
Oct 12 07:57:28 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 15
Oct 12 07:57:43 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 21
Oct 12 07:58:04 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
Oct 12 07:58:12 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 16
Oct 12 07:58:28 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
Oct 12 07:58:36 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 12
Oct 12 07:58:48 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 14
Oct 12 07:59:02 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13
Oct 12 07:59:15 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 12
Oct 12 07:59:27 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 12
Oct 12 07:59:39 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 15
Oct 12 07:59:54 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 11
Oct 12 08:00:05 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 15
Oct 12 08:00:20 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 17
Oct 12 08:00:37 instance2 dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 1
Oct 12 08:00:38 instance2 dhclient: No DHCPOFFERS received.
Oct 12 08:01:13 instance2 kernel: [119711.712185] nfs: server 192.168.2.253 not responding, still trying
Oct 12 08:17:01 instance2 CRON[11515]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Oct 12 09:17:01 instance2 CRON[11519]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly) 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20121012/19df17a1/attachment.html>