<div dir="ltr"><div class="gmail_default" style="font-family:courier new,monospace">Adding that rule outside of the namespaces seemed to do the trick! </div><div class="gmail_default" style="font-family:courier new,monospace">
<br></div><div class="gmail_default" style="font-family:courier new,monospace">Thanks Lorin.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Nov 26, 2013 at 9:21 AM, Lorin Hochstein <span dir="ltr"><<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div class="im">On Tue, Nov 26, 2013 at 9:18 AM, Samuel Winchenbach <span dir="ltr"><<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:'courier new',monospace">I will give it a shot. Does it matter that it is inside the namespace? </div>
</div></blockquote></div><div><br>I believe the iptables rules are local to network namespaces, so it would need to be inside of the dhcp namespace.<br><br> </div><div class="im"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr"><div style="font-family:'courier new',monospace"> I noticed this rule outside of all the namespaces: </div>
<div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">Chain POSTROUTING (policy ACCEPT 166K packets, 50M bytes)</font></div>
<div><font face="courier new, monospace"> pkts bytes target prot opt in out source destination </font></div><div><font face="courier new, monospace"> 176M 224G nova-api-POSTROUTING all -- * * <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> </font></div>
<div><font face="courier new, monospace"> 0 0 CHECKSUM udp -- * virbr0 <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> udp dpt:68 CHECKSUM fill</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">Thanks!</font></div><div><font face="courier new, monospace"><br></font></div></div></div></blockquote><div><br><br></div>
</div><div>That rule only applies to the virbr0 bridge, which isn't used by OpenStack. That rule would only be in effect if you started up a VM directly through libvirt and specified the use of libvirt networking. Since OpenStack VMs aren't attached to virbr0, the packets are never going to hit that rule.<span class="HOEnZb"><font color="#888888"><br>
<br></font></span></div><span class="HOEnZb"><font color="#888888"><div>Lorin<br></div></font></span><div><div class="h5"><div><br><br><br></div><div><br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr"><div><div><font face="courier new, monospace">
</font></div></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Nov 25, 2013 at 10:43 PM, Lorin Hochstein <span dir="ltr"><<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Mon, Nov 25, 2013 at 7:27 PM, Samuel Winchenbach <span dir="ltr"><<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:'courier new',monospace"><div>Hi All,</div><div><br>
</div>
<div>So I am experiencing a rather frustrating problem. I have three nodes running quantum dhcp agent (for high availability) and nova-compute. I found that certain VMs do not get an address if all three DHCP agents are running. Here is the output of dhcpdump on the tap interface to one of the VMs:</div>
<div><br></div><div>* WITHOUT test2 (worked - using test1 and test3)</div><div><a href="http://pastie.org/pastes/8508325/text" target="_blank">http://pastie.org/pastes/8508325/text</a><div style="width:16px;min-height:16px;display:inline-block">
</div><br></div><div><br></div><div>* WITH test2 (did not work - using test1, test2, test3)</div><div><a href="http://pastie.org/pastes/8508340/text" target="_blank">http://pastie.org/pastes/8508340/text</a></div>
<div><br></div><div>The log files look fine, no errors in dnsmasq.log or dhcp-agent.log</div><div><br></div><div>Here is what _seems_ to be in common when things do not work:</div>
<div>* the guest vm is using udhcpc as the DHCP client</div><div>* It is possible this only occurs when the DHCP agent is running on the same node as the VM.</div><div>
<br></div></div></div></blockquote><div><br></div></div></div><div>If it's only failing when the DHCP server is on the same host as VM, my first guess would be the infamous DHCP checksum issue.<br><br></div><div>Try adding this rule to the DHCP network namespace on each node that has a DHCP agent (change <uuid> to the appropriate id):<br>
<br>ip netns exec qdhcp-<uuid> iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill<br></div></div><br><br>This is happens if your machine is configured for hardware offloading of the
UDP checksum calculations, and your DHCP packets don't get proper checksums
if they don't cross a physical NIC. Certain DHCP clients barf if the
checksum is invalid.<span><font color="#888888"><br><br clear="all"></font></span></div><span><font color="#888888"><div class="gmail_extra"><br></div><div class="gmail_extra">Lorin<br></div>
<div class="gmail_extra">-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div>
<div>Nimbis Services, Inc.</div><div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div></div>
</div></font></span></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><div><div class="h5"><br><br clear="all"><br>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div><div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div>
</div>
</div></div></div></div>
</blockquote></div><br></div>