<div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote">On Fri, Mar 22, 2013 at 11:55 PM, Lorin Hochstein <span dir="ltr"><<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote">
<div class="im">On Thu, Mar 21, 2013 at 12:00 PM, Vishvananda Ishaya <span dir="ltr"><<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word">Well phooey:<div><br></div>
<div><div> 987 if network_ref['multi_host']:</div><div> 988 _add_dhcp_mangle_rule(dev)</div><div><br></div><div>The mangle rule is only added my nova-network in multihost mode.</div><div><br></div><div>
Can you verify whether or not adding the rule on the compute or network node fixes it?</div><div><br></div><div>That way we can either remove the check on multi_host or add it in plug_vif on the</div><div>compute host.</div>
<div><br></div></div></div></blockquote><div><br></div><div><br></div></div><div>I'll check on this and get back to you.</div><div><br></div><div>As an aside, note that we're *not* running with the vhost-net kernel module loaded, and the mangle rule only gets applied if this module is loaded: </div>
<div><br></div><div><a href="https://github.com/openstack/nova/blob/master/nova/network/linux_net.py#L885" target="_blank">https://github.com/openstack/nova/blob/master/nova/network/linux_net.py#L885</a><br></div><div><div>
<br></div><div>884 def _add_dhcp_mangle_rule(dev):</div><div>885 if not os.path.exists('/dev/vhost-net'):</div><div>886 return</div></div><div><br></div><div>So, either this situation can occur even without vhost-net, or I'm hitting a different issue.</div>
<div class="im">
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word">
<div><div></div><div>BTW:</div><div><div><br></div><div><blockquote type="cite"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div><div dir="ltr">iptables -D POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill</div></div></blockquote></div></div></div></blockquote></div></div></div></blockquote><br></div><div><br></div></div><div>
that should be -A not -D</div><div><br></div></div></div></blockquote><div><br></div></div><div>D'oh! I'll make sure that's correct when I do the testing.<span class=""><font color="#888888"><br></font></span></div>
<span class=""><font color="#888888"><div><br></div><div><br></div></font></span></div></div></div></blockquote><div><br></div><div><br></div><div>OK, I've tested this again, and I'm having the same problem. I'm able to get DHCP addresses for Ubuntu instances, but not CentOS ones. If I do a "tcpdump" on the "vnetX" interface, I can see the DHCP request and replies.<br>
</div><div><br></div><div>
<p class="">listening on vnet1, link-type EN10MB (Ethernet), capture size 65535 bytes</p>
<p class="">14:20:15.124839 IP 10.40.0.2.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:6b:d3:44, length 300</p>
<p class="">14:20:48.204962 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:5a:e9:f9, length 300</p>
<p class="">14:20:48.205023 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:5a:e9:f9, length 300</p>
<p class="">14:20:48.205596 IP 10.40.0.1.67 > 10.40.0.6.68: BOOTP/DHCP, Reply, length 320</p></div><div><br></div><div style>But if I do the tcpdump on eth0 inside the CentOS instances, I don't see the DHCP reply packets. They aren't making it from vnet1 to eth0. </div>
<div style><br></div><div style>This is Folsom with nova-network, running in FlatDHCP, non-multi host, on Ubuntu12.04. </div><div style><br></div><div style>I tried adding the iptables rule, but alas, it didn't resolve my issue.</div>
<div style><br></div><div style>iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM --checksum-fill<br></div><div style><br></div><div style>Here are the various things I've tried</div><div style><br></div>
<div style>* Adding the checksum rule to iptables nova-network node</div><div style>* Adding the checksum rule to the nova-compute node</div><div style>* Setting libvirt_use_virtio_for_bridge to "yes" and "no" (restarting nova-compute, re-launching instances)</div>
<div style>* With and without vhost_net loaded in nova-compute (restarting nova-compute, re-launching instances)</div><div style>* Disabling ipv6 inside of the CentOS guest</div><div style><br></div><div style>If I VNC into the instance and put a static IP on it, like this, it still doesn't have connectivity to the outside:</div>
<div style><br></div><div style>ip addr add <a href="http://10.40.0.2/16">10.40.0.2/16</a> broadcast 10.40.255.255 dev eth0<br></div><div style><br></div><div style>Since it works with Ubuntu but not CentOS guests, on the same compute node, I assume there's something about the configuration of the CentOS guest that isn't working properly with my setup. But, at this point, I'm really stumped.</div>
<div style><br></div><div style><br></div><div style>Lorin</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><font color="#888888"><div></div><div>Lorin</div></font></span><div><div class="h5">
<div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word">
<div><div></div><div>Vish</div><div><br></div><div><div><div><div>On Mar 20, 2013, at 1:43 PM, Lorin Hochstein <<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>> wrote:</div>
<br></div></div><blockquote type="cite"><div><div><div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote">On Wed, Mar 20, 2013 at 4:15 PM, Nathanael Burton <span dir="ltr"><<a href="mailto:nathanael.i.burton@gmail.com" target="_blank">nathanael.i.burton@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div><div>On Wed, Mar 20, 2013 at 3:51 PM, Lorin Hochstein <span dir="ltr"><<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div><div dir="ltr">I'm doing a Folsom deployment with FlatDHCP (not multihost).<div>
<br></div><div>When I try to boot a quantal image, the instance doesn't pick up the DHCP lease. I've confirmed that dnsmasq is sending out the DHCPOFFER, and I can see by tcpdump on the compute host that the DHCP packets are making it to the vnet0 interface.<br>
</div><div><br></div><div><br></div><div>Note that I tried adding this iptables rule as mentioned here <<a href="https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/issues/14" target="_blank">https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/issues/14</a>>, but that didn't resolve it.</div>
<div><br></div><div> iptables -D POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill<br></div><div><div><br></div><div><br></div><div><br></div><div><div>However, the problem goes away if I change this setting on the compute hosts in /etc/nova/nova.conf</div>
<div><br></div><div> libvirt_use_virtio_for_bridges=true</div><div><br></div><div>to:</div><div><br></div><div> libvirt_use_virtio_for_bridges=false<br></div></div><div><br></div><div><br></div><div>Anybody know what would cause this?<br>
</div><div><br><div><br></div><div>I'm on Ubuntu 12.04 with the cloud-archive packages, with KVM as the hypervisor</div><span><font color="#888888"><div><br></div><div><br></div></font></span></div></div></div></div>
</blockquote></div>You didn't restart nova-network without killing and restarting dnsmasq, did you?<br><br>Nate<br><br><br></div></div>
</blockquote></div><br>Of course not! (Well, maybe...). But just tried again, killing dnsmasq and restarting nova-network doesn't seem to help. I'm guessing the issue is confined to the compute node, and since I'm not running multihost, I don't think I even need to restart nova-network each time I make a virtio-related change on the compute node...</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">Lorin<br><br clear="all"><div><br></div>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div><div>
<a href="http://www.nimbisservices.com/" target="_blank">www.nimbisservices.com</a></div>
</div>
</div></div></div></div><div>
_______________________________________________<br>Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
</div></blockquote></div><br></div></div></blockquote></div></div></div><div><div class="h5"><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div>
<div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div>
</div>
</div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div><div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div>
</div>
</div></div>