<div dir="ltr">That was my instinct, but I've tried it both ways (toggling libvirt_use_virtio_for_bridge, restarting nova-compute, launching new instance), and vnc'd into the instance to confirmed that in one case the virtio_net drivers were loaded, and in another case, they weren't, and the result was the same. But it doesn't seem to be related. It's really baffling.<div>
<div><br></div><div>Lorin<br>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 3, 2013 at 4:47 PM, Joe Topjian <span dir="ltr"><<a href="mailto:joe.topjian@cybera.ca" target="_blank">joe.topjian@cybera.ca</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">That's really bizarre -- especially since it's only CentOS images. Do you think it might be something with virtio compatibility?<div>
<br></div><div>I'm hesitant to lean on it being a compute/controller issue since other images work.</div>
</div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 3, 2013 at 2:41 PM, Lorin Hochstein <span dir="ltr"><<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I've tested with multiple ones, including the CentOS6 image from that page, as well as several we have rolled on our own.<div>
<br></div><div>Right now I'm testing by manually putting on the IP by doing:</div>
<div><br></div><div>ip addr add <a href="http://10.40.0.4/16" target="_blank">10.40.0.4/16</a> broadcast 10.40.255.255 dev eth0</div><div><br></div><div>I can't ping out at all. If I try to arping out, and then tcpdump, just like in the DHCP case, I can see the ARP request and replies on vnet0 of the host:</div>
<div><br></div><div><div>root@c220-2:~# tcpdump -i vnet0 arp</div><div>tcpdump: WARNING: vnet0: no IPv4 address assigned</div><div>tcpdump: verbose output suppressed, use -v or -vv for full protocol decode</div></div><div>
<div>16:34:42.109067 ARP, Request who-has 10.40.0.1 (Broadcast) tell 10.40.0.4, length 28</div><div>16:34:42.109085 ARP, Request who-has 10.40.0.1 (Broadcast) tell 10.40.0.4, length 28</div><div>16:34:42.109216 ARP, Reply 10.40.0.1 is-at 54:78:1a:86:50:c9 (oui Unknown), length 46</div>
<div><br></div><div><br></div><div>But if I tcpdump on eth0 in the guest, I only see the arp requests, not the replies..</div><div><br></div><div><br></div><div>Lorin</div></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Wed, Apr 3, 2013 at 4:26 PM, Joe Topjian <span dir="ltr"><<a href="mailto:joe.topjian@cybera.ca" target="_blank">joe.topjian@cybera.ca</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">What CentOS images are you using? These have worked for me:<div><br></div><div><a href="https://github.com/rackerjoe/oz-image-build" target="_blank">https://github.com/rackerjoe/oz-image-build</a><br></div>
</div><div><div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Wed, Apr 3, 2013 at 2:13 PM, Lorin Hochstein <span dir="ltr"><<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi Joe:<div><br></div><div>It happens immediately thereafter. CentOS images have never worked on our setup.</div><div><br></div><div>Lorin<br><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 3, 2013 at 3:30 PM, Joe Topjian <span dir="ltr"><<a href="mailto:joe.topjian@cybera.ca" target="_blank">joe.topjian@cybera.ca</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Lorin,<div><br></div><div>Does this happen shortly after the guests were created? Or usually a few hours/days later? If the latter, are these guests seeing large amounts of bandwidth?</div>
<div>
<br></div><div>Thanks,</div><div>Joe</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Wed, Apr 3, 2013 at 1:16 PM, Lorin Hochstein <span dir="ltr"><<a href="mailto:lorin@nimbisservices.com" target="_blank">lorin@nimbisservices.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hi all:<div><br></div><div>I'm having a strange issue where networking on my CentOS guests isn't working properly, but things are working fine with my Ubuntu guests.</div>
<div><br></div><div>I'm running Folsom on Ubuntu 12.04, nova-network, not multi-host.<br>
</div><div><br></div><div>The first symptom is that CentOS instances don't get IP addresses via DHCP. If I trace the DHCP requests and replies using tcpdump, I can see the reply from dnsmasq reach the vnetX interface of the compute host, but it doesn't get to the eth0 interface of the compute host. (I'm at a loss here about how to debug something like that).</div>
<div><br></div><div>If I try to statically configure an IP address on the guest instead, networking still doesn't work. I can't ping anything on the subnet, and I don't even see the icmp traffic on vnetX of the host.</div>
<div><br></div><div>I've tried this twiddling the following options, but no change in behavior:</div><div><br></div><div>* Adding the following rule to nova-network node: <span style="font-family:arial,sans-serif;font-size:12.727272033691406px">iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM --checksum-fill </span></div>
<div style="font-family:arial,sans-serif;font-size:12.727272033691406px">* Adding the same rule to nova-compute node</div><div style="font-family:arial,sans-serif;font-size:12.727272033691406px">* Setting libvirt_use_virtio_for_bridge to "yes" and "no" (restarting nova-compute, re-launching instances)</div>
<div style="font-family:arial,sans-serif;font-size:12.727272033691406px"><div>* With and without vhost_net loaded in nova-compute (restarting nova-compute, re-launching instances)</div><div>* Disabling iIpv6 inside of the CentOS guest</div>
</div><div style="font-family:arial,sans-serif;font-size:12.727272033691406px"><br></div><div>Has anybody encountered this before?</div><span><font color="#888888"><div><br></div><div>Lorin</div><div><br></div>
<div>-- <br><div dir="ltr">
Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div><div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div></div>
</div></font></span></div>
<br></div></div>_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br></blockquote></div><span><font color="#888888"><br><br clear="all"><span><font color="#888888"><div><br></div>-- <br>Joe Topjian<div>Systems Administrator</div><div>Cybera Inc.</div><div><br></div><div>
<a href="http://www.cybera.ca" target="_blank">www.cybera.ca</a></div>
<div><br></div><div><font color="#666666"><span>Cybera</span><span> is a not-for-profit organization that works to spur and support innovation, for the economic benefit of Alberta, through the use of cyberinfrastructure.</span></font></div>
</font></span></font></span></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div>
<div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div>
</div>
</font></span></div></div></div>
</blockquote></div><br><br clear="all"><span><font color="#888888"><div><br></div>-- <br>Joe Topjian<div>Systems Administrator</div><div>Cybera Inc.</div><div><br></div><div><a href="http://www.cybera.ca" target="_blank">www.cybera.ca</a></div>
<div>
<br></div><div><font color="#666666"><span>Cybera</span><span> is a not-for-profit organization that works to spur and support innovation, for the economic benefit of Alberta, through the use of cyberinfrastructure.</span></font></div>
</font></span></div><span><font color="#888888">
</font></span></div></div></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div>
<div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div>
</div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Joe Topjian<div>Systems Administrator</div><div>Cybera Inc.</div><div><br></div><div><a href="http://www.cybera.ca" target="_blank">www.cybera.ca</a></div><div>
<br></div><div><font color="#666666"><span>Cybera</span><span> is a not-for-profit organization that works to spur and support innovation, for the economic benefit of Alberta, through the use of cyberinfrastructure.</span></font></div>
</div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div><div><a href="http://www.nimbisservices.com" target="_blank">www.nimbisservices.com</a></div>
</div>
</div></div></div></div>