<div dir="ltr"><div><br></div><div>[ <a href="https://bugs.launchpad.net/nova/+bug/1270943">https://bugs.launchpad.net/nova/+bug/1270943</a> ]</div><div><br></div><div><div>I am running Grizzly on Ubuntu 13.04 (so the network service == Quantum). Nova runs Quantum with LibvirtHybridOVSBridgeDriver and LinuxOVSInterfaceDriver, while Quantum is configured to use GRE tunnels. Further, Quantum runs on a dedicated node and VLAN.</div>
<div><br></div><div>Starting in mid-December, new Compute nodes that came online were unable to spin new VMs. At the moment the nova-compute.log indicated that the instance had spawned successfully, the hypervisor system crashed with the following console dump message (last screen's worth): <a href="http://pastebin.com/004MYzvR">http://pastebin.com/004MYzvR</a>.</div>
<div><br></div><div>The installation of the Compute packages are controlled by puppet:</div><div><br></div><div><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 nova-common</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 nova-compute</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 nova-compute-kvm</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1.9.0-0ubuntu1 openvswitch-common</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1.9.0-0ubuntu1 openvswitch-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">datapath-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">dkms</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1.9.0-0ubuntu1 openvswitch-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">datapath-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">source</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1.9.0-0ubuntu1 openvswitch-switch</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:1.0.3-0ubuntu1 python-cinderclient</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1.4-0ubuntu1 python-glance</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:0.9.0-0ubuntu1.2 python-glanceclient</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1.4-0ubuntu1.</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1 python-keystone</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:0.2.3-0ubuntu2.2 python-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">keystoneclient</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 python-nova</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2.13.0-0ubuntu1 python-novaclient</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:1.1.0-0ubuntu1 python-oslo.config</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 python-quantum</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2.2.0-0ubuntu1 python-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">quantumclient</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:1.3.0-0ubuntu1 python-swiftclient</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 quantum-common</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 quantum-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">plugin-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">openvswitch</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">1:2013.1-0ubuntu2 quantum-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">plugin-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">openvswitch-</span><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">agent</span><br>
</div><div><br></div><div>The kernel being used is *not* controlled by Puppet and ends up being whatever the latest and greatest version is in raring-updates. The kernels in use: 3.8.0.34.52. I tried upgrading to 3.8.0.35.53 when it became available, but that had no effect.</div>
<div><br></div><div>I'm lost. No idea how to debug this.</div></div><br clear="all"><div><br></div>-- <br>\*..+.-<br>--Greg Chavez<br>+//..;};
</div>