<div dir="ltr"><div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="line-height:16px;background-color:rgb(253,253,253)">Hi community, the weirdest thing happened to one of our openstack regions, running more than 200 vms</span></font></div>
<div class="gmail_default"><font face="courier new, monospace" color="#000000"><br></font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000">This region is :</font></div><div class="gmail_default">
<font face="courier new, monospace" color="#000000">* openstack essex 2012.1.4</font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000">* ubuntu 12.04.2</font></div><div class="gmail_default">
<font face="courier new, monospace" color="#000000">* Linux DC4-r59-02vms 3.2.0-49-generic #75-Ubuntu SMP Tue Jun 18 17:39:32 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux</font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000"><br style="line-height:16px;background-color:rgb(253,253,253)">
<span style="line-height:16px;background-color:rgb(253,253,253)">On Saturday all 16 nodes increased suddenly with no reason the network throughput at the exaclty same time and saturated themselves (peaking at 500Mb/s), it was something that involved only that vlan/region.</span><br style="line-height:16px;background-color:rgb(253,253,253)">
<br style="line-height:16px;background-color:rgb(253,253,253)"><span style="line-height:16px;background-color:rgb(253,253,253)">Something that was weird, is that graphics shows that lots of traffic was getting out from the vms via the vnet interfaces and from the host itself, but you see nothing inside the virtual machines, on the contrary. (im attaching graphics of everything) </span><br style="line-height:16px;background-color:rgb(253,253,253)">
<span style="line-height:16px;background-color:rgb(253,253,253)">We see lots of packages discarded from the tor and aggregation of this region, but we dont know what happened that caused this burst in bandwidth.</span></font></div>
<div class="gmail_default"><font face="courier new, monospace" color="#000000"><br style="line-height:16px;background-color:rgb(253,253,253)"><span style="line-height:16px;background-color:rgb(253,253,253)">Here's the graphics of:</span></font></div>
<div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="line-height:16px;background-color:rgb(253,253,253)">-----------------------</span></font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="line-height:16px;background-color:rgb(253,253,253)">The view from the compute node, increasing drastically the bandwidth usage, same happened to all 15 compute nodes.</span></font></div>
<div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px"><a href="http://oi58.tinypic.com/246261k.jpg">http://oi58.tinypic.com/246261k.jpg</a></span><br>
</span></font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px"><br></span></span></font></div><div class="gmail_default">
<font color="#000000" face="courier new, monospace"><span style="line-height:16px;background-color:rgb(253,253,253)">The view of the vms traffic from the compute perspective, of every vnet, in theory, increasing the traffic.</span></font></div>
<div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px"><a href="http://oi62.tinypic.com/2el9j0o.jpg">http://oi62.tinypic.com/2el9j0o.jpg</a></span><br>
</span></font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px"><br></span></span></font></div><div class="gmail_default">
<font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px">The view from inside of the vms, totally decreasing the traffic because of this kind of self saturation from every compute (this view is of one compute)</span></span></font></div>
<div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px"><a href="http://oi59.tinypic.com/vh991l.jpg">http://oi59.tinypic.com/vh991l.jpg</a></span><br>
</span></font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px"><br></span></span></font></div><div class="gmail_default">
<font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px">any tip or experience regarding what happened ?</span></span></font></div><div class="gmail_default">
<font face="courier new, monospace" color="#000000"><span style="background-color:rgb(253,253,253)"><span style="line-height:16px">thanks as allways</span></span></font></div><div><div dir="ltr"><font face="courier new, monospace" color="#000000"><br>
</font></div></div><div dir="ltr"><div class="gmail_default"><font face="courier new, monospace" color="#000000">--</font></div><div class="gmail_default"><font face="courier new, monospace" color="#000000">@lejandrito</font></div>
<br></div>
</div>