Open Stack

Hi community, the weirdest thing happened to one of our openstack regions,
running more than 200 vms

This region is :
* openstack essex 2012.1.4
* ubuntu 12.04.2
* Linux DC4-r59-02vms 3.2.0-49-generic #75-Ubuntu SMP Tue Jun 18 17:39:32
UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

On Saturday all 16 nodes increased suddenly with no reason the network
throughput at the exaclty same time and saturated themselves (peaking at
500Mb/s), it was something that involved only that vlan/region.

Something that was weird, is that graphics shows that lots of traffic was
getting out from the vms via the vnet interfaces and from the host itself,
but you see nothing inside the virtual machines, on the contrary. (im
attaching graphics of everything)
We see lots of packages discarded from the tor and aggregation of this
region, but we dont know what happened that caused this burst in bandwidth.

Here's the graphics of:
-----------------------
The view from the compute node, increasing drastically the bandwidth usage,
same happened to all 15 compute nodes.
http://oi58.tinypic.com/246261k.jpg

The view of the vms traffic from the compute perspective, of every vnet, in
theory, increasing the traffic.
http://oi62.tinypic.com/2el9j0o.jpg

The view from inside of the vms, totally decreasing the traffic because of
this kind of self saturation from every compute (this view is of one
compute)
http://oi59.tinypic.com/vh991l.jpg

any tip or experience regarding what happened ?
thanks as allways

--
@lejandrito
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140219/0ecf93a4/attachment.html>