[Openstack-operators] [Openstack] Extreme network throughput tuning with KVM as hypervisor
george.shuklin at gmail.com
Thu Jan 16 10:06:08 UTC 2014
Upgrade OVS to version >= 1.11.
I don't know it will work with neutron or not, but OVS 1.10 (and 1.4,
and any version <1.11) is just not production ready.
Way to reproduce problem:
hping3 --flood --rand-source ANY_FLOATING_IP.
It kills any hosts with older OVS up to the level 'connection timeout'.
On 11.01.2014 21:12, Alejandro Comisario wrote:
> Well, its been a long time since we use nova with KVM, we got over the
> many thousand vms, and still, something doesnt feel right.
> We are using ubuntu 12.04 kernel 3.2.0-[40-48], tuned sysctl with lots
> of parameters, and everything ... works, you can say, quite well.
> But here's the deal, we have an special networking scenario that is,
> EVERYTHING IS APIS, everything is throughput, no bandwidth.
> Every 2x1Gb bonded compute node, doesnt get over the [200Mb/s -
> 400Mb/s] but its handling hundreds of thousands requests per minute to
> the vms.
> And once in a while, gives you the sensation that everything goes to
> hell, timeouts from aplications over there, response times from apis
> going from 10ms to 200ms over there, 20ms delays happening between the
> vm ETH0 and the VNET interface, etc.
> So, since its a massive scenario to tune, we never kinda, nailedon
> WHERE TO give this 1, 2 or 3 final buffer/ring/affinity tune to make
> everything work from the compute side.
> I know its a little awkward, but im craving, and jaunting for
> community real life examples regarding "HIGH THROUGHPUT" tuning with
> KVM scenarios, dark linux or if someone can help me go through
> configurations that might sound weird / unnecesary / incorrect.
> For those who are wondering, well ... i dont know what you have, lets
> start with this.
> COMPUTE NODES (99% of them, different vendors, but ...)
> * 128/256 GB of ram
> * 2 hexacores with HT enabled
> * 2x1Gb bonded interfaces (want to know the more than 20 models we are
> using, just ask for it)
> * Multi queue interfaces, pined via irq to different cores
> * ubuntu 12.04 kernel 3.2.0-[40-48]
> * Linux bridges, no VLAN, no open-vswitch
> I want to try to keep the networking appliances ( TOR's, AGGR, CORES )
> as out of the picture as possible.
> im thinking "i hope this thread gets great, in time"
> So, ready to learn as much as i can.
> Thank you openstack community, as allways.
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-operators