[Openstack-operators] Neutron nf_conntrack performance
arahal at iweb.com
Mon Aug 18 14:54:25 UTC 2014
Le 2014-08-16 02:44, Stuart Fox a écrit :
> Hey other ops!
> Im having a serious problem with my neutron router getting spin locked
> in nf_conntrack_tuple_taken.
> Has anybody else experienced it?
> As the incoming request rate goes up, so nf_conntrack_tuple_taken runs
> very hot on CPU0 causing ksoftirqd/0 to run at 100%. At that point
> internal pings on the GRE network go sky high and its game over.
> Ubuntu 14.04/Icehouse 2014.1.1 on an ibm x3550 with 4 10G intel nics.
> eth0 - Mgt
> eth1 - GRE
> eth2 - Public
> eth3 - unused
We ran into similar problems with softirq.
While the problem often comes from load and how it's handled, an initial
workaround we applied was this:
for i in $(find /sys/class/net/eth*/queues -type d -name "rx-*"); do
echo $i; echo fffffff > $i/rps_cpus ; done
This sets all CPUs to handle recieved packets.
(note that this assumes your interfaces are called ethX. this turns out
to no longer be true on ubuntu 14.04)
More information about the OpenStack-operators