[Openstack-operators] Neutron nf_conntrack performance

Ahmed RAHAL arahal at iweb.com
Mon Aug 18 14:54:25 UTC 2014


Le 2014-08-16 02:44, Stuart Fox a écrit :
> Hey other ops!
> Im having a serious problem with my neutron router getting spin locked
> in nf_conntrack_tuple_taken.
> Has anybody else experienced it?
> As the incoming request rate goes up, so nf_conntrack_tuple_taken runs
> very hot on CPU0 causing ksoftirqd/0 to run at 100%. At that point
> internal pings on the GRE network go sky high and its game over.
> Ubuntu 14.04/Icehouse 2014.1.1 on an ibm x3550 with 4 10G intel nics.
> eth0 - Mgt
> eth1 - GRE
> eth2 - Public
> eth3 - unused

We ran into similar problems with softirq.
While the problem often comes from load and how it's handled, an initial 
workaround we applied was this:

for i in $(find /sys/class/net/eth*/queues -type d -name "rx-*"); do 
echo $i; echo fffffff > $i/rps_cpus ; done

This sets all CPUs to handle recieved packets.

(note that this assumes your interfaces are called ethX. this turns out 
to no longer be true on ubuntu 14.04)


Ahmed Rahal

More information about the OpenStack-operators mailing list