Open Stack

Hi,

Le 2014-08-16 02:44, Stuart Fox a écrit :
> Hey other ops!
>
> Im having a serious problem with my neutron router getting spin locked
> in nf_conntrack_tuple_taken.
> Has anybody else experienced it?
> As the incoming request rate goes up, so nf_conntrack_tuple_taken runs
> very hot on CPU0 causing ksoftirqd/0 to run at 100%. At that point
> internal pings on the GRE network go sky high and its game over.
>
> Ubuntu 14.04/Icehouse 2014.1.1 on an ibm x3550 with 4 10G intel nics.
> eth0 - Mgt
> eth1 - GRE
> eth2 - Public
> eth3 - unused

We ran into similar problems with softirq.
While the problem often comes from load and how it's handled, an initial 
workaround we applied was this:

for i in $(find /sys/class/net/eth*/queues -type d -name "rx-*"); do 
echo $i; echo fffffff > $i/rps_cpus ; done

This sets all CPUs to handle recieved packets.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/network-rps.html

(note that this assumes your interfaces are called ethX. this turns out 
to no longer be true on ubuntu 14.04)

HTH.

-- 
Ahmed Rahal