Network policer behavior
v at prokofev.me
Fri Feb 19 11:52:11 UTC 2021
I've encountered some really bizarre things today, and want to discuss it
for a sanity check.
This is probably more appropriate for a libvirt mailing list, but I want
some second opinion first.
libvirtd (libvirt) 4.0.0
qemu-x86_64 version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.15)
2 instances were launched a while ago with Zabbix Appliance 5.0.7 LTS
Both of them were "hacked" shortly after and became a part of a botnet that
participated in DDOS attacks via SYN-flood.
Sad, but nothing out of the ordinary so far.
Now to the bizarre part.
Both of the instances had QoS configured via instance metadata that limited
them for 100Mb/s in/out. I've checked it on the compute side - it was
correctly applied there too. This metadata was tested years ago with
usual iperf tcp/udp tests with 1-10 flows - it worked perfectly.
Both of the instances landed on the same compute node.
And both of them are ignoring network policer and sending about 400Mb/s of
SYN-flood traffic each on their respective tap interface, so about 800Mb/s
were flowing out of the compute node switch port.
So I've shut down 1 instance - 2nd one traffic rose to about 600Mb/s - ok,
they probably were contesting some resources.
Now I apply a qos-policy to the port of the remaining instance - that
does the trick, I can see on the switch port that compute node traffic went
down to the expected level, but CPU context switches on the compute node
increased almost 3 times, and traffic on the tap interface rise to 1.6Gb/s!
What I can't understand is why libvirt network policer does not handle this
case? Why does implementing qos-policy actually increase traffic on the tap
I can't exactly say if it's the tc that is responsible for context switches
increase, or traffic generated by the instance.
At first I thought that my Zabbix went crazy, so I double checked. It seems
it takes its data for net.if.in key from /proc/net/dev and it appears to be
Any ideas appreciated.
compute2:~$ virsh domiftune instance-0000141d tap834d76e9-5f
inbound.peak : 12500
inbound.burst : 3125
inbound.floor : 0
outbound.peak : 12500
outbound.burst : 3125
 openstack port set --qos-policy 100Mbps
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the openstack-discuss