I'm curious. A lot of network cards support offloaded vxlan traffic these days so the processor isn't doing much work. Is this issue really a problem?

On Tue, 2019-07-09 at 00:17 +0000, Fox, Kevin M wrote: the issue is not really with the cpu overhead of tunnel decapsulation it is with the cross numa latency incurred when crossing the qpi bus so even with hardware accleration on the nic there is a perfomace degradation if you go across a numa nodes. the optimal solution is to have multiple nics, one per numa node and bond them then affinities the vms mac to the numa local bond peer such that no traffic has to travers the numa node but that is non trivaial to do and is not supported by openstack natively. you would have to have an agent of or cron job that actuly a did the tuning after a vm is spawned but it could be an interesting experiment if someone wanted to code it up.

Thanks, Kevin ________________________________ From: Guo, Ruijing [ruijing.guo@intel.com] Sent: Sunday, July 07, 2019 5:47 PM To: openstack-dev@lists.openstack.org; openstack@lists.openstack.org Subject: [Neutron] NUMA aware VxLAN

Hi,

Existing neutron ML2 support one VxLAN for tenant network. In NUMA case, VM 0 can be created in node 0 and VM 1 can be created in node 1 and VxLAN is in node 0.

VM1 need to cross node, which cause some performance downgrade. Does someone have this performance issue? Does Neutron community have plan to enhance it?

nova has a spec called numa aware vswitchs https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/num... that allow you to declare teh numa affinity of tunnels and physnets on a host. this will not allow you to have multiple tunnel enpoint ips but it will allow you force instnace with a numa toploty to be colocated on the same numa node as the network backend.

Thanks, -Ruijing