Open Stack

On 04/14/2016 07:10 PM, Kenny Ji-work wrote:
> Hi all,
>
> In the environment of openstack kilo, I test the bandwidth in the scene
> which VxLan being used. The result show that the vxlan can only support
> up to 1 gbits bandwidth. Is this a bug or any else issue, or is there
> some hotfix to solve the issue? Thank you for answering!

I'm glossing over some details, but broadly speaking, a single network 
flow cannot take advantage of more than one CPU in a system.  And while 
network speeds have been continuing to increase, per-core speeds haven't 
really gone up much over the last five to ten years.

So, to get "speed/link rate" networking stacks have become dependent on 
stateless offloads - Checksum Offload (CKO) TCP Segmentation Offload 
(TSO/GSO) and Generic Receive Offload 9GRO).  And until somewhat 
recently, NICs did not offer stateless offloads for VxLAN-encapsulated 
traffic.  So, one effectively has a "dumb" NIC without stateless 
offloads.  And depending on what sort of processor you have, that limit 
could be down around 1 Gbit/s.  Only some of the more recent 10GbE NICs 
offer stateless offload of VxLAN-encapsulated traffic, and similarly 
their more recent drivers and networking stacks.

In olden days, before the advent of stateless offloads there was a rule 
of thumb - 1 Mbit/s per MHz.  That was with "pure" bare-iron networking 
- no VMs, no encapsulation.  Even then it was a bit hand-wavy, and may 
have originated in the land of SPARC processors.  But hopefully it 
conveys the idea of what it means to lose the stateless offloads.

So, it would be good to know what sort of CPUs are involved (down to the 
model names and frequencies) as well as the NICs involved - again, full 
naming, not just the brand name.

And it is just a paranoid question, but is there any 1 Gbit/s networking 
in your setup at all?

happy benchmarking,

rick jones