[openstack-dev] [neutron][kilo] - vxlan's max bandwidth

Sławek Kapłoński slawek at kaplonski.pl
Sat Apr 16 11:40:36 UTC 2016


Hello,

What MTU have You got configured on VMs? I had issue with performance on
vxlan network with standard MTU (1500) but when I configured Jumbo
frames on vms and on hosts then it was much better.

-- 
Best regards / Pozdrawiam
Sławek Kapłoński
slawek at kaplonski.pl

On Fri, 15 Apr 2016, Rick Jones wrote:

> On 04/14/2016 07:10 PM, Kenny Ji-work wrote:
> >Hi all,
> >
> >In the environment of openstack kilo, I test the bandwidth in the scene
> >which VxLan being used. The result show that the vxlan can only support
> >up to 1 gbits bandwidth. Is this a bug or any else issue, or is there
> >some hotfix to solve the issue? Thank you for answering!
> 
> I'm glossing over some details, but broadly speaking, a single network flow
> cannot take advantage of more than one CPU in a system.  And while network
> speeds have been continuing to increase, per-core speeds haven't really gone
> up much over the last five to ten years.
> 
> So, to get "speed/link rate" networking stacks have become dependent on
> stateless offloads - Checksum Offload (CKO) TCP Segmentation Offload
> (TSO/GSO) and Generic Receive Offload 9GRO).  And until somewhat recently,
> NICs did not offer stateless offloads for VxLAN-encapsulated traffic.  So,
> one effectively has a "dumb" NIC without stateless offloads.  And depending
> on what sort of processor you have, that limit could be down around 1
> Gbit/s.  Only some of the more recent 10GbE NICs offer stateless offload of
> VxLAN-encapsulated traffic, and similarly their more recent drivers and
> networking stacks.
> 
> In olden days, before the advent of stateless offloads there was a rule of
> thumb - 1 Mbit/s per MHz.  That was with "pure" bare-iron networking - no
> VMs, no encapsulation.  Even then it was a bit hand-wavy, and may have
> originated in the land of SPARC processors.  But hopefully it conveys the
> idea of what it means to lose the stateless offloads.
> 
> So, it would be good to know what sort of CPUs are involved (down to the
> model names and frequencies) as well as the NICs involved - again, full
> naming, not just the brand name.
> 
> And it is just a paranoid question, but is there any 1 Gbit/s networking in
> your setup at all?
> 
> happy benchmarking,
> 
> rick jones
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160416/ae56b9fc/attachment.pgp>


More information about the OpenStack-dev mailing list