[openstack-dev] [neutron][kilo] - vxlan's max bandwidth

Akihiro Motoki amotoki at gmail.com
Wed Apr 20 09:47:42 UTC 2016

2016-04-20 14:53 GMT+09:00 Ihar Hrachyshka <ihrachys at redhat.com>:
> Ian Wells <ijw.ubuntu at cack.org.uk> wrote:
>> Right. Note that custom MTU works out of the box only starting from
>> Mitaka.
>> It's been in from at least Kilo (give or take a some bugfixes, it seems,
>> all of which deserve backporting).
> It never worked as you would expect, though indeed a lot of code to
> calculate MTU was in place.
>> - It won’t work in OVS hybrid setup, where intermediate devices (qbr) will
>> still have mtu = 1500, that will result in Jumbo frames dropped. We have
>> backports to fix it in Liberty at: https://review.openstack.org/305782 and
>> https://review.openstack.org/#/c/285710/
>> Indeed, you can actively request the MTU per virtual network as you create
>> them, subject to segment_mtu and path_mtu indicating they're achievable.
> No. MTU is not available for setting during network creation. It’s only
> calculated as per get_mtu() [relying on path_mtu and physical_network_mtus
> and segment_mtu; note the latter is renamed since Mitaka into
> global_physnet_mtu].
>> In this instance, configure your switches with a 9000 MTU and set
>> segment_mtu = path_mtu = 9000.  The virtual network MTU will then default to
>> 8950 for a VXLAN network (the biggest possible packet inside VXLAN in that
>> circumstance) and you can choose to set it to anything else below that
>> number as you net-create.  The MTU should be correctly advertised by DHCP
>> when set.
> As I said, no, you can’t set it on network creation.
> Also, having the network using 8950 is not enough for Jumbo frames, because
> till Mitaka (and the next minor Liberty release) Nova was not using that
> value to bump MTU for intermediate devices participating in the hybrid
> bridge setup.

In my understanding, nova honors network_device_mtu configurtaion and
when we configure network_device_mtu to 8950, veth devices are
configured to mtu 8950
and the MTU of qbr bridges will be set to 8950. It seems a linux
bridge adjusts its MTU
to the largest MTU among connected interfaces.

I got the following the following results.

$ ip -o l | grep 9cdd22c-ec
11: qbra9cdd22c-ec: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc
noqueue state UP mode DEFAULT group default \    link/ether
5e:7f:e2:aa:91:71 brd ff:ff:ff:ff:ff:ff
12: qvoa9cdd22c-ec: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 8950
qdisc pfifo_fast master ovs-system state UP mode DEFAULT group default
qlen 1000\    link/ether 56:7a:79:31:cc:aa brd ff:ff:ff:ff:ff:ff
13: qvba9cdd22c-ec: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 8950
qdisc pfifo_fast master qbra9cdd22c-ec state UP mode DEFAULT group
default qlen 1000\    link/ether 5e:7f:e2:aa:91:71 brd
14: tapa9cdd22c-ec: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc
pfifo_fast master qbra9cdd22c-ec state UNKNOWN mode DEFAULT group
default qlen 500\    link/ether fe:16:3e:b1:a0:71 brd

If we don't need to per-network MTU, we can configure nova to use larger MTU.
Am I missing the context?

>> I hope you don't find you have to do what Akihiro suggests.  That was good
>> advice about three releases back but nowadays it actually breaks the code
>> that's there to deal with MTUs properly.
> Yes, indeed it’s not needed since Kilo to modify dnsmasq configuration to
> set the option. advertise_mtu is now True since Mitaka, and for earlier
> releases, you just set it to True explicitly.

Sorry for confusing you all and thanks for correcting me.
It seems I was confused with one older version.
As others pointed out, we have advertise_mtu in Kilo.


> Ihar
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list