Open Stack

Hi all,

I try to set up a plateforme with plugin ML2 and mechanism driver
openvswitch (VXLAN) and l2-pop with a Jumbo frame physical network.
I've got a Cisco Nexus fabric of 10Gb/s and the jumbo frame size is 9216 octets.

I set the MTU on the physical interface used for VXLAN tunneling to 9216 octets.

The vif driver is the OVS hybrid one to be able to do security groups.
My test case is:

- 2 physical compute nodes: CN#1 and CN#2
- One virtual network: NET#1 with subnet 10.0.0.0/24
- 2 VMs plug onto the virtual network NET#1:
    - VM#1 (10.0.0.3) on CN#1
    - VM#2 (10.0.0.4) on CN#2

After the VM creation, I set manually the MTU on veth interfaces (qvo
and qvb) to 9166 (9216 minus 50 (the VXLAN overhead)) on both compute
nodes.
I don't change anything on the interface offloading configuration [1]
and in the guest OS VM the MTU is 1500.

1/ ICMP test OK
When I do a ping with the largest packet size ('ping -s 9138
10.0.0.4', ping VM#2 from VM#1(9138 + 8 ICMP + 20 IP = 9166)), I saw
ICMP packet going into the VXLAN tunnel with the maximum MTU size
(9234: 9216 + 14 ETH + 4 802.1q).
The packet is fragmented by the VM (packets of 1514 (1500 + 14 ETH)
captured from the tap interface) but the qbr defragment them (packets
of 9180 (9166 + 14 ETH) captured from qbr, qvb or qvo interfaces) and
then packet are encapsulated into VXLAN tunnel on the wire (packets of
9234 ( 9180 + 50 VXLAN (14 ETH + 20 IP + 8 UDP + 8 VXLAN) + 4 802.1q
captured on the physical interface of the VXLAN tunnel)

VM#1 tap <-- ~7 x 1514 --> qbr <-- 9180 --> qvb <-- 9180 --> qvo <--
OVS flows + tunneling --> ethx <-- 9234 --> wire

2/ TCP test KO
With the same configuration as above, if I make a TCP perf test
(iperf), the packets size still have a classic MTU value (1500 + VXLAN
overhead) on the wire.

VM#1 tap <-- ~44000 --> qbr <-- ~44000 --> qvb <-- 1514 --> qvo <--
OVS flows + tunneling --> ethx <-- 1568 --> wire

3/ UDP test KO

VM#1 tap <-- 1512 ??? --> qbr <-- ~1512 --> qvb <-- 1512 --> qvo <--
OVS flows + tunneling --> ethx <-- 1566 --> wire


It seems like an offloading problem. I use intel NIC card with driver ixgbe [2].
Any thought ?

[1] http://paste.openstack.org/show/54347/
[2] http://paste.openstack.org/show/54357/

Regards,
Édouard.