[neutron][ovn] both vxlan and geneve cannot get full bandwidth
Hello. I am going with ovn for my system but when I use vxlan and geneve, my instance cannot get full bandwidth(12 gbps)-instance to instance with private network. It is ok if I use vlan or floating ip for instances. My env: Openstack 2025.1 Kolla Ansible master-2025.1 Ubuntu 22.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4 Ps give me some points. Thank Nguyen Huu Khoi
Hello. Could I have some ideas on it please? Nguyen Huu Khoi On Thu, Apr 17, 2025 at 9:53 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello.
I am going with ovn for my system but when I use vxlan and geneve, my instance cannot get full bandwidth(12 gbps)-instance to instance with private network. It is ok if I use vlan or floating ip for instances.
My env:
Openstack 2025.1 Kolla Ansible master-2025.1 Ubuntu 22.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4
Ps give me some points.
Thank
Nguyen Huu Khoi
What is your testing methodology? Is it iPerf? Trex? Something you can share so we might recreate and compare on our side? -- James Denton Principal Architect Rackspace Private Cloud - OpenStack james.denton@rackspace.com From: Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> Date: Monday, April 28, 2025 at 7:28 AM To: OpenStack Discuss <openstack-discuss@lists.openstack.org> Subject: Re: [neutron][ovn] both vxlan and geneve cannot get full bandwidth CAUTION: This message originated externally, please use caution when clicking on links or opening attachments! Hello. Could I have some ideas on it please? Nguyen Huu Khoi On Thu, Apr 17, 2025 at 9:53 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com<mailto:nguyenhuukhoinw@gmail.com>> wrote: Hello. I am going with ovn for my system but when I use vxlan and geneve, my instance cannot get full bandwidth(12 gbps)-instance to instance with private network. It is ok if I use vlan or floating ip for instances. My env: Openstack 2025.1 Kolla Ansible master-2025.1 Ubuntu 22.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4 Ps give me some points. Thank Nguyen Huu Khoi
Hi. I used iper3 for testing. It happened only with geneve and vxlan. I can get full bandwidth with vlan. My env: Openstack 2025.1 Kolla Ansible master-2025.1 Ubuntu 24.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4 Noticed: I used distributed floating IP for my system. I tried without distributed floating IP and got the same result. Nguyen Huu Khoi On Mon, Apr 28, 2025 at 11:15 PM James Denton <james.denton@rackspace.com> wrote:
What is your testing methodology? Is it iPerf? Trex? Something you can share so we might recreate and compare on our side?
--
James Denton
Principal Architect
Rackspace Private Cloud - OpenStack
james.denton@rackspace.com
*From: *Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> *Date: *Monday, April 28, 2025 at 7:28 AM *To: *OpenStack Discuss <openstack-discuss@lists.openstack.org> *Subject: *Re: [neutron][ovn] both vxlan and geneve cannot get full bandwidth
*CAUTION:* This message originated externally, please use caution when clicking on links or opening attachments!
Hello.
Could I have some ideas on it please?
Nguyen Huu Khoi
On Thu, Apr 17, 2025 at 9:53 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello.
I am going with ovn for my system but when I use vxlan and geneve, my instance cannot get full bandwidth(12 gbps)-instance to instance with private network. It is ok if I use vlan or floating ip for instances.
My env:
Openstack 2025.1
Kolla Ansible master-2025.1
Ubuntu 22.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4
Ps give me some points.
Thank
Nguyen Huu Khoi
On 2025-04-28 23:22:29 +0700 (+0700), Nguyễn Hữu Khôi wrote: [...]
It happened only with geneve and vxlan. I can get full bandwidth with vlan. [...]
If you're seeing significant bandwidth reduction with encapsulated virtual networks, you may be fragmenting and doubling the number of frames needed. Make sure the MTU of the physical network is sufficiently larger than the encapsulated MTU to account for the encapsulation overhead. -- Jeremy Stanley
Hello. My physical network has 9000 MTU and when doing tests I use 1500 MTU for both vlan and geneve. Nguyen Huu Khoi On Tue, Apr 29, 2025 at 4:05 AM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2025-04-28 23:22:29 +0700 (+0700), Nguyễn Hữu Khôi wrote: [...]
It happened only with geneve and vxlan. I can get full bandwidth with vlan. [...]
If you're seeing significant bandwidth reduction with encapsulated virtual networks, you may be fragmenting and doubling the number of frames needed. Make sure the MTU of the physical network is sufficiently larger than the encapsulated MTU to account for the encapsulation overhead. -- Jeremy Stanley
When you use vxlan or geneve, by default, the CPU will handle the encapsulation/desencapsulation of each packets. That is why you may need to use jumbo frames, set your NIC to handle the encapsulation/desencapsulation, and set irqbalance on all your cores. Furthermore, you should have a look on network offloading options (with your favorite browser). Le jeu. 17 avr. 2025 à 16:54, Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> a écrit :
Hello.
I am going with ovn for my system but when I use vxlan and geneve, my instance cannot get full bandwidth(12 gbps)-instance to instance with private network. It is ok if I use vlan or floating ip for instances.
My env:
Openstack 2025.1 Kolla Ansible master-2025.1 Ubuntu 22.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4
Ps give me some points.
Thank
Nguyen Huu Khoi
Hello, After increasing the geneve mtu to 8942 then my network got neary full bandwidth, I will do some tests and go back. Nguyen Huu Khoi On Wed, Apr 30, 2025 at 7:39 PM Vincent Godin <vince.mlist@gmail.com> wrote:
When you use vxlan or geneve, by default, the CPU will handle the encapsulation/desencapsulation of each packets. That is why you may need to use jumbo frames, set your NIC to handle the encapsulation/desencapsulation, and set irqbalance on all your cores. Furthermore, you should have a look on network offloading options (with your favorite browser).
Le jeu. 17 avr. 2025 à 16:54, Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> a écrit :
Hello.
I am going with ovn for my system but when I use vxlan and geneve, my instance cannot get full bandwidth(12 gbps)-instance to instance with private network. It is ok if I use vlan or floating ip for instances.
My env:
Openstack 2025.1 Kolla Ansible master-2025.1 Ubuntu 22.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4
Ps give me some points.
Thank
Nguyen Huu Khoi
Hello. I check and see that: tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: on tx-tcp-mangleid-segmentation: on tx-tcp6-segmentation: on Do they handle encapsulation/desencapsulation or I need do something? Nguyen Huu Khoi On Thu, May 1, 2025 at 7:36 AM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello,
After increasing the geneve mtu to 8942 then my network got neary full bandwidth, I will do some tests and go back.
Nguyen Huu Khoi
On Wed, Apr 30, 2025 at 7:39 PM Vincent Godin <vince.mlist@gmail.com> wrote:
When you use vxlan or geneve, by default, the CPU will handle the encapsulation/desencapsulation of each packets. That is why you may need to use jumbo frames, set your NIC to handle the encapsulation/desencapsulation, and set irqbalance on all your cores. Furthermore, you should have a look on network offloading options (with your favorite browser).
Le jeu. 17 avr. 2025 à 16:54, Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> a écrit :
Hello.
I am going with ovn for my system but when I use vxlan and geneve, my instance cannot get full bandwidth(12 gbps)-instance to instance with private network. It is ok if I use vlan or floating ip for instances.
My env:
Openstack 2025.1 Kolla Ansible master-2025.1 Ubuntu 22.04 with uplink 25gbps x 2 for bonding, lacp 802.3ad, layer3+4
Ps give me some points.
Thank
Nguyen Huu Khoi
participants (4)
-
James Denton
-
Jeremy Stanley
-
Nguyễn Hữu Khôi
-
Vincent Godin