[kolla][neutron][ovn] High ovs-dbserver CPU Usage with OVN in Kolla-ansible Large-Scale Deployment
Hi, I have a Wallaby cluster with 5 controller and 130 compute node. Host OS: Rocky 8.7 Deploy tool: Kolla-ansbile kolla_base_distro(container images): centos Network Mode：**OVN** After completing the deployment, the functionality for creating a single instance works as expected, and instances are created successfully. However, the process is extremely slow. When attempting to create instances in bulk, the operation fails on a large scale. During this time, I observed that the ovsdb-server is consuming close to 100% CPU usage. Even without any tasks being processed, the CPU usage remains consistently around 90%, just from the cluster's regular operations. I redeployed the cluster and switched the network mode to Open vSwitch (OVS), and everything works as expected with normal performance. Could anyone offer guidance on what might be causing this issue? I’m aiming to use the OVN mode with Kolla in a large-scale OpenStack cluster (100+ nodes). What configuration or adjustments would I need to make? Thank you in advance for your help! Han Guangyu
I am also interested in large scale OVN deployment, the problem is there is not a single source of information available about OVN scaling. Everyone has their own story in different ways. I am running 300 compute nodes with OVS mode and life is beautiful. But the future is OVN and trying to find light otherside of the tunnel. May I ask why you are using Rocky linux? On Tue, Oct 15, 2024 at 11:49 PM 韩光宇 <hanguangyu2@gmail.com> wrote:
Hi,
I have a Wallaby cluster with 5 controller and 130 compute node.
Host OS: Rocky 8.7 Deploy tool: Kolla-ansbile kolla_base_distro(container images): centos Network Mode：**OVN**
After completing the deployment, the functionality for creating a single instance works as expected, and instances are created successfully. However, the process is extremely slow. When attempting to create instances in bulk, the operation fails on a large scale. During this time, I observed that the ovsdb-server is consuming close to 100% CPU usage. Even without any tasks being processed, the CPU usage remains consistently around 90%, just from the cluster's regular operations.
I redeployed the cluster and switched the network mode to Open vSwitch (OVS), and everything works as expected with normal performance.
Could anyone offer guidance on what might be causing this issue? I’m aiming to use the OVN mode with Kolla in a large-scale OpenStack cluster (100+ nodes). What configuration or adjustments would I need to make?
Thank you in advance for your help!
Han Guangyu
Hi, Yes, I am similarly troubled by "not a single source of information available" in large-scale OVN deployment!
May I ask why you are using Rocky linux? It's a forced choice for me in this deployment, but from a technical standpoint, I believe it can deliver software quality consistent with Red Hat, so I won't reject it.
Satish Patel <satish.txt@gmail.com> 于2024年10月16日周三 11:44写道：
I am also interested in large scale OVN deployment, the problem is there is not a single source of information available about OVN scaling. Everyone has their own story in different ways. I am running 300 compute nodes with OVS mode and life is beautiful. But the future is OVN and trying to find light otherside of the tunnel.
May I ask why you are using Rocky linux?
On Tue, Oct 15, 2024 at 11:49 PM 韩光宇 <hanguangyu2@gmail.com> wrote:
Hi,
I have a Wallaby cluster with 5 controller and 130 compute node.
Host OS: Rocky 8.7 Deploy tool: Kolla-ansbile kolla_base_distro(container images): centos Network Mode：**OVN**
After completing the deployment, the functionality for creating a single instance works as expected, and instances are created successfully. However, the process is extremely slow. When attempting to create instances in bulk, the operation fails on a large scale. During this time, I observed that the ovsdb-server is consuming close to 100% CPU usage. Even without any tasks being processed, the CPU usage remains consistently around 90%, just from the cluster's regular operations.
I redeployed the cluster and switched the network mode to Open vSwitch (OVS), and everything works as expected with normal performance.
Could anyone offer guidance on what might be causing this issue? I’m aiming to use the OVN mode with Kolla in a large-scale OpenStack cluster (100+ nodes). What configuration or adjustments would I need to make?
Thank you in advance for your help!
Han Guangyu
participants (2)
-
Satish Patel
-
韩光宇