Re: [neutron-vpnaas][kolla-ansible][victoria] Growing memory consumption with only one VPN connection
Hi, 1. There is only one, bidirectional vpn connection between project networks in two regions 2. there is no (or almost no) traffic on this vpn link 3. second region is on one host (kolla all-in-one) and situation looks the same- with vpn service enabled there is excessive Memory use (growing also about 5-6MB/ minute and forcing the system to use swapfile) there is no big cpu usage (ofcourse also due to no traffic inside VPN) It's the same case even if there's no vpn connection configured. Best regards Adam Tomas Wysłane z iPhone'a
Wiadomość napisana przez Vinh Nguyen Duc <vinhducnguyen1708@gmail.com> w dniu 29.06.2021, o godz. 11:04:
Hi Adam, Do you think the VPN connection is use CPU and memory to encrypt the VPN traffic?
On Mon, Jun 28, 2021 at 14:30 Adam Tomas <bkslash@poczta.onet.pl <mailto:bkslash@poczta.onet.pl>> wrote: Hi, I have problem with neutron vpnaas - after enabling vpnaas plugin everything was ok at first. I was able to create VPN connection and communication is correct. But after a week of running vpnaas (only one VPN connection created/working!) I’ve noticed, that neutron-vpnaas takes more and more memory. I have 5 processes on each controller (there should be always five? Or it is changing dynamically?):
42435 1545384 0.6 8.6 5802516 5712412 ? S Jun15 110:50 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545389 0.6 8.5 5735832 5645856 ? S Jun15 112:16 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545378 0.5 8.5 5734192 5643620 ? S Jun15 108:09 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545372 0.5 8.5 5731128 5641436 ? S Jun15 109:07 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545369 0.6 8.4 5637084 5547392 ? S Jun15 114:21 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf
now neutron_server takes over 27G of RAM on each controller:
neutron_server running 10.2 27.2G
After stopping all neutron containers and starting it again neutron takes a lot less memory:
neutron_server running 0.5 583M
but memory usage keeps growing (about 5-6 M every minute). What’s wrong? When I disable vpnaas globally there’s no problem with excessive memory usage, so it’s vpnaas for sure…
Best regards Adam Tomas
Hi, On wtorek, 29 czerwca 2021 16:08:38 CEST at wrote:
Hi, 1. There is only one, bidirectional vpn connection between project networks in two regions 2. there is no (or almost no) traffic on this vpn link 3. second region is on one host (kolla all-in-one) and situation looks the same- with vpn service enabled there is excessive Memory use (growing also about 5-6MB/ minute and forcing the system to use swapfile)
there is no big cpu usage (ofcourse also due to no traffic inside VPN) It's the same case even if there's no vpn connection configured. Best regards Adam Tomas Wysłane z iPhone'a
Wiadomość napisana przez Vinh Nguyen Duc <vinhducnguyen1708@gmail.com> w dniu 29.06.2021, o godz. 11:04:
Hi Adam, Do you think the VPN connection is use CPU and memory to encrypt the VPN traffic?
On Mon, Jun 28, 2021 at 14:30 Adam Tomas <bkslash@poczta.onet.pl <mailto:bkslash@poczta.onet.pl>> wrote: Hi, I have problem with neutron vpnaas - after enabling vpnaas plugin everything was ok at first. I was able to create VPN connection and communication is correct. But after a week of running vpnaas (only one VPN connection created/working!) I’ve noticed, that neutron-vpnaas takes more and more memory. I have 5 processes on each controller (there should be always five? Or it is changing dynamically?):
42435 1545384 0.6 8.6 5802516 5712412 ? S Jun15 110:50 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545389 0.6 8.5 5735832 5645856 ? S Jun15 112:16 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545378 0.5 8.5 5734192 5643620 ? S Jun15 108:09 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545372 0.5 8.5 5731128 5641436 ? S Jun15 109:07 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf 42435 1545369 0.6 8.4 5637084 5547392 ? S Jun15 114:21 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf
now neutron_server takes over 27G of RAM on each controller:
neutron_server running 10.2 27.2G
After stopping all neutron containers and starting it again neutron takes a lot less memory:
neutron_server running 0.5 583M
but memory usage keeps growing (about 5-6 M every minute). What’s wrong? When I disable vpnaas globally there’s no problem with excessive memory usage, so it’s vpnaas for sure…
Best regards Adam Tomas
Please open Launchpad bug for that https://bugs.launchpad.net/neutron/+filebug - it will be easier to track it there. Please include what versions are You using exactly and how to reproduce the issue. -- Slawek Kaplonski Principal Software Engineer Red Hat
Hi Sławek, I’ll do that soon, still testing some options nad observing results. Best regards Adam Tomas
Please open Launchpad bug for that https://bugs.launchpad.net/neutron/+filebug - it will be easier to track it there. Please include what versions are You using exactly and how to reproduce the issue.
-- Slawek Kaplonski Principal Software Engineer Red Hat
participants (3)
-
at
-
bkslash
-
Slawek Kaplonski