Open Stack

Same memory problems with Ubuntu 16.04 and Mitaka packages included by
default. 

Looks like each new version uses more and more memory. I began using 4GB
in Juno but now I use 8GB in Mitaka. I need HA so I need three servers
-> 24GB . 

It looks like each component spends low memory but, there are a lot of
components! 

The memory usage increases with time. Heat module and horizon are my
biggests problems. 

This is my top after rebooting yesterday at 17:00 : 

+++ 

top - 07:57:59 up 13:43, 1 user, load average: 1.12, 0.96, 0.91
Tasks: 275 total, 4 running, 271 sleeping, 0 stopped, 0 zombie
%Cpu(s): 19.7 us, 6.4 sy, 0.0 ni, 70.8 id, 0.9 wa, 0.0 hi, 1.3 si, 0.9
st
KiB Mem : 8175464 total, 2376492 free, 4866088 used, 932884 buff/cache
KiB Swap: 248828 total, 248828 free, 0 used. 2890088 avail Mem 

 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 
 3246 mysql 20 0 699632 235384 15040 S 0.9 2.9 8:05.70 mysqld 
 3260 rabbitmq 20 0 1309020 193068 5140 S 0.0 2.4 6:46.55 beam.smp 
 7648 nova 20 0 381248 166412 8344 S 0.0 2.0 0:15.51 nova-api 
 7650 nova 20 0 380440 165652 8344 S 0.0 2.0 0:15.18 nova-api 
 7554 nova 20 0 374644 160024 8344 S 0.0 2.0 0:03.55 nova-api 
 7555 nova 20 0 374388 159824 8400 S 1.7 2.0 0:22.22 nova-api 
 6915 nova 20 0 361488 157552 15128 S 0.9 1.9 7:41.08 nova-api 
 6865 glance 20 0 613692 142644 19988 S 0.9 1.7 7:22.74 glance-api 
 6916 neutron 20 0 341992 142012 15220 S 1.7 1.7 27:45.77 neutron-server

 6791 nova 20 0 298848 139456 12236 S 0.0 1.7 1:34.21 nova-scheduler 
 6912 nova 20 0 310588 139296 12596 S 0.0 1.7 1:18.89 nova-cert 
 6938 nova 20 0 296560 137484 12392 S 0.0 1.7 1:18.61 nova-consoleaut 
 7533 neutron 20 0 346136 136416 5600 S 0.0 1.7 9:34.36 neutron-server 
 7534 neutron 20 0 346260 136340 5452 S 0.9 1.7 16:26.55 neutron-server 
 7532 neutron 20 0 343696 134192 5600 S 0.0 1.6 0:11.06 neutron-server 
 7428 nova 20 0 287608 123452 7436 R 3.4 1.5 34:39.59 nova-conductor 
 7427 nova 20 0 287200 123028 7436 S 2.6 1.5 34:33.64 nova-conductor 
 7535 neutron 20 0 329496 119204 5324 S 1.7 1.5 15:45.72 neutron-server 
 2647 heat 20 0 360108 118964 18724 S 1.7 1.5 7:52.39 heat-engine 
 6944 nova 20 0 268900 115664 12196 S 0.9 1.4 7:16.63 nova-conductor 
 7246 glance 20 0 602392 114728 7532 S 0.0 1.4 0:00.66 glance-api 
 7245 glance 20 0 602392 114656 7532 S 0.0 1.4 0:00.68 glance-api 
 7315 heat 20 0 367120 111952 8976 S 0.0 1.4 0:19.07 heat-engine 
 7316 heat 20 0 367120 111948 8976 S 0.0 1.4 0:19.32 heat-engine 
 7318 heat 20 0 367120 111944 8976 S 0.0 1.4 0:19.41 heat-engine 
 7314 heat 20 0 367120 111932 8976 S 0.0 1.4 0:21.93 heat-engine 
 7010 nova 20 0 293424 110600 14784 S 0.0 1.4 0:23.89 nova-novncproxy 
 2639 heat 20 0 351792 110524 18732 S 0.0 1.4 0:01.91 heat-api 
 2642 heat 20 0 351276 109872 18560 S 0.0 1.3 0:02.26 heat-api-cfn 
 6871 glance 20 0 219776 106296 11764 S 0.0 1.3 0:01.84 glance-registry 
 7155 glance 20 0 227556 104544 6492 S 0.0 1.3 0:00.80 glance-registry 
 7154 glance 20 0 227044 104188 6492 S 0.0 1.3 0:00.74 glance-registry 
 7790 neutron 20 0 249672 97980 12176 R 5.2 1.2 27:52.52 neutron-openvsw

 6918 neutron 20 0 248556 96980 12508 S 1.7 1.2 15:48.44 neutron-l3-agen

 6946 neutron 20 0 248192 96276 12168 S 1.7 1.2 15:36.30 neutron-dhcp-ag

 429 keystone 20 0 415944 95992 10152 S 0.0 1.2 0:03.23 apache2 
 426 keystone 20 0 415944 95980 10152 S 0.0 1.2 0:03.52 apache2 
 7183 heat 20 0 351792 95880 4084 S 0.0 1.2 0:00.32 heat-api 
 7184 heat 20 0 351792 95876 4084 S 0.0 1.2 0:00.29 heat-api 
 427 keystone 20 0 415688 95768 10152 S 0.0 1.2 0:02.94 apache2 
 425 keystone 20 0 415688 95764 10152 S 0.0 1.2 0:03.00 apache2 
 6913 neutron 20 0 239924 92332 12176 S 2.6 1.1 18:50.89 neutron-lbaas-a

 6914 neutron 20 0 243752 91964 12164 S 1.7 1.1 15:10.37 neutron-metadat

 7172 neutron 20 0 244264 87276 7092 S 1.7 1.1 14:23.84 neutron-metadat 
20015 neutron 20 0 186940 46012 4252 S 0.0 0.6 0:01.06 neutron-ns-meta 
18217 neutron 20 0 186524 45832 4304 S 0.0 0.6 0:00.82 neutron-ns-meta 
16980 neutron 20 0 186412 45784 4380 S 0.0 0.6 0:00.51 neutron-ns-meta 
20083 neutron 20 0 186548 45756 4284 S 0.0 0.6 0:00.59 neutron-ns-meta 

+++ 

When controller is getting out of memory everything goes extremely slow.
Restarting apache2 and heat solves the problem. ;-) 

Be lucky next week. :-) 

On 2016-08-18 20:53, Satish Patel wrote: 

> How many folks using mitaka in production? I have search on google and
> i didn't find people using mitaka because its very new and people are
> not use to with it
> 
> On Thu, Aug 18, 2016 at 2:45 PM, Satish Patel <satish.txt at gmail.com> wrote: we are running out of time.. this installation will go to production
> in next week.. I need to deliver something soon.
> 
> I am installing liberty now. and see how it works..!
> 
> On Thu, Aug 18, 2016 at 1:41 PM, Nasir Mahmood <nasir.mahmood at gmail.com> wrote: Did you try to locate whats causing to use all your memory. Try using htop,
> truss/strace and other debugging tools. You might also be a victim of
> resource bottlenecks, like your HDD are serving slow or it may be that your
> neutron conf is inappropriate at some point. If this ain't a production
> environment, try troubleshooting rather than going for old installation.
> 
> On Aug 18, 2016 21:56, "Satish Patel" <satish.txt at gmail.com> wrote:
> 
> I have recently install mitaka openstack using RDO on HP DL360 G9
> server with 16GB memory. but after installation it took all of my
> memory and suck all my swap memory too. and it was extremely slow. I
> have found neutron-server and nova-conductor and other component was
> eating most memory and growing. is this normal?
> 
> I didn't find related solution on google so now i have no option
> except going back to older liberty version.
> 
> Any suggestion folks?
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
> Post to : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]

 _______________________________________________
 Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
 Post to : openstack at lists.openstack.org
 Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]

 

Links:
------
[1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160819/33ae38d1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MitakaMemoryUsage.jpg
Type: image/jpeg
Size: 77546 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160819/33ae38d1/attachment.jpg>