[Openstack] openstack mitaka high memory issue

Jose Manuel Ferrer Mosteiro jmferrer.paradigmatecnologico at gmail.com
Mon Aug 22 06:43:57 UTC 2016


 

In my case I choose Mitaka because Ubuntu 16.04 LTS has 5 years of free
support without updates breaking anything. I can program updates and
reboots, go home, receive zabbix messages at 18:00 saying the services
had been rebooted and are OK, ... and sleep well. 

I tried the automatic "yum update" with CentOS but they changed some
version of some neutron related package and some config file var.
Networking began to fail while I was at home. :-( 

In my case the version is not important because people does not use it
as a real cloud. They use it to deploy virtual machines, like a remote
Virtualbox with balancing as a service to deploy OpenShift and
Kubernetes. 

Maybe I am the only person that uses heat to deploy. :-( 

The question is, your system will be used as a virtual machines deployer
like a classic vmware deployment or as an IAAS like AWS or GCE? If it
will be used like a tipical vmware deployment in my opinion any
OpenStack supported version could be the right choice. 

On 2016-08-19 21:30, Satish Patel wrote: 

> I moved from mikata to Liberty and now i can see stability its using
> 4G on controller now. Happy!..
> 
> What big stuff i am going to miss if i run my production on liberty?
> 
> On Fri, Aug 19, 2016 at 2:43 AM, Jose Manuel Ferrer Mosteiro
> <jmferrer.paradigmatecnologico at gmail.com> wrote: 
> 
>> Same memory problems with Ubuntu 16.04 and Mitaka packages included by
>> default.
>> 
>> Looks like each new version uses more and more memory. I began using 4GB in
>> Juno but now I use 8GB in Mitaka. I need HA so I need three servers -> 24GB
>> .
>> 
>> It looks like each component spends low memory but, there are a lot of
>> components!
>> 
>> The memory usage increases with time. Heat module and horizon are my
>> biggests problems.
>> 
>> This is my top after rebooting yesterday at 17:00 :
>> 
>> +++
>> 
>> top - 07:57:59 up 13:43, 1 user, load average: 1.12, 0.96, 0.91
>> Tasks: 275 total, 4 running, 271 sleeping, 0 stopped, 0 zombie
>> %Cpu(s): 19.7 us, 6.4 sy, 0.0 ni, 70.8 id, 0.9 wa, 0.0 hi, 1.3 si, 0.9
>> st
>> KiB Mem : 8175464 total, 2376492 free, 4866088 used, 932884 buff/cache
>> KiB Swap: 248828 total, 248828 free, 0 used. 2890088 avail Mem
>> 
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 3246 mysql 20 0 699632 235384 15040 S 0.9 2.9 8:05.70 mysqld
>> 3260 rabbitmq 20 0 1309020 193068 5140 S 0.0 2.4 6:46.55 beam.smp
>> 7648 nova 20 0 381248 166412 8344 S 0.0 2.0 0:15.51 nova-api
>> 7650 nova 20 0 380440 165652 8344 S 0.0 2.0 0:15.18 nova-api
>> 7554 nova 20 0 374644 160024 8344 S 0.0 2.0 0:03.55 nova-api
>> 7555 nova 20 0 374388 159824 8400 S 1.7 2.0 0:22.22 nova-api
>> 6915 nova 20 0 361488 157552 15128 S 0.9 1.9 7:41.08 nova-api
>> 6865 glance 20 0 613692 142644 19988 S 0.9 1.7 7:22.74
>> glance-api
>> 6916 neutron 20 0 341992 142012 15220 S 1.7 1.7 27:45.77
>> neutron-server
>> 6791 nova 20 0 298848 139456 12236 S 0.0 1.7 1:34.21
>> nova-scheduler
>> 6912 nova 20 0 310588 139296 12596 S 0.0 1.7 1:18.89
>> nova-cert
>> 6938 nova 20 0 296560 137484 12392 S 0.0 1.7 1:18.61
>> nova-consoleaut
>> 7533 neutron 20 0 346136 136416 5600 S 0.0 1.7 9:34.36
>> neutron-server
>> 7534 neutron 20 0 346260 136340 5452 S 0.9 1.7 16:26.55
>> neutron-server
>> 7532 neutron 20 0 343696 134192 5600 S 0.0 1.6 0:11.06
>> neutron-server
>> 7428 nova 20 0 287608 123452 7436 R 3.4 1.5 34:39.59
>> nova-conductor
>> 7427 nova 20 0 287200 123028 7436 S 2.6 1.5 34:33.64
>> nova-conductor
>> 7535 neutron 20 0 329496 119204 5324 S 1.7 1.5 15:45.72
>> neutron-server
>> 2647 heat 20 0 360108 118964 18724 S 1.7 1.5 7:52.39
>> heat-engine
>> 6944 nova 20 0 268900 115664 12196 S 0.9 1.4 7:16.63
>> nova-conductor
>> 7246 glance 20 0 602392 114728 7532 S 0.0 1.4 0:00.66
>> glance-api
>> 7245 glance 20 0 602392 114656 7532 S 0.0 1.4 0:00.68
>> glance-api
>> 7315 heat 20 0 367120 111952 8976 S 0.0 1.4 0:19.07
>> heat-engine
>> 7316 heat 20 0 367120 111948 8976 S 0.0 1.4 0:19.32
>> heat-engine
>> 7318 heat 20 0 367120 111944 8976 S 0.0 1.4 0:19.41
>> heat-engine
>> 7314 heat 20 0 367120 111932 8976 S 0.0 1.4 0:21.93
>> heat-engine
>> 7010 nova 20 0 293424 110600 14784 S 0.0 1.4 0:23.89
>> nova-novncproxy
>> 2639 heat 20 0 351792 110524 18732 S 0.0 1.4 0:01.91 heat-api
>> 2642 heat 20 0 351276 109872 18560 S 0.0 1.3 0:02.26
>> heat-api-cfn
>> 6871 glance 20 0 219776 106296 11764 S 0.0 1.3 0:01.84
>> glance-registry
>> 7155 glance 20 0 227556 104544 6492 S 0.0 1.3 0:00.80
>> glance-registry
>> 7154 glance 20 0 227044 104188 6492 S 0.0 1.3 0:00.74
>> glance-registry
>> 7790 neutron 20 0 249672 97980 12176 R 5.2 1.2 27:52.52
>> neutron-openvsw
>> 6918 neutron 20 0 248556 96980 12508 S 1.7 1.2 15:48.44
>> neutron-l3-agen
>> 6946 neutron 20 0 248192 96276 12168 S 1.7 1.2 15:36.30
>> neutron-dhcp-ag
>> 429 keystone 20 0 415944 95992 10152 S 0.0 1.2 0:03.23 apache2
>> 426 keystone 20 0 415944 95980 10152 S 0.0 1.2 0:03.52 apache2
>> 7183 heat 20 0 351792 95880 4084 S 0.0 1.2 0:00.32 heat-api
>> 7184 heat 20 0 351792 95876 4084 S 0.0 1.2 0:00.29 heat-api
>> 427 keystone 20 0 415688 95768 10152 S 0.0 1.2 0:02.94 apache2
>> 425 keystone 20 0 415688 95764 10152 S 0.0 1.2 0:03.00 apache2
>> 6913 neutron 20 0 239924 92332 12176 S 2.6 1.1 18:50.89
>> neutron-lbaas-a
>> 6914 neutron 20 0 243752 91964 12164 S 1.7 1.1 15:10.37
>> neutron-metadat
>> 7172 neutron 20 0 244264 87276 7092 S 1.7 1.1 14:23.84
>> neutron-metadat
>> 20015 neutron 20 0 186940 46012 4252 S 0.0 0.6 0:01.06
>> neutron-ns-meta
>> 18217 neutron 20 0 186524 45832 4304 S 0.0 0.6 0:00.82
>> neutron-ns-meta
>> 16980 neutron 20 0 186412 45784 4380 S 0.0 0.6 0:00.51
>> neutron-ns-meta
>> 20083 neutron 20 0 186548 45756 4284 S 0.0 0.6 0:00.59
>> neutron-ns-meta
>> 
>> +++
>> 
>> When controller is getting out of memory everything goes extremely slow.
>> Restarting apache2 and heat solves the problem. ;-)
>> 
>> Be lucky next week. :-)
>> 
>> On 2016-08-18 20:53, Satish Patel wrote:
>> 
>> How many folks using mitaka in production? I have search on google and
>> i didn't find people using mitaka because its very new and people are
>> not use to with it
>> 
>> On Thu, Aug 18, 2016 at 2:45 PM, Satish Patel <satish.txt at gmail.com> wrote:
>> 
>> we are running out of time.. this installation will go to production
>> in next week.. I need to deliver something soon.
>> 
>> I am installing liberty now. and see how it works..!
>> 
>> On Thu, Aug 18, 2016 at 1:41 PM, Nasir Mahmood <nasir.mahmood at gmail.com>
>> wrote:
>> 
>> Did you try to locate whats causing to use all your memory. Try using htop,
>> truss/strace and other debugging tools. You might also be a victim of
>> resource bottlenecks, like your HDD are serving slow or it may be that your
>> neutron conf is inappropriate at some point. If this ain't a production
>> environment, try troubleshooting rather than going for old installation.
>> 
>> On Aug 18, 2016 21:56, "Satish Patel" <satish.txt at gmail.com> wrote:
>> 
>> I have recently install mitaka openstack using RDO on HP DL360 G9
>> server with 16GB memory. but after installation it took all of my
>> memory and suck all my swap memory too. and it was extremely slow. I
>> have found neutron-server and nova-conductor and other component was
>> eating most memory and growing. is this normal?
>> 
>> I didn't find related solution on google so now i have no option
>> except going back to older liberty version.
>> 
>> Any suggestion folks?
>> 
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
>> Post to : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
>> 
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
>> Post to : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
>> 
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
>> Post to : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
 

Links:
------
[1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160822/9302347c/attachment.html>


More information about the Openstack mailing list