Open Stack

My instances are using much more memory that expected.  The amount free
memory (free + cached) is under 3G on my servers even though the compute
nodes are configured to reserve 32G.

Here's my setup:
Release: Ice House
Server mem: 256G
Qemu version: 2.0.0+dfsg-2ubuntu1.1
Networking: Contrail 1.20
Block storage: Ceph 0.80.7
Hypervisor OS: Ubuntu 12.04
memory over-provisioning is disabled
kernel version: 3.11.0-26-generic

On nova.conf
reserved_host_memory_mb = 32768

Info on instances:
- root volume is file backed (qcow2) on the hypervisor local storage
- each instance has a rbd volume mounted from Ceph
- no swap file/partition

I've confirmed, via nova-compute.log, that nova is respecting the
reserved_host_memory_mb directive and is not over-provisioning.  On some
hypervisors, nova-compute says there's 4GB available for use even though
the OS has less that 4G left (free +cached)!

I've also summed up the memory from /etc/libvir/qemu/*.xml files and the
total looks good.

Each hypervisor hosts about 45-50 instances.

Is there good way to calculate the actual usage of each QEMU process?

PS: I've tried free, summing up RSS, and smem but none of them can tell me
where's the missing mem.

thx
mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150623/e53717f7/attachment.html>