[openstack-dev] [nova] NUMA, huge pages, and scheduling

Chris Friesen chris.friesen at windriver.com
Tue Jun 14 14:27:54 UTC 2016

On 06/13/2016 02:17 PM, Paul Michali wrote:
> Hmm... I tried Friday and again today, and I'm not seeing the VMs being evenly
> created on the NUMA nodes. Every Cirros VM is created on nodeid 0.
> I have the m1/small flavor (@GB) selected and am using hw:numa_nodes=1 and
> hw:mem_page_size=2048 flavor-key settings. Each VM is consuming 1024 huge pages
> (of size 2MB), but is on nodeid 0 always. Also, it seems that when I reach 1/2
> of the total number of huge pages used, libvirt gives an error saying there is
> not enough memory to create the VM. Is it expected that the huge pages are
> "allocated" to each NUMA node?

Yes, any given memory page exists on one NUMA node, and a single-NUMA-node VM 
will be constrained to a single host NUMA node and will use memory from that 
host NUMA node.

You can see and/or adjust how many hugepages are available on each NUMA node via 
/sys/devices/system/node/nodeX/hugepages/hugepages-2048kB/* where X is the host 
NUMA node number.


More information about the OpenStack-dev mailing list