Open Stack

Sorry, I do forget that.

Still 2 issues left:

Issue 1:

    After enabled hugepage:

        cat /proc/meminfo |grep -i huge
            AnonHugePages:   1685504 kB
            HugePages_Total:    3072
            HugePages_Free:     3072
            HugePages_Rsvd:        0
            HugePages_Surp:        0
            Hugepagesize:       2048 kB

        cat /proc/sys/vm/nr_hugepages
            3072

        I still get error in compute node even after I change the flavor from m1.medium  to m1.small:

        2015-02-03 09:03:23.562 ERROR nova.compute.manager [-] [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c] Instance failed to spawn
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c] Traceback (most recent call last):
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/opt/stack/nova/nova/compute/manager.py", line 2306, in _build_resources
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     yield resources
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/opt/stack/nova/nova/compute/manager.py", line 2176, in _build_and_run_instance
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     flavor=flavor)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2320, in spawn
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     block_device_info=block_device_info)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4236, in _create_domain_and_network
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     power_on=power_on)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4167, in _create_domain
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     LOG.error(err)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     six.reraise(self.type_, self.value, self.tb)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4157, in _create_domain
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     domain.createWithFlags(launch_flags)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     result = proxy_call(self._autowrap, f, *args, **kwargs)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     rv = execute(f, *args, **kwargs)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     six.reraise(c, e, tb)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     rv = meth(*args, **kwargs)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 993, in createWithFlags
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c] libvirtError: internal error: Unable to find any usable hugetlbfs mount for 2048 KiB
        2015-02-03 09:03:23.562 TRACE nova.compute.manager [instance: 4220d096-55fe-4d01-89b3-dc937e34c97c]




Issue 2:
    I removed the hugepage setting:

        nova flavor-show m1.small
        +----------------------------+------------------------+
        | Property                   | Value                  |
        +----------------------------+------------------------+
        | OS-FLV-DISABLED:disabled   | False                  |
        | OS-FLV-EXT-DATA:ephemeral  | 0                      |
        | disk                       | 20                     |
        | extra_specs                | {"hw:numa_nodes": "1"} |
        | id                         | 2                      |
        | name                       | m1.small               |
        | os-flavor-access:is_public | True                   |
        | ram                        | 2048                   |
        | rxtx_factor                | 1.0                    |
        | swap                       |                        |
        | vcpus                      | 1                      |
        +----------------------------+------------------------+

    The I get error:
        2015-02-03 09:15:43.921 ERROR nova.compute.manager [-] [instance: 833cfac5-4fac-438b-acff-579b41ee5729] Instance failed to spawn
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729] Traceback (most recent call last):
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/opt/stack/nova/nova/compute/manager.py", line 2306, in _build_resources
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     yield resources
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/opt/stack/nova/nova/compute/manager.py", line 2176, in _build_and_run_instance
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     flavor=flavor)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2320, in spawn
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     block_device_info=block_device_info)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4236, in _create_domain_and_network
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     power_on=power_on)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4167, in _create_domain
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     LOG.error(err)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     six.reraise(self.type_, self.value, self.tb)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4157, in _create_domain
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     domain.createWithFlags(launch_flags)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     result = proxy_call(self._autowrap, f, *args, **kwargs)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     rv = execute(f, *args, **kwargs)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     six.reraise(c, e, tb)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     rv = meth(*args, **kwargs)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 993, in createWithFlags
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729] libvirtError: internal error: process exited while connecting to monitor: 2015-02-03T01:15:43.331770Z qemu-system-x86_64: -object memory-backend-ram,size=2048M,id=ram-node0,host-nodes=0,policy=bind: NUMA node binding are not supported by this QEMU
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]
        2015-02-03 09:15:43.921 TRACE nova.compute.manager [instance: 833cfac5-4fac-438b-acff-579b41ee5729]


What should I do to solve this ?

Thanks.
-chen


-----Original Message-----
From: Daniel P. Berrange [mailto:berrange at redhat.com] 
Sent: Monday, February 02, 2015 5:54 PM
To: Li, Chen
Cc: openstack at lists.openstack.org
Subject: Re: [Openstack] [Nova][virt-driver-numa-placement]How to enbale instance with numa ?

>     nova flavor-key m1.medium set hw:mem_page_size=large

This second settings says that Nova must use large pages for the VM memory.
Did you actually reserve any large pages for use by VMs on the host in question ? This is an upfront admin task, not something Nova can do itself.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|