<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle18
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m trying to enable numa for my instance based on <a href="https://blueprints.launchpad.net/nova/+spec/virt-driver-numa-placement">
https://blueprints.launchpad.net/nova/+spec/virt-driver-numa-placement</a>.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m working under Ubuntu 14.10, with libvirt <span style="color:#4F81BD">
1.2.8:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#4F81BD"> virsh -v<o:p></o:p></span></p>
<p class="MsoNormal"> 1.2.8<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My compute node has only 1 numa node:<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#4F81BD"> dmesg |grep numa<o:p></o:p></span></p>
<p class="MsoNormal"> [ 0.000000] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#4F81BD"> numactl --hardware<o:p></o:p></span></p>
<p class="MsoNormal"> available: 1 nodes (0)<o:p></o:p></p>
<p class="MsoNormal"> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23<o:p></o:p></p>
<p class="MsoNormal"> node 0 size: 15983 MB<o:p></o:p></p>
<p class="MsoNormal"> node 0 free: 8085 MB<o:p></o:p></p>
<p class="MsoNormal"> node distances:<o:p></o:p></p>
<p class="MsoNormal"> node 0<o:p></o:p></p>
<p class="MsoNormal"> 0: 10<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I installed a fresh devstack.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And updated flavor <span style="color:#1F497D">m1.medium</span> with:<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> nova flavor-key m1.medium set hw:numa_nodes=1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> nova flavor-key m1.medium set hw:mem_page_size=large<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal">Then I try to start instance with command:<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span>nova boot --flavor m1.medium --image cirros-0.3.2-x86_64-uec --nic net-id=9b2afc82-b9d0-49ce-be21-732f3af506eb test1<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The instance start failed due to reason:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">| fault | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/opt/stack/nova/nova/conductor/manager.py\", line 651, in build_instances |<o:p></o:p></p>
<p class="MsoNormal">| | request_spec, filter_properties) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/opt/stack/nova/nova/scheduler/utils.py\", line 333, in wrapped |<o:p></o:p></p>
<p class="MsoNormal">| | return func(*args, **kwargs) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 52, in select_destinations |<o:p></o:p></p>
<p class="MsoNormal">| | context, request_spec, filter_properties) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 37, in __run_method |<o:p></o:p></p>
<p class="MsoNormal">| | return getattr(self.instance, __name)(*args, **kwargs) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/opt/stack/nova/nova/scheduler/client/query.py\", line 34, in select_destinations |<o:p></o:p></p>
<p class="MsoNormal">| | context, request_spec, filter_properties) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/opt/stack/nova/nova/scheduler/rpcapi.py\", line 114, in select_destinations |<o:p></o:p></p>
<p class="MsoNormal">| | request_spec=request_spec, filter_properties=filter_properties) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line 156, in call |<o:p></o:p></p>
<p class="MsoNormal">| | retry=self.retry) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line 90, in _send |<o:p></o:p></p>
<p class="MsoNormal">| | timeout=timeout, retry=retry) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 417, in send |<o:p></o:p></p>
<p class="MsoNormal">| | retry=retry) |<o:p></o:p></p>
<p class="MsoNormal">| | File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 408, in _send |<o:p></o:p></p>
<p class="MsoNormal">| | raise result |<o:p></o:p></p>
<p class="MsoNormal">| | ", "created": "2015-02-02T06:23:41Z"} |<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The log in compute.log:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.736 AUDIT nova.compute.manager [req-7e1b0ad2-0349-4afc-961b-322b3334f67a admin demo] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb]
<span style="color:#4F81BD">Starting instance...<o:p></o:p></span></p>
<p class="MsoNormal">2015-02-02 14:59:47.738 DEBUG oslo_messaging._drivers.amqpdriver [-] MSG_ID is c323d0b819414dc38fe0188baef2cc98 from (pid=22569) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:378<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.738 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID is 2ce2555b27e6438db8c8d9043ab2a2f7. from (pid=22569) _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:224<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.900 DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" acquired by "instance_claim" :: waited 0.000s from (pid=22569) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:430<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.900 DEBUG nova.compute.resource_tracker [-] Memory overhead for 4096 MB instance; 0 MB from (pid=22569) instance_claim /opt/stack/nova/nova/compute/resource_tracker.py:130<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.902 AUDIT nova.compute.claims [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb] Attempting claim: memory 4096 MB, disk 40 GB<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.902 AUDIT nova.compute.claims [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb] Total memory: 15983 MB, used: 512.00 MB<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.902 AUDIT nova.compute.claims [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb] memory limit: 23974.50 MB, free: 23462.50 MB<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.902 AUDIT nova.compute.claims [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb] Total disk: 915 GB, used: 0.00 GB<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.902 AUDIT nova.compute.claims [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb] disk limit not specified, defaulting to unlimited<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.903 DEBUG oslo_messaging._drivers.amqpdriver [-] MSG_ID is 8010409e00644a43a4672f705e83a6cd from (pid=22569) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:378<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.903 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID is 2c8e3b9dc627498d878fc7a8db7e92eb. from (pid=22569) _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:224<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.913 DEBUG nova.compute.resources.vcpu [-] Total CPUs: 24 VCPUs, used: 0.00 VCPUs from (pid=22569) test /opt/stack/nova/nova/compute/resources/vcpu.py:51<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.913 DEBUG nova.compute.resources.vcpu [-] CPUs limit not specified, defaulting to unlimited from (pid=22569) test /opt/stack/nova/nova/compute/resources/vcpu.py:55<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.913 DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" released by "instance_claim" :: held 0.013s from (pid=22569) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:442<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.913 DEBUG nova.compute.manager [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb]
<span style="color:#4F81BD">Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology</span>. from (pid=22569) _build_and<o:p></o:p></p>
<p class="MsoNormal">_run_instance /opt/stack/nova/nova/compute/manager.py:2183<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.914 DEBUG nova.compute.utils [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb]
<span style="color:#4F81BD">Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology</span>. from (pid=22569) notify_about<o:p></o:p></p>
<p class="MsoNormal">_instance_usage /opt/stack/nova/nova/compute/utils.py:324<o:p></o:p></p>
<p class="MsoNormal">2015-02-02 14:59:47.914 DEBUG nova.compute.manager [-] [instance: 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb] Build of instance 1a555c06-b1fd-405d-bc1a-d20a8ac1b7fb was re-scheduled: Insufficient compute resources: Requested instance NUMA topo<o:p></o:p></p>
<p class="MsoNormal">logy cannot fit the given host NUMA topology. from (pid=22569) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2080<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Anyone know why this happened ??<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Also, if the compute node can’t support <span style="color:#4F81BD">
instance NUMA topology </span>, isn’t it should failed in scheduler ???<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
<p class="MsoNormal">-chen<o:p></o:p></p>
</div>
</body>
</html>