<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi,<div class=""><br class=""></div><div class="">The issue is apparently solved; we found a solution here <a href="https://www.stackhpc.com/tripleo-numa-vcpu-pinning.html" class="">https://www.stackhpc.com/tripleo-numa-vcpu-pinning.html</a> where libvirt and qemu-kvm version restrictions where indicated. The CentOS7.3 repo has an older qemu-kvm version (1.5.3) than the one needed (>= 2.1.0), so we added the kvm-common repo, as recommended by the web. Now 1 host is returned (Filter NUMATopologyFilter returned 1 hosts) and the guest VM has the desired cpu topology.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><div class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><span style="background-color: rgb(255, 255, 255);" class="">-- </span><br class=""><p style="margin-bottom: 0in;" class="">Met vriendelijke groeten / Best regards,</p><p style="margin-bottom: 0in;" class=""><b class="">Paco Bernabé</b></p><p style="margin-bottom: 0in;" class="">Senior Systemsprogrammer | SURFsara | Science Park 140 | 1098XG Amsterdam | T +31 610 961 785 | <a class="moz-txt-link-abbreviated" href="mailto:paco@surfsara.nl">paco@surfsara.nl</a> | <a class="moz-txt-link-abbreviated" href="http://www.surfsara.nl">www.surfsara.nl</a></p></div><div class=""><br class=""></div></div></div></div></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span><br class="Apple-interchange-newline"><span><img apple-inline="yes" id="C17AC126-7C44-40D0-BC69-E17CE1A530DB" src="cid:F52D3696-25B3-4C0A-9B4C-CB60E1FFD30F@wp.surfsara.nl" class=""></span>
</span></span></div>
</div><br class=""><div><blockquote type="cite" class=""><div class="">Op 13 apr. 2017, om 11:38 heeft Paco Bernabé <<a href="mailto:Francisco.Bernabe@surfsara.nl" class="">Francisco.Bernabe@surfsara.nl</a>> het volgende geschreven:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi,<div class=""><br class=""></div><div class="">More info; in de log file of the nova-scheduler we see messages like (<HOST> is de compute host name):</div><div class=""><br class=""></div><div class=""><ul class="MailOutline"><li class=""><HOST>, <HOST> fails NUMA topology requirements. No host NUMA topology while the instance specified one. host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/numa_topology_filter.py:100</li><li class="">Filter NUMATopologyFilter returned 0 hosts</li></ul><div class=""><br class=""></div><div class="">So, we are not sure if the filters are ok in nova.conf:</div><div class=""><br class=""></div></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Courier New" class="">scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,CoreFilter,NUMATopologyFilter</font></div></div></blockquote><div class=""><div class=""><br class=""></div></div><div class=""><div class="">
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><span style="background-color: rgb(255, 255, 255);" class="">-- </span><br class=""><p style="margin-bottom: 0in;" class="">Met vriendelijke groeten / Best regards,</p><p style="margin-bottom: 0in;" class=""><b class="">Paco Bernabé</b></p><p style="margin-bottom: 0in;" class="">Senior Systemsprogrammer | SURFsara | Science Park 140 | 1098XG Amsterdam | T +31 610 961 785 | <a class="moz-txt-link-abbreviated" href="mailto:paco@surfsara.nl">paco@surfsara.nl</a> | <a class="moz-txt-link-abbreviated" href="http://www.surfsara.nl/">www.surfsara.nl</a></p></div><div class=""><br class=""></div></div></div></div></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span class=""><br class="Apple-interchange-newline"><span class=""><span id="cid:F52D3696-25B3-4C0A-9B4C-CB60E1FFD30F@wp.surfsara.nl"><signature.jpg></span></span>
</span></span></div>
</div><br class=""><div class=""><blockquote type="cite" class=""><div class="">Op 13 apr. 2017, om 09:34 heeft Paco Bernabé <<a href="mailto:Francisco.Bernabe@surfsara.nl" class="">Francisco.Bernabe@surfsara.nl</a>> het volgende geschreven:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Hi,</div><div class=""><br class=""></div><div class="">After reading the following articles:</div><div class=""><br class=""></div><div class=""><ul class="MailOutline"><li class=""><a href="https://docs.openstack.org/admin-guide/compute-flavors.html" class="">https://docs.openstack.org/admin-guide/compute-flavors.html</a></li><li class=""><a href="http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-topology-awareness-in-openstack-compute/" class="">http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-topology-awareness-in-openstack-compute/</a></li><li class=""><a href="http://openstack-in-production.blogspot.nl/2015/08/numa-and-cpu-pinning-in-high-throughput.html" class="">http://openstack-in-production.blogspot.nl/2015/08/numa-and-cpu-pinning-in-high-throughput.html</a></li><li class=""><a href="http://www.stratoscale.com/blog/openstack/cpu-pinning-and-numa-awareness/" class="">http://www.stratoscale.com/blog/openstack/cpu-pinning-and-numa-awareness/</a></li></ul></div><div class=""><br class=""></div><div class="">We are not able yet to expose the NUMA config to the guest VM. This is the configuration of one of our compute nodes:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Courier New" class=""># lscpu</font></div></div><div class=""><div class=""><font face="Courier New" class="">Architecture: x86_64</font></div></div><div class=""><div class=""><font face="Courier New" class="">CPU op-mode(s): 32-bit, 64-bit</font></div></div><div class=""><div class=""><font face="Courier New" class="">Byte Order: Little Endian</font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">CPU(s): 48</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">On-line CPU(s) list: 0-47</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">Thread(s) per core: 2</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">Core(s) per socket: 12</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">Socket(s): 2</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">NUMA node(s): 4</b></font></div></div><div class=""><div class=""><font face="Courier New" class="">Vendor ID: GenuineIntel</font></div></div><div class=""><div class=""><font face="Courier New" class="">CPU family: 6</font></div></div><div class=""><div class=""><font face="Courier New" class="">Model: 79</font></div></div><div class=""><div class=""><font face="Courier New" class="">Model name: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz</font></div></div><div class=""><div class=""><font face="Courier New" class="">Stepping: 1</font></div></div><div class=""><div class=""><font face="Courier New" class="">CPU MHz: 2266.085</font></div></div><div class=""><div class=""><font face="Courier New" class="">BogoMIPS: 4404.00</font></div></div><div class=""><div class=""><font face="Courier New" class="">Virtualization: VT-x</font></div></div><div class=""><div class=""><font face="Courier New" class="">L1d cache: 32K</font></div></div><div class=""><div class=""><font face="Courier New" class="">L1i cache: 32K</font></div></div><div class=""><div class=""><font face="Courier New" class="">L2 cache: 256K</font></div></div><div class=""><div class=""><font face="Courier New" class="">L3 cache: 15360K</font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">NUMA node0 CPU(s): 0-5,24-29</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">NUMA node1 CPU(s): 6-11,30-35</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">NUMA node2 CPU(s): 12-17,36-41</b></font></div></div><div class=""><div class=""><font face="Courier New" class=""><b class="">NUMA node3 CPU(s): 18-23,42-47</b></font></div></div></blockquote><div class=""><br class=""></div><div class=""><br class=""></div><div class="">And this is the flavour configuration:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Courier New" class="">OS-FLV-DISABLED:disabled | False </font></div></div><div class=""><div class=""><font face="Courier New" class="">OS-FLV-EXT-DATA:ephemeral | 2048 </font></div></div><div class=""><div class=""><font face="Courier New" class="">disk | 30 </font></div></div><div class=""><div class=""><b class=""><font face="Courier New" class="">extra_specs | {"hw:numa_nodes": "8", "hw:numa_cpus.0": "0-5", "hw:numa_cpus.1": "6-11", "hw:numa_cpus.2": "12-17", "hw:numa_cpus.3": "18-23", "hw:numa_cpus.4": "24-29", "hw:numa_cpus.5": "30-35", "hw:numa_cpus.6": "36-41", "hw:numa_cpus.7": "42-45", "hw:numa_mem.7": "16384", "hw:numa_mem.6": "24576", "hw:numa_mem.5": "24576", "hw:numa_mem.4": "24576", "hw:numa_mem.3": "24576", "hw:numa_mem.2": "24576", "hw:numa_mem.1": "24576", "hw:numa_mem.0": "24576"} </font></b></div></div><div class=""><div class=""><font face="Courier New" class="">os-flavor-access:is_public | True ram | 188416 rxtx_factor | 1.0 vcpus | 46</font></div></div></blockquote><div class=""><br class=""></div><div class="">We have set 8 Numa nodes, because we read that non-continous ranges of CPUs are not supported in CentOS7 and the solution is to create 2 times the number of Numa nodes. What you see below is what is passed to libvirt on the compute node:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Courier New" class=""><cpu mode='host-passthrough'></font></div></div><div class=""><div class=""><font face="Courier New" class=""> <topology sockets=’46' cores='1' threads='1'/></font></div></div><div class=""><div class=""><font face="Courier New" class=""></cpu></font></div></div></blockquote><div class=""><br class=""></div><div class="">But we want something like:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><font face="Courier New" class=""><cpu mode='host-passthrough'></font></div></div><div class=""><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><font face="Courier New" class=""><span class="Apple-tab-span" style="white-space:pre"> </span><numa></font></div></div><div class=""><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><font face="Courier New" class=""><span class="Apple-tab-span" style="white-space:pre"> </span><cell id='0' cpus=‘0-5’ memory=‘24576’/></font></div></div><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><div class=""><div class="separator" style="font-variant-ligatures: normal; clear: both;"><font face="Courier New" class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><cell id=‘1' cpus=‘6-11’ memory=‘24576'/></font></div></div><div class=""></div></div><div class=""><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><font face="Courier New" class=""><span class="Apple-tab-span" style="white-space:pre"> </span></font><font face="Courier New" class="">…</font></div></div><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><div class="separator" style="font-variant-ligatures: normal; clear: both;"><div class=""><div class="separator" style="font-variant-ligatures: normal; clear: both;"><font face="Courier New" class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><cell id=‘6' cpus=’36-41’ memory=‘24576'/></font></div></div><div class=""></div></div><div class=""></div></div><div class=""><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><font face="Courier New" class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><cell id=‘7' cpus=’42-45' memory=‘16384'/></font></div></div><div class=""><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><font face="Courier New" class=""><span class="Apple-tab-span" style="white-space:pre"> </span></numa></font></div></div><div class=""><div class="separator" style="font-variant-ligatures: normal; orphans: 2; widows: 2; background-color: rgb(255, 255, 255); clear: both;"><font face="Courier New" class=""></cpu></font></div></div></blockquote><div class=""><br class="webkit-block-placeholder"></div><div class="">We have edited nova.conf at the compute node with the parameter and value cpu_mode=host-passthrough. On the nova scheduler we have added NumaTopologyFilter to the parameter scheduler_default_filters in nova.conf. Of course, we have restarted all openstack services at the controller and the nova-compute at the compute node.</div><div class=""><br class=""></div><div class="">We also have tried with a simpler version with the following extra specs:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><font face="Courier New" class="">| extra_specs | {"hw:numa_cpus.0": "0,1,2,3,4,5", "hw:numa_nodes": "1", "hw:numa_mem.0": "24576"} |</font></div></blockquote><div class=""><br class="webkit-block-placeholder"></div><div class="">But we still see:</div><div class=""><br class=""></div><div class=""><blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;" class=""><div class=""><font face="Courier New" class=""><cpu mode='host-passthrough'></font></div><div class=""><font face="Courier New" class=""> <topology sockets=’6' cores='1' threads='1'/></font></div><div class=""><font face="Courier New" class=""></cpu></font></div><div class=""><br class=""></div></blockquote></div><div class="">Any idea? I’m sure there must be something that we have skipped. Has the pinning something to do? What we understand is that it’s only for performance, but that should be the next step and it wouldn’t interfere in what we are trying to achieve. Thanks in advance.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><span style="background-color: rgb(255, 255, 255);" class="">-- </span><br class=""><p style="margin-bottom: 0in;" class="">Met vriendelijke groeten / Best regards,</p><p style="margin-bottom: 0in;" class=""><b class="">Paco Bernabé</b></p><p style="margin-bottom: 0in;" class="">Senior Systemsprogrammer | SURFsara | Science Park 140 | 1098XG Amsterdam | T +31 610 961 785 | <a class="moz-txt-link-abbreviated" href="mailto:paco@surfsara.nl">paco@surfsara.nl</a> | <a class="moz-txt-link-abbreviated" href="http://www.surfsara.nl/">www.surfsara.nl</a></p></div><div class=""><br class=""></div></div></div></div></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span class=""><br class="Apple-interchange-newline"><span class=""><span id="cid:F52D3696-25B3-4C0A-9B4C-CB60E1FFD30F@wp.surfsara.nl" class=""><signature.jpg></span></span>
</span></span></div>
</div><br class=""></div></div>_______________________________________________<br class="">OpenStack-operators mailing list<br class=""><a href="mailto:OpenStack-operators@lists.openstack.org" class="">OpenStack-operators@lists.openstack.org</a><br class=""><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br class=""></div></blockquote></div><br class=""></div></div>_______________________________________________<br class="">OpenStack-operators mailing list<br class=""><a href="mailto:OpenStack-operators@lists.openstack.org" class="">OpenStack-operators@lists.openstack.org</a><br class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators<br class=""></div></blockquote></div><br class=""></div></body></html>