<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Courier New";
panose-1:2 7 3 9 2 2 5 2 4 4;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:404885643;
mso-list-template-ids:-961097012;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style>
</head>
<body bgcolor="white" lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US">Fully agreed … great documentation. I’ve added the link to all the CERN blogs to include Recent 2017 documentation is now at h<a href="ttp://docs.openstack.org/admin-guide/compute-adv-config.html">ttp://docs.openstack.org/admin-guide/compute-adv-config.html</a>
so that people find the latest recommendations.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US">Tim<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-left:36.0pt"><b><span style="font-family:Calibri;color:black">From:
</span></b><span style="font-family:Calibri;color:black">Arne Wiebalck <arne.wiebalck@cern.ch><br>
<b>Date: </b>Tuesday, 7 February 2017 at 20:20<br>
<b>To: </b>Steve Gordon <sgordon@redhat.com><br>
<b>Cc: </b>"openstack@lists.openstack.org" <openstack@lists.openstack.org><br>
<b>Subject: </b>Re: [Openstack] nova assign isntances to cpu pinning<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:36.0pt">On 07 Feb 2017, at 20:05, Steve Gordon <<a href="mailto:sgordon@redhat.com">sgordon@redhat.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><br>
<br>
----- Original Message -----<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:36.0pt">From: "Arne Wiebalck" <<a href="mailto:arne.wiebalck@cern.ch">arne.wiebalck@cern.ch</a>><br>
To: "Steve Gordon" <<a href="mailto:sgordon@redhat.com">sgordon@redhat.com</a>><br>
Cc: "Manuel Sopena Ballesteros" <<a href="mailto:manuel.sb@garvan.org.au">manuel.sb@garvan.org.au</a>>,
<a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Sent: Tuesday, February 7, 2017 2:00:23 PM<br>
Subject: Re: [Openstack] nova assign isntances to cpu pinning<br>
<br>
<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:36.0pt">On 07 Feb 2017, at 18:57, Steve Gordon <<a href="mailto:sgordon@redhat.com">sgordon@redhat.com</a>> wrote:<br>
<br>
----- Original Message -----<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:36.0pt">From: "Arne Wiebalck" <<a href="mailto:Arne.Wiebalck@cern.ch">Arne.Wiebalck@cern.ch</a><br>
<<a href="mailto:Arne.Wiebalck@cern.ch">mailto:Arne.Wiebalck@cern.ch</a>>><br>
To: "Manuel Sopena Ballesteros" <<a href="mailto:manuel.sb@garvan.org.au">manuel.sb@garvan.org.au</a><br>
<<a href="mailto:manuel.sb@garvan.org.au">mailto:manuel.sb@garvan.org.au</a>>><br>
Cc: <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a> <<a href="mailto:openstack@lists.openstack.org">mailto:openstack@lists.openstack.org</a>><br>
Sent: Tuesday, February 7, 2017 2:46:39 AM<br>
Subject: Re: [Openstack] nova assign isntances to cpu pinning<br>
<br>
Manuel,<br>
<br>
Rather than with aggregate metadata we assign instances to NUMA nodes via<br>
flavor extra_specs,<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-left:36.0pt"><br>
These are not necessarily mutually exclusive, the purpose of the aggregates<br>
in the documentation (and assumed in the original design) is to segregate<br>
compute nodes for dedicated guests (if using truly dedicated CPU by<br>
setting "hw:cpu_policy" to "dedicated" as Chris mentions) from those for<br>
over-committed guests. If you are only using the NUMA node alignment (as<br>
shown below) this doesn't apply, because it's only guaranteeing how many<br>
nodes your guest will be spread across not that it will have dedicated<br>
access to the CPU(s) it is on. Folks who want truly dedicated vCPU:pCPU<br>
mappings should still use the aggregates, unless *only* running workloads<br>
with dedicated CPU needs.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-left:36.0pt"><br>
Right: we’re using cells to separate computing-intensive from overcommitted<br>
resources, configured the flavor<br>
extra-specs for NUMA nodes (and huge pages) only in the compute part and have<br>
made good experiences<br>
with this setup.<br>
Depending on the use case and the individual deployment, there are certainly<br>
different options to set things up.<br>
If not already available somewhere, it may be good to document the options<br>
depending on needs and setup?<br>
<br>
Cheers,<br>
Arne<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-left:36.0pt"><br>
A fairly decent amount of it was contributed by some of the Nova folks who worked on this functionality and ended up in the Admin Guide here:<br>
<br>
<a href="http://docs.openstack.org/admin-guide/compute-adv-config.html">http://docs.openstack.org/admin-guide/compute-adv-config.html</a><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt">Awesome, thanks for pointing this out!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"> Arne<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:36.0pt">-Steve<br>
<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:36.0pt">i.e. nova flavor-show reports something like<br>
<br>
—><br>
| extra_specs | {"hw:numa_nodes": "1"} |<br>
<—<br>
<br>
for our NUMA-aware flavors.<br>
<br>
This seems to work pretty well and gives the desired performance<br>
improvement.<br>
<br>
Cheers,<br>
Arne<br>
<br>
<br>
<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-left:36.0pt">On 07 Feb 2017, at 01:19, Manuel Sopena Ballesteros<br>
<<a href="mailto:manuel.sb@garvan.org.au">manuel.sb@garvan.org.au</a>> wrote:<br>
<br>
Hi,<br>
<br>
I am trying to isolate my instances by cpu socket in order to improve my<br>
NUMA hardware performance.<br>
<br>
[root@openstack-dev ~(keystone_admin)]# nova aggregate-set-metadata numa<br>
pinned=true<br>
Metadata has been successfully updated for aggregate 1.<br>
+----+--------+--------------------+--------+--------------------+<br>
| Id | Name | Availability Zone | Hosts | Metadata |<br>
+----+--------+--------------------+--------+--------------------+<br>
| 1 | numa | - | |<br>
| 'pinned=true' |<br>
+----+--------+--------------------+--------+--------------------+<br>
<br>
I have done the changes on the nova metadata but my admin can’t see the<br>
instances<br>
<br>
[root@openstack-dev ~(keystone_admin)]# nova aggregate-add-host numa<br>
4d4f3c3f-2894-4244-b74c-2c479e296ff8<br>
ERROR (NotFound): Compute host 4d4f3c3f-2894-4244-b74c-2c479e296ff8 could<br>
not be found. (HTTP 404) (Request-ID:<br>
req-286985d8-d6ce-429e-b234-dd5eac5ad62e)<br>
<br>
And the user who has access to those instances does not have privileges<br>
to<br>
add the hosts<br>
<br>
[root@openstack-dev ~(keystone_myuser)]# nova aggregate-add-host numa<br>
4d4f3c3f-2894-4244-b74c-2c479e296ff8<br>
ERROR (Forbidden): Policy doesn't allow<br>
os_compute_api:os-aggregates:index<br>
to be performed. (HTTP 403) (Request-ID:<br>
req-a5687fd4-c00d-4b64-af9e-bd5a82eb99c1)<br>
<br>
What would be the recommended way to do this?<br>
<br>
Thank you very much<br>
<br>
Manuel Sopena Ballesteros | Big data Engineer<br>
Garvan Institute of Medical Research<br>
The Kinghorn Cancer Centre, 370 Victoria Street, Darlinghurst, NSW 2010<br>
T: + 61 (0)2 9355 5760 | F: +61 (0)2 9295 8507 | E:<br>
<a href="mailto:manuel.sb@garvan.org.au">manuel.sb@garvan.org.au</a><br>
<br>
NOTICE<br>
Please consider the environment before printing this email. This message<br>
and any attachments are intended for the addressee named and may contain<br>
legally privileged/confidential/copyright information. If you are not the<br>
intended recipient, you should not read, use, disclose, copy or<br>
distribute<br>
this communication. If you have received this message in error please<br>
notify us at once by return email and then delete both messages. We<br>
accept<br>
no liability for the distribution of viruses or similar in electronic<br>
communications. This notice should not be removed.<br>
_______________________________________________<br>
Mailing list:<br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack<br>
Post to : openstack@lists.openstack.org<br>
Unsubscribe :<br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt">
<br>
--<br>
Arne Wiebalck<br>
CERN IT<br>
<br>
_______________________________________________<br>
Mailing list:<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : openstack@lists.openstack.org<br>
Unsubscribe :<br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack<o:p></o:p></p>
</blockquote>
</blockquote>
</blockquote>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>