Open Stack

Fully agreed … great documentation. I’ve added the link to all the CERN blogs to include Recent 2017 documentation is now at http://docs.openstack.org/admin-guide/compute-adv-config.html so that people find the latest recommendations.

Tim


From: Arne Wiebalck <arne.wiebalck at cern.ch>
Date: Tuesday, 7 February 2017 at 20:20
To: Steve Gordon <sgordon at redhat.com>
Cc: "openstack at lists.openstack.org" <openstack at lists.openstack.org>
Subject: Re: [Openstack] nova assign isntances to cpu pinning


On 07 Feb 2017, at 20:05, Steve Gordon <sgordon at redhat.com<mailto:sgordon at redhat.com>> wrote:



----- Original Message -----

From: "Arne Wiebalck" <arne.wiebalck at cern.ch<mailto:arne.wiebalck at cern.ch>>
To: "Steve Gordon" <sgordon at redhat.com<mailto:sgordon at redhat.com>>
Cc: "Manuel Sopena Ballesteros" <manuel.sb at garvan.org.au<mailto:manuel.sb at garvan.org.au>>, openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Sent: Tuesday, February 7, 2017 2:00:23 PM
Subject: Re: [Openstack] nova assign isntances to cpu pinning



On 07 Feb 2017, at 18:57, Steve Gordon <sgordon at redhat.com<mailto:sgordon at redhat.com>> wrote:

----- Original Message -----

From: "Arne Wiebalck" <Arne.Wiebalck at cern.ch<mailto:Arne.Wiebalck at cern.ch>
<mailto:Arne.Wiebalck at cern.ch>>
To: "Manuel Sopena Ballesteros" <manuel.sb at garvan.org.au<mailto:manuel.sb at garvan.org.au>
<mailto:manuel.sb at garvan.org.au>>
Cc: openstack at lists.openstack.org<mailto:openstack at lists.openstack.org> <mailto:openstack at lists.openstack.org>
Sent: Tuesday, February 7, 2017 2:46:39 AM
Subject: Re: [Openstack] nova assign isntances to cpu pinning

Manuel,

Rather than with aggregate metadata we assign instances to NUMA nodes via
flavor extra_specs,

These are not necessarily mutually exclusive, the purpose of the aggregates
in the documentation (and assumed in the original design) is to segregate
compute nodes for dedicated guests (if using truly dedicated CPU by
setting "hw:cpu_policy" to "dedicated" as Chris mentions) from those for
over-committed guests. If you are only using the NUMA node alignment (as
shown below) this doesn't apply, because it's only guaranteeing how many
nodes your guest will be spread across not that it will have dedicated
access to the CPU(s) it is on. Folks who want truly dedicated vCPU:pCPU
mappings should still use the aggregates, unless *only* running workloads
with dedicated CPU needs.

Right: we’re using cells to separate computing-intensive from overcommitted
resources, configured the flavor
extra-specs for NUMA nodes (and huge pages) only in the compute part and have
made good experiences
with this setup.
Depending on the use case and the individual deployment, there are certainly
different options to set things up.
If not already available somewhere, it may be good to document the options
depending on needs and setup?

Cheers,
Arne

A fairly decent amount of it was contributed by some of the Nova folks who worked on this functionality and ended up in the Admin Guide here:

http://docs.openstack.org/admin-guide/compute-adv-config.html

Awesome, thanks for pointing this out!

Cheers,
 Arne



-Steve


i.e. nova flavor-show reports something like

—>
| extra_specs                | {"hw:numa_nodes": "1"} |
<—

for our NUMA-aware flavors.

This seems to work pretty well and gives the desired performance
improvement.

Cheers,
Arne




On 07 Feb 2017, at 01:19, Manuel Sopena Ballesteros
<manuel.sb at garvan.org.au<mailto:manuel.sb at garvan.org.au>> wrote:

Hi,

I am trying to isolate my instances by cpu socket in order to improve my
NUMA hardware performance.

[root at openstack-dev ~(keystone_admin)]# nova aggregate-set-metadata numa
pinned=true
Metadata has been successfully updated for aggregate 1.
+----+--------+--------------------+--------+--------------------+
| Id | Name | Availability Zone | Hosts | Metadata            |
+----+--------+--------------------+--------+--------------------+
| 1  | numa   | -                               |             |
| 'pinned=true'     |
+----+--------+--------------------+--------+--------------------+

I have done the changes on the nova metadata but my admin can’t see the
instances

[root at openstack-dev ~(keystone_admin)]# nova aggregate-add-host numa
4d4f3c3f-2894-4244-b74c-2c479e296ff8
ERROR (NotFound): Compute host 4d4f3c3f-2894-4244-b74c-2c479e296ff8 could
not be found. (HTTP 404) (Request-ID:
req-286985d8-d6ce-429e-b234-dd5eac5ad62e)

And the user who has access to those instances does not have privileges
to
add the hosts

[root at openstack-dev ~(keystone_myuser)]# nova aggregate-add-host numa
4d4f3c3f-2894-4244-b74c-2c479e296ff8
ERROR (Forbidden): Policy doesn't allow
os_compute_api:os-aggregates:index
to be performed. (HTTP 403) (Request-ID:
req-a5687fd4-c00d-4b64-af9e-bd5a82eb99c1)

What would be the recommended way to do this?

Thank you very much

Manuel Sopena Ballesteros | Big data Engineer
Garvan Institute of Medical Research
The Kinghorn Cancer Centre, 370 Victoria Street, Darlinghurst, NSW 2010
T: + 61 (0)2 9355 5760 | F: +61 (0)2 9295 8507 | E:
manuel.sb at garvan.org.au<mailto:manuel.sb at garvan.org.au>

NOTICE
Please consider the environment before printing this email. This message
and any attachments are intended for the addressee named and may contain
legally privileged/confidential/copyright information. If you are not the
intended recipient, you should not read, use, disclose, copy or
distribute
this communication. If you have received this message in error please
notify us at once by return email and then delete both messages. We
accept
no liability for the distribution of viruses or similar in electronic
communications. This notice should not be removed.
_______________________________________________
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

--
Arne Wiebalck
CERN IT

_______________________________________________
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170207/6880075b/attachment.html>