Open Stack

----- Original Message -----
> From: "Arne Wiebalck" <arne.wiebalck at cern.ch>
> To: "Steve Gordon" <sgordon at redhat.com>
> Cc: "Manuel Sopena Ballesteros" <manuel.sb at garvan.org.au>, openstack at lists.openstack.org
> Sent: Tuesday, February 7, 2017 2:00:23 PM
> Subject: Re: [Openstack] nova assign isntances to cpu pinning
> 
> 
> > On 07 Feb 2017, at 18:57, Steve Gordon <sgordon at redhat.com> wrote:
> > 
> > ----- Original Message -----
> >> From: "Arne Wiebalck" <Arne.Wiebalck at cern.ch
> >> <mailto:Arne.Wiebalck at cern.ch>>
> >> To: "Manuel Sopena Ballesteros" <manuel.sb at garvan.org.au
> >> <mailto:manuel.sb at garvan.org.au>>
> >> Cc: openstack at lists.openstack.org <mailto:openstack at lists.openstack.org>
> >> Sent: Tuesday, February 7, 2017 2:46:39 AM
> >> Subject: Re: [Openstack] nova assign isntances to cpu pinning
> >> 
> >> Manuel,
> >> 
> >> Rather than with aggregate metadata we assign instances to NUMA nodes via
> >> flavor extra_specs,
> > 
> > These are not necessarily mutually exclusive, the purpose of the aggregates
> > in the documentation (and assumed in the original design) is to segregate
> > compute nodes for dedicated guests (if using truly dedicated CPU by
> > setting "hw:cpu_policy" to "dedicated" as Chris mentions) from those for
> > over-committed guests. If you are only using the NUMA node alignment (as
> > shown below) this doesn't apply, because it's only guaranteeing how many
> > nodes your guest will be spread across not that it will have dedicated
> > access to the CPU(s) it is on. Folks who want truly dedicated vCPU:pCPU
> > mappings should still use the aggregates, unless *only* running workloads
> > with dedicated CPU needs.
> 
> Right: we’re using cells to separate computing-intensive from overcommitted
> resources, configured the flavor
> extra-specs for NUMA nodes (and huge pages) only in the compute part and have
> made good experiences
> with this setup.
> Depending on the use case and the individual deployment, there are certainly
> different options to set things up.
> If not already available somewhere, it may be good to document the options
> depending on needs and setup?
> 
> Cheers,
>  Arne

A fairly decent amount of it was contributed by some of the Nova folks who worked on this functionality and ended up in the Admin Guide here:

http://docs.openstack.org/admin-guide/compute-adv-config.html

Thanks,

Steve

> > -Steve
> > 
> >> i.e. nova flavor-show reports something like
> >> 
> >> —>
> >> | extra_specs                | {"hw:numa_nodes": "1"} |
> >> <—
> >> 
> >> for our NUMA-aware flavors.
> >> 
> >> This seems to work pretty well and gives the desired performance
> >> improvement.
> >> 
> >> Cheers,
> >> Arne
> >> 
> >> 
> >> 
> >>> On 07 Feb 2017, at 01:19, Manuel Sopena Ballesteros
> >>> <manuel.sb at garvan.org.au> wrote:
> >>> 
> >>> Hi,
> >>> 
> >>> I am trying to isolate my instances by cpu socket in order to improve my
> >>> NUMA hardware performance.
> >>> 
> >>> [root at openstack-dev ~(keystone_admin)]# nova aggregate-set-metadata numa
> >>> pinned=true
> >>> Metadata has been successfully updated for aggregate 1.
> >>> +----+--------+--------------------+--------+--------------------+
> >>> | Id | Name | Availability Zone | Hosts | Metadata            |
> >>> +----+--------+--------------------+--------+--------------------+
> >>> | 1  | numa   | -                               |             |
> >>> | 'pinned=true'     |
> >>> +----+--------+--------------------+--------+--------------------+
> >>> 
> >>> I have done the changes on the nova metadata but my admin can’t see the
> >>> instances
> >>> 
> >>> [root at openstack-dev ~(keystone_admin)]# nova aggregate-add-host numa
> >>> 4d4f3c3f-2894-4244-b74c-2c479e296ff8
> >>> ERROR (NotFound): Compute host 4d4f3c3f-2894-4244-b74c-2c479e296ff8 could
> >>> not be found. (HTTP 404) (Request-ID:
> >>> req-286985d8-d6ce-429e-b234-dd5eac5ad62e)
> >>> 
> >>> And the user who has access to those instances does not have privileges
> >>> to
> >>> add the hosts
> >>> 
> >>> [root at openstack-dev ~(keystone_myuser)]# nova aggregate-add-host numa
> >>> 4d4f3c3f-2894-4244-b74c-2c479e296ff8
> >>> ERROR (Forbidden): Policy doesn't allow
> >>> os_compute_api:os-aggregates:index
> >>> to be performed. (HTTP 403) (Request-ID:
> >>> req-a5687fd4-c00d-4b64-af9e-bd5a82eb99c1)
> >>> 
> >>> What would be the recommended way to do this?
> >>> 
> >>> Thank you very much
> >>> 
> >>> Manuel Sopena Ballesteros | Big data Engineer
> >>> Garvan Institute of Medical Research
> >>> The Kinghorn Cancer Centre, 370 Victoria Street, Darlinghurst, NSW 2010
> >>> T: + 61 (0)2 9355 5760 | F: +61 (0)2 9295 8507 | E:
> >>> manuel.sb at garvan.org.au
> >>> 
> >>> NOTICE
> >>> Please consider the environment before printing this email. This message
> >>> and any attachments are intended for the addressee named and may contain
> >>> legally privileged/confidential/copyright information. If you are not the
> >>> intended recipient, you should not read, use, disclose, copy or
> >>> distribute
> >>> this communication. If you have received this message in error please
> >>> notify us at once by return email and then delete both messages. We
> >>> accept
> >>> no liability for the distribution of viruses or similar in electronic
> >>> communications. This notice should not be removed.
> >>> _______________________________________________
> >>> Mailing list:
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> >>> Post to     : openstack at lists.openstack.org
> >>> Unsubscribe :
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> >> 
> >> --
> >> Arne Wiebalck
> >> CERN IT
> >> 
> >> _______________________________________________
> >> Mailing list:
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> >> Post to     : openstack at lists.openstack.org
> >> Unsubscribe :
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> >>