Open Stack

On Mon, 2019-11-11 at 11:58 +0000, Wang, Huaqiang wrote:
> > -----Original Message-----
> > From: Balázs Gibizer <balazs.gibizer at est.tech>
> > Sent: Friday, November 8, 2019 3:10 PM
> > To: openstack-discuss <openstack-discuss at lists.openstack.org>
> > Subject: [nova][ptg] pinned and unpinned CPUs in one instance
> > 
> > spec: https://review.opendev.org/668656
> > 
> > Agreements from the PTG:
> > 
> > How we will test it:
> > * do functional test with libvirt driver, like the pinned cpu tests we have
> > today
> > * donyd's CI supports nested virt so we can do pinned cpu testing but not
> > realtime. As this CI is still work in progress we should not block on this.
> > * coverage inhttps://opendev.org/x/whitebox-tempest-pluginis a nice to
> > have
> > 
> > Naming: use the 'shared' and 'dedicated' terminology
> > 
> > Support both the hw:pinvcpus=3 and the resources:PCPU=2 flavor extra
> > specs syntaxtbut not in the same flavor. The resources:PCPU=2 syntax will
> > have less expression power until nova models NUMA in placement. So nova
> > will try to evenly distribute PCPUs between numa nodes. If it not possible we
> > reject the request and ask the user to use the
> > hw:pinvcpus=3 syntax.
> > 
> > Realtime mask is an exclusion mask, any vcpus not listed there has to be in
> > the dedicated set of the instance.
> > 
> > TODOInvestigate whether we want to enable NUMA by default
> > * Pros: Simpler, everything is NUMA by default
> > * Cons: We'll either have to break/make configurablethe 1:1 guest:host
> > NUMA mapping else we won't be able to boot e.g. a 40 core shared instance
> > on a 40 core, 2 NUMA node host
> 
> For the case of 'booting a 40 core shared instance on 40 core 2NUMA node' that will
> not be covered by the new 'mixed' policy. It is just a legacy 'shared' instance with no
> assumption about instance NUMA topology. 

Correct. However, this investigation refers to *all* instances, not
just those using the 'mixed' policy. For the 'mixed' policy, I assume
we'll need to apply a virtual NUMA topology since we currently apply
one for instances using the 'dedicated' policy.

> By the way if you want a 'shared' instance, with 40 cores, to be scheduled on a host
> of 40cores, 2 NUMA nodes, you also need to register all host cores as 'shared' cpus
> through 'conf.compute.cpu_shared_set'. 
> 
> For instance with 'mixed' policy, what I want to propose is the instance should
> demand at least one 'dedicated'(or PCPU) core. Thus, any 'mixed' instance or 'dedicated'
> instance will not be scheduled one this host due to no PCPU available on this host.
> 
> And also, a 'mixed' instance should also demand at least one 'shared' (or VCPU) core.
> a 'mixed' instance demanding all cores from PCPU resource should be considered as
> an invalid one. And an instance demanding all cores from PCPU resource is just a
> legacy 'dedicated' instance, which CPU allocation policy is 'dedicated'.
> 
> In conclusion,  a instance with the policy of 'mixed' 
> -. demands at least one 'dedicated' cpu and at least one 'shared' cpu. 
> -. with NUMA topology by default due to requesting pinned cpu
> 
> In my understanding the cons does not exist by making above rules. 
> 
> Br
> Huaqiang
> 
> > 
> > Cheers,
> > gibi