Open Stack

On Mon, 2023-09-04 at 18:13 -0300, Jorge Visentini wrote:
> Yes, yes, that I understand.
> 
> I know that for example if I want to use host_passthrough then I must use
> cpu_sockets == numa_nodes.
no even without cpu_mode=host_passthough  you shoudl use cpu_sockets == numa_nodes.
The reason for that is that the windows and to a lesser degree linux scheduler
tends to make better schduling desicions when the numa nodes and sockets are aligned.

intel supported cluster on die since around sandybridge/ivybridge but it was not until
very recentlly with amd epic platform and the move to chiplet design that the windows
and linux kernels schdulers really got optimised to properly handel multipel numa nodes
per socket.

in the very early days fo openstack qemu/libvirt strongly prefer generating toplogies with 1
socket, 1 core and 1 thread per vcpu. that is still what we defautl to today in nova because
10+ years ago that outperformed other toplogies in some test done in a non openstack env i.e. with just libvirt/qemu

when i got involed in openstack 10 years ago the cracks were already forming in that analasy and wehn we
started to look at addign cpu pinnng and numa support the initall benchmarks we did show no benift to 1 socket per
vcpu and infact if you had hyperthread on the host it actully perfromed worse.

for legacy reason we did not change the defautl toplogy but our recomendation has been to 
1 socket per numa node and 1 thread per host thread by default 

so 8 VCPU with a normal floatign vm with no special numa toplogy shoudl be 

1 socket 2 thread and 4 cpus if the host has smt/hyperthreading enable or 
1 socket 1 threadn and 8 cpus. not 8 sockets,  1 thread and 1 cpu which is the defualt toplogy.

> *My question is more conceptual, for me to understand.*
no worries
> 
> *For example:* If I have a physical host with 1 physical processor (1
> socket), can I define my instance with 2, 4, 8+ sockets? I mean, is it good
> practice? Is it correct to define the instance with 1 socket and increase
> the amount of socket colors?

you do not need to align the virtual toplogy to the hsot toplogy in anyway.

so on a host with 1 socket and 1 thread per core and 64 cores
you can create a vm with 16 sockets and 2 thread per core and 1 core per socket

largely this wont impact performace vs
1 sockets and 1 thread per core and 32 core per socket

which woudl be our recomended toplogy.

what you are trying to do with the virtual toplogy is create a config that enabel the guest
kernel schduler to make good choices

historically kernel schduler were largely not numa aware but did consider moving a process
between socket to be very costly and as a result it avoided it.
that meant for old kernels (windows 2012 or linux (centos 6/linux 2.x era)) it was more
effeicnt to have 1 socket per vcpu

that change in the linux side many year ago adn windows a lot more recently.
now its more imporant to the scudherl to knwo about the numa toplogy and thread
topopligy. i.e. in the old model fo 1 socket per vcpu if you had hyperthreading enabld
the guest kernel would expect that it can run n process in parralle when infact
it really can only run n/2 so match the theread count to the host tread count allow
the guest kernel to make better choices.

setting thread=2 when the host has thread=1 also tends to have littel downside for what its worth.

for what its worth on the linux side its my understanding that the kernel also does som extra work per socket
that can be elimated if you use 1 socket instead of 32 but that is negligible unless your dealing
with realtime workloads where it might matter.


> 
> Not sure if I could explain my question... In short, what is the
> relationship between the socket, cores and virtual thread and the socket,
> cores and physical thread.
> PS: I'm not into the issue of passthrough or not.

yep so conceptually as a user o fopenstack you should have no awareness of the hsot platform.
specirfcaly as an unprivladged user you are not even ment to know if its libvirt/kvm or vmware
from an api prespective.

form an openstack admin point of view its in yoru interest to craft your flavors and default
images to be as power efficent as possible. the best way to acihve power effeicncy is often
to make the workload perform better so that it can idle sooner and you can take one
step in that direction by using some of your knowlage of the hardware to turn your images.

as an end user because you are not ment to knwo the toplogy of the host systems you
generally shoudl benchmark your workload but you shoudl not really expect a large delta
regardless of what you choose today.


openstack is not a virtualisation plathform, its a cloud plathform and while we supprot
some enhanced paltform awareness features like cpu pinning this si largely intened to be
someing the cloud admin configured and the end user just selects rather then something
an end user should have to deeply understand.

so tl;dr virtual cpu tpopligies are about optimisting the vm for the workload not the host.
optimising for the host can give a minimal performance uplift for the vm but the virutal
cpu toplogoy is intentionally decloupeld form the host toplogy.

the virtual numa topology feature is purely a performance enhancement feature
and is implictly tied to a host. ie. if you ask for 2 numa nodes we will pin
each guest numa node to a seperate host numa node. virtual numa topligecs
and cpu toplogies had two entrily diffent design goals. numa is tied to the
host toplogy to optimise for hardware constraits, cpu topliges are not tied
to the host hardware as they are optimising for guest software constraits 
(i.e. windows server only support 4 sockets so if you need more the 4 cpus you cannot
have 1 socket pre vcpu)

> 
> Em seg., 4 de set. de 2023 às 13:10, <smooney at redhat.com> escreveu:
> 
> > in general the only parameter you want to align to the physical host is
> > the number of thread
> > 
> > so if the phsyical host has 2 thread per physical core then its best to
> > also do that in the vm
> > 
> > we generally recommend setting the number of virutal socket equal to the
> > number of virutal numa nodes
> > if the vm has no explict numa toploly then you should set sockets=1
> > else hw:cpu_sockets==hw:numa_nodes is our recomendation.
> > 
> > for windows in partcalar the default config generated is suboptimal as
> > windows client only supprot 1-2 sockets
> > and windows serverver maxes out at 4 i believe.
> > 
> > On Mon, 2023-09-04 at 11:51 -0300, Jorge Visentini wrote:
> > > Hello Team,
> > > 
> > > What is the difference between creating an instance with *4* or *8
> > virtual
> > > sockets*, since the hypervisor has only *4 physical sockets*.
> > > My question is where do sockets, cores and virtual threads fit into the
> > > physical hardware. I think this question is not just related to
> > Openstack,
> > > but with any virtualization.
> > > 
> > > My hypervisor configuration is as follows:
> > > 
> > > CPU(s): 192
> > > Online CPU(s) list: 0-191
> > > Thread(s) per core: 2
> > > Core(s) per socket: 24
> > > Socket(s): 4
> > > NUMA node(s): 4
> > > 
> > > Do you have any documentation that I can read and understand better?
> > > 
> > > That we have a nice week!
> > 
> > 
>