Thanks Melanie! It helps. VMs are spread very well. Tony
-----Original Message----- From: melanie witt <melwittt@gmail.com> Sent: Tuesday, October 27, 2020 2:25 PM To: Tony Liu <tonyliu0592@hotmail.com>; openstack- discuss@lists.openstack.org Subject: Re: [nova-scheduler] always the same compute was selected?
On 10/27/20 14:13, Tony Liu wrote:
Hi,
I have a Ussuri setup with 5 compute nodes (same resource spec). When launch 5 Cirros VMs in a batch, they are all launched on the same compute node. All 5 nodes have sufficient resource.
AWIK, Nova scheduler picks a compute randomly after going through filtering. But 5 VMs all on the same compute is quite coincidental. Could it be something missing? Is there any control to such random pick? Any way to make a better balancing?
Nova scheduler will weigh all of the candidate hosts via the weighers configured and then by default, it will pick randomly from a set of only 1 [1]. This will result in a "packing" behavior if the weighers are not sufficiently configured to result in "spreading" instances (example: cpu/ram/disk weighers).
To make the scheduler pick randomly from a larger subset of candidate compute hosts, increase the host_subset_size [1].
To make the scheduler randomly shuffle candidate hosts when weighting is the same, set shuffle_best_same_weighed_hosts to True [2].
If you have > 1000 compute hosts (via multi-cell) and need to randomize the default 1000 results that come from placement [3], set randomize_allocation_candidates to True.
You likely will only be interested in the first two options I described. Tune host_subset_size until you get the desired scheduling behavior.
Hope this helps, -melanie
[1] https://docs.openstack.org/nova/ussuri/configuration/config.html#filter_ scheduler.host_subset_size [2] https://docs.openstack.org/nova/ussuri/configuration/config.html#filter_ scheduler.shuffle_best_same_weighed_hosts [3] https://docs.openstack.org/nova/ussuri/configuration/config.html#schedul er.max_placement_results [4] https://docs.openstack.org/placement/ussuri/configuration/config.html#pl acement.randomize_allocation_candidates