On Thu, 2024-02-29 at 14:26 +0100, Tobias Urdin wrote:

Hello,

We implement something like this ourselves based on internal metrics collections and then handle some parts of the scheduling like you said, basicly reimplementing simple logic for scheduling and then the deciding factor is the weighting based on resource consumption, then basically live-migrate the instances around to accomodate it.

I don’t like that approach either, my hope was that there was or could be in the future a way to ask Placement for allocation candidates beforehand instead of us doing a lot of “guessing”, then applying our weighting after we have some candicates.

this would be a pretty problematic approch. we could do this but it would be racy form the clinets perspecitive. just because the allcoation candiate was valid when you got it does nto mean it will pass nova filter or that it will still be valid later when you actully try to do the migration. so i dont reallly think accpating an allcoation candiate externallysi something nova shoudl do for any api going froward. we may be able to enable your usecase via a differnt method but this would likely not be a good approch.

But I haven’t had time to look into any of that recently.

Best regards Tobias

...

On 29 Feb 2024, at 14:18, Pavlo Shchelokovskyy <pshchelokovskyy@mirantis.com> wrote:

Hi all,

Recently I've been looking into various load optimizations tooling for OpenStack (like Watcher), and noticed a lacking feature that could make such tooling much more robust and useful.

The optimization tooling usually collects some metric about compute hosts and instances that nova may be totally unaware of (e.g. power consumption of a compute node in Watts), and then tries to reshuffle the instances between hypervisors to optimize said metric in some sense. The problem is, the tool has no idea beforehand if nova will actually allow moving this given instance to this given hypervisor. I am talking about resource consumption, resource provider traits, image and aggregate metadata matching, numa placement, server group affinities etc. To know all this for sure, the optimization tool would have to duplicate most of the nova scheduler and good part of placement inside of itself - which is not feasible. This is why AFAIU in Watcher all available strategies mention that they expect "any VM to be migrateable to any hypervisor", which is far from many if not most real life deployments.

What I suggest is adding a new API to nova to effectively have a 'migration dry run' - it should return the list of target hosts a given instance may be migrated to, accounting for all the knowledge nova has about the instance and the hypervisors. The list can be already sorted by nova weighers to signal the preferred choice from nova side.

...

AFAIU such a new API would not require any drastic changes in Nova - the most of the code is already there, used during migrations, only that at the last step, instead of choosing the best compute and actually initiating the migration process, the result of filtering and weighing will be returned to the caller in some form.

we would need 2 api chagnes that could/should be in the same api microversion one for the --dry-run to just return the

This might be possible. its not as invaisive as it first appears as the schduler RPC api already has a select_host or simialr function that the conductor uses to get a weighed and filetered list of host and then it calims the allocation candiate in placment. so the nova rest api could perhaps call the same rpc api and just return the list of hosts. if we also enhanced the live migration api to asspcate a list of host instead of a single host you could then rewight the list yourslef based on the metics you have. list of possibel hosts and the second change is taking a list of possibel host on the normal migration call. if you want to only choose one host after your weighing then it woudl not require the second api change. that siad i really do think a list is likely better for watchers usecase. nova when it recievs the host or list of host woudl still call placment and all the filters again but we could make the schduler skip the weighers if a list of host was provided to preserve the order that was specifed in the list.

...

Would there be interest in such an API? If yes, I could start writing and proposing a spec.

my initall reaction to the title was hell no as i assuemd you wanted ot od more then just get a set of canidate hosts applying our filters and weighers to them. a more useful approch may be to reframe this as follows.

""" As an admin, i would like to be able to query for a set of valid host to migrate an instnace too without starting the migration. As a user with the project manager role, i have the capablity to specify a host to boot too but i dont know what host are available to me, i would like to have an api to list the possible host i can spwan an instance on based on a flavor and image. """ so instead of adding dry-run to the migration api we could add a new api "candidate-hosts" that can take either an instance by uuid or a flavor and image pair and jsut return the hosts that the schduler would return to the conductor when we call select_destionations we have not implmented project manager as part of our SRBAC policies yet in nova but i know gmann is interested in the project manager boot to host usecase and the stubeling block has historically been how to let the project admin know what hosts they can use without giveing them access to the hypervisor api. a "candidate-hosts" api would solve both problems in a relatively clean manner. so i would not be oppsed to a spec around that general idea. regards sean.

...

Best regards, -- Dr. Pavlo Shchelokovskyy Principal Software Engineer Mirantis Inc www.mirantis.com <http://www.mirantis.com/>