<div dir="ltr">Could be nice to have that metric exposed inside the API for nova-hypervisors.<div><br></div><div>We scrape those with Prometheus and an exporter so we could have a bit more visibility. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 16, 2022 at 1:11 PM Franck VEDEL <<a href="mailto:franck.vedel@univ-grenoble-alpes.fr">franck.vedel@univ-grenoble-alpes.fr</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;"><div><blockquote type="cite">yes this is logged in the schduler at debug level</blockquote><br></div><div>is it this ?</div><div><br></div><div>2022-02-16 10:18:26.802 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.build_failure_weight_multiplier = 1000000.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.802 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.cpu_weight_multiplier = 1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.802 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.802 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.disk_weight_multiplier = 1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.803 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.io_ops_weight_multiplier = -1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.804 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.pci_weight_multiplier = 1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.804 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.ram_weight_multiplier = 1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.805 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.soft_affinity_weight_multiplier = 1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.805 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.805 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.806 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] metrics.weight_multiplier = 1.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.806 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] metrics.weight_of_unavailable = -10000.0 log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div>2022-02-16 10:18:26.806 8 DEBUG oslo_service.service [req-629e8eaf-9e0e-471a-b99c-957459b6c9af - - - - -] metrics.weight_setting = [] log_opt_values /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_config/cfg.py:2611</div><div><br></div><div><br></div><div>
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div><span style="color:rgb(31,73,125);font-family:Arial,sans-serif;font-size:10pt">Franck </span></div></div></div><div style="font-family:Calibri,sans-serif;font-size:11pt;margin:0cm 0cm 0.0001pt"></div></div></div>
</div>
<div><br><blockquote type="cite"><div>Le 16 févr. 2022 à 13:45, Sean Mooney <<a href="mailto:smooney@redhat.com" target="_blank">smooney@redhat.com</a>> a écrit :</div><br><div><div>On Wed, 2022-02-16 at 10:52 +0100, Franck VEDEL wrote:<br><blockquote type="cite">Thank’s a lot ! <br>I changed the settings and indeed, it seems to work. This distribution of instances is really interesting. I learn a lot.<br>Question: is it possible to view the calculated weight when choosing a server?<br>Otherwise, thanks again, really<br></blockquote>yes this is logged in the schduler at debug level<br><blockquote type="cite"><br>Franck<br><br><blockquote type="cite">Le 16 févr. 2022 à 03:35, Tony Liu <<a href="mailto:tonyliu0592@hotmail.com" target="_blank">tonyliu0592@hotmail.com</a>> a écrit :<br><br>Build failure could be caused by different things, networking, storage, hypervisor, etc.<br>For example, failure caused by Neutron service, that doesn't mean this hypervisor is<br>not healthy, but because of that weigher, even Neutron service is recovered, this<br>hypervisor is still excluded from holding instance. This doesn't make sense.<br>I wouldn't enable this weigher until it's smart enough to know the failure is caused<br>by hypervisor itself, but not anywhere else.<br><br>Tony<br>________________________________________<br>From: Laurent Dumont <<a href="mailto:laurentfdumont@gmail.com" target="_blank">laurentfdumont@gmail.com</a>><br>Sent: February 15, 2022 05:00 PM<br>To: Tony Liu<br>Cc: Franck VEDEL; openstack-discuss<br>Subject: Re: [kolla-ansible][nova]Problem with distribution of instance on servers<br><br>In a healthy setup, should build_failure_weight_multiplier be triggered?<br><br>From the doc, tweaking this might mean you try to schedule and built instances on computes that are not healthy.<br><br>On Tue, Feb 15, 2022 at 6:38 PM Tony Liu <<a href="mailto:tonyliu0592@hotmail.com" target="_blank">tonyliu0592@hotmail.com</a><<a href="mailto:tonyliu0592@hotmail.com" target="_blank">mailto:tonyliu0592@hotmail.com</a>>> wrote:<br>Enable debug logging on nova-scheduler, you will see how the winner is picked.<br>I had the same issue before, caused by the build-failure weigher enabled by default.<br>setting build_failure_weight_multiplier to 0 resolved issue for me. Instances are<br>balanced by weighers (compute and memory) as expected.<br>shuffle_best_same_weighed_hosts and host_subset_size are not necessary, unless<br>it's required by certain cases.<br><br>Tony<br>________________________________________<br>From: Laurent Dumont <<a href="mailto:laurentfdumont@gmail.com" target="_blank">laurentfdumont@gmail.com</a><<a href="mailto:laurentfdumont@gmail.com" target="_blank">mailto:laurentfdumont@gmail.com</a>>><br>Sent: February 15, 2022 12:54 PM<br>To: Franck VEDEL<br>Cc: openstack-discuss<br>Subject: Re: [kolla-ansible][nova]Problem with distribution of instance on servers<br><br>There are two settings we've tweaked in the past in Nova.<br><br>shuffle_best_same_weighed_hosts --> Allow more spreading in the case of computes with the exact same specs/weights.<br>host_subset_size --> Helps with concurrent requests to get different hosts<br><br>Before that, we saw the same behavior with Openstack stacking VM on single computes. It still respects anti-affinity, but I don't see a good reason to not spread as a default. Changing these two was enough to allow our spread to get a little better.<br><br>On Tue, Feb 15, 2022 at 11:19 AM Franck VEDEL <<a href="mailto:franck.vedel@univ-grenoble-alpes.fr" target="_blank">franck.vedel@univ-grenoble-alpes.fr</a><<a href="mailto:franck.vedel@univ-grenoble-alpes.fr" target="_blank">mailto:franck.vedel@univ-grenoble-alpes.fr</a>><<a href="mailto:franck.vedel@univ-grenoble-alpes.fr" target="_blank">mailto:franck.vedel@univ-grenoble-alpes.fr</a><<a href="mailto:franck.vedel@univ-grenoble-alpes.fr" target="_blank">mailto:franck.vedel@univ-grenoble-alpes.fr</a>>>> wrote:<br>Hello,<br>I seem to have a problem that I hadn't seen.<br>I have 3 servers for my openstack, built with Kolla-ansible, I'm in Victoria version.<br>I had simply put the 3 servers in the [compute] part of the multinode file, at first it worked, but for some time all the VMs are placed on server 1.<br><br>The 3 servers are operational, identical. here are 3 screenshots to show it. (on the images, the instances on servers 2 and 3 are present because it worked correctly, but no more instances are created on these servers now)<br>[<a>cid:17eff2778356f37a4481</a>]<br>[<a>cid:17eff277835e47aa83c2</a>]<br>[<a>cid:17eff2778356f53d34a3</a>]<br><br><br>I tried to understand how the instances are distributed on the servers, but in my case, I don't understand why none are assigned to the 2nd and 3rd server.<br>How to find the problem? It should be nova-scheduler . Do you have to do anything special? Go see if a parameter has a bad value?<br><br><br>Thanks in advance if you can help me.<br><br>Franck VEDEL<br><br></blockquote><br></blockquote><br></div></div></blockquote></div><br></div></blockquote></div>