We are running placement under apache:
https://paste.fedoraproject.org/paste/mZviLVe5xONPsXfLqdxI6A
The placement error logs show a lot of GETs but no errors:
https://paste.fedoraproject.org/paste/xDVGaXEdoQ5Z3wHv17Lezg
We are planning to use NUMA but haven't started yet. It's probably a config error. Where should I be looking? This is our nova config on the controllers:
https://paste.fedoraproject.org/paste/kNe1eRimk4ifrAuuN790bg
-----Original Message----- From: Sean Mooney smooney@redhat.com Sent: Tuesday, November 12, 2019 12:22 PM To: Albert Braden albertb@synopsys.com; openstack-discuss@lists.openstack.org Subject: Re: Scheduler sends VM to HV that lacks resources
On Tue, 2019-11-12 at 19:42 +0000, Albert Braden wrote:
If I create 20 VMs at once, at least one of them fails with "Exceeded maximum number of retries." When I look at the logs I see that the scheduler sent the VM to a host that doesn't have enough CPU "Free vcpu 14.00 VCPU < requested 16 VCPU."
https://urldefense.proofpoint.com/v2/url?u=https-3A__paste.fedoraproject.org...
I thought that this must be caused by a race condition, so I stopped the scheduler and conductor on 2 controllers, and then created 20 more VMs. Now I see the logs only on controller 3, and some of the failures are now saying "Unable to establish connection to <LB>" but I still see the single scheduler sending VMs to a host that lacks resources "Free vcpu 14.00 VCPU < requested 16 VCPU."
https://urldefense.proofpoint.com/v2/url?u=https-3A__paste.fedoraproject.org...
I'm looking at my nova.conf but don't see anything misconfigured. My filters are pretty standard:
enabled_filters = RetryFilter,AvailabilityZoneFilter,CoreFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter, ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,DifferentHostFilter,SameHostFilter
What should I be looking for here? Why would a single scheduler send a VM to a host that is too full? We have lots of compute hosts that are not full:
https://urldefense.proofpoint.com/v2/url?u=https-3A__paste.fedoraproject.org...
This is the command line I used:
openstack server create --flavor s1.16cx120g --image QSC-P-CentOS6.6-19P1-v4 --network vg-network --max 20 alberttestB
what version of openstack are you running? if its not using placement then this behaviour is expected as the resources are not claimed untill the vm is booted on the node so there is and interval where the scudler is selecting hosts where you can race with other vm boot.
if you are using placement and you are not using numa or pci pass though, which you do not appear to be based on your enabled filters, then this should not happen and we should dig deeper as there is likely a bug either in your configuration or in nova.