<html><head></head><body><div class="ydp8295c74fyahoo-style-wrap" style="font-family: times new roman, new york, times, serif; font-size: 16px;"><div></div>
<div dir="ltr" data-setdir="false">I am running stable Queens with hundreds of ironic baremetal nodes.<br></div><div dir="ltr" data-setdir="false">Things are mostly stable but occasionally some baremetal node provisions are failing. These failures have been tracked to </div><div dir="ltr" data-setdir="false">nova placement failure leading to 409 errors.</div><div dir="ltr" data-setdir="false">My nova and baremetal filters do NOT have the 3 filters you mention.</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><div><div dir="ltr" data-setdir="false">[root@sc-control03 objects]# grep filter /etc/nova/nova.conf | grep filters<br></div><div dir="ltr" data-setdir="false"><div># * enabled_filters<br></div><div>#enabled_filters=RetryFilter,AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter</div><div dir="ltr" data-setdir="false"><div><div>#use_baremetal_filters=false</div><div>#baremetal_enabled_filters=RetryFilter,AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ExactRamFilter,ExactDiskFilter,ExactCoreFilter<br></div></div></div><div><br></div><div><br style="color: rgb(0, 0, 0); font-family: times new roman, new york, times, serif; font-size: 16px;"></div></div></div><br></div><div dir="ltr" data-setdir="false">The baremetal nodes are all using resource class. My image does NOT have the changes for<br></div><div dir="ltr" data-setdir="false"><a href="https://review.opendev.org/#/c/565841" rel="nofollow" target="_blank" class="">https://review.opendev.org/#/c/565841</a><br></div><div><br></div><div><br></div><div><br></div><div dir="ltr" data-setdir="false">Ultimately, nova-conductor is reported "<span><span style="color: rgb(0, 0, 0); font-family: times new roman, new york, times, serif; font-size: 16px;">NoValidHost: No valid host was found. There are not enough hosts available"</span></span></div><div dir="ltr" data-setdir="false">This has been traced to nova-placement-api "<span><span style="color: rgb(0, 0, 0); font-family: times new roman, new york, times, serif; font-size: 16px;">Allocation for CUSTOM_RRR430 on resource provider 3cacac3f-9af0-4e39-9bc8-d1f362bdb730 violates min_unit, max_unit, or step_size. Requested: 2, min_unit: 1, max_unit: 1, step_size: 1"</span></span></div><div><br></div><div dir="ltr" data-setdir="false">Any pointers on what next steps I should be looking at ?</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">thanks,</div><div dir="ltr" data-setdir="false">Fred.</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Relevant logs: </div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">nova-conductor.log</div><div dir="ltr" data-setdir="false"><div><div>2019-11-12 10:26:02.593 1666486 ERROR nova.conductor.manager [req-fa1bfb2e-c765-432d-aa66-e16db8329312 - - - - -] Failed to schedule instances: NoValidHost_Remote: No valid host was found. There are not enough hosts available.</div><div>Traceback (most recent call last):</div><div><br></div><div> File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner</div><div> return func(*args, **kwargs)</div><div><br></div><div> File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 154, in select_destinations</div><div> allocation_request_version, return_alternates)</div><div><br></div><div> File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 91, in select_destinations</div><div> allocation_request_version, return_alternates)</div><div><br></div><div> File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 243, in _schedule</div><div> claimed_instance_uuids)</div><div><br></div><div> File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 280, in _ensure_sufficient_hosts</div><div> raise exception.NoValidHost(reason=reason)</div><div><br></div><div>NoValidHost: No valid host was found. There are not enough hosts available.</div><div><br></div></div><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">nova-placement-api.log <br></div><div dir="ltr" data-setdir="false"><span><span style="color: rgb(0, 0, 0); font-family: times new roman, new york, times, serif; font-size: 16px;">3cacac3f-9af0-4e39-9bc8-d1f362bdb730 = resource ID of baremetal node</span></span><br></div><div dir="ltr" data-setdir="false"><span><span style="color: rgb(0, 0, 0); font-family: times new roman, new york, times, serif; font-size: 16px;"><span><span style="color: rgb(0, 0, 0); font-family: times new roman, new york, times, serif; font-size: 16px;">84ea2b90-06b2-489e-92ea-24b859b3c997 = instance ID</span></span><br></span></span></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">2019-11-12 10:26:02.427 4161131 INFO nova.api.openstack.placement.requestlog [req-66a6dc45-8326-4e24-9216-fc77099303ba 1ee9f9bf77294e8e8bf50bb35c581689 acf8cd411e5e4751a61d1ed54e8e874d - default default] 10.33.24.13 "GET /allocations/84ea2b90-06b2-489e-92ea-24b859b3c997" status: 200 len: 111 microversion: 1.0<br></div><div dir="ltr" data-setdir="false"><div><div><br></div><div>2019-11-12 10:26:02.461 4161129 WARNING nova.objects.resource_provider [req-6d79841e-6abe-490e-b79b-8d88b04215af 1ee9f9bf77294e8e8bf50bb35c581689 </div><div>acf8cd411e5e4751a61d1ed54e8e874d - default default] Allocation for CUSTOM_Z370_A on resource provider 3cacac3f-9af0-4e39-9bc8-d1f362bdb730 violates min_unit, max_unit, or step_size. Requested: 2, min_unit: 1, max_unit: 1, step_size: 1</div><div><br></div><div>2019-11-12 10:26:02.568 4161129 INFO nova.api.openstack.placement.requestlog [req-6d79841e-6abe-490e-b79b-8d88b04215af 1ee9f9bf77294e8e8bf50bb35c581689 acf8cd411e5e4751a61d1ed54e8e874d - default default] 10.33.24.13 "PUT /allocations/84ea2b90-06b2-489e-92ea-24b859b3c997" status: 409 len: 383 microversion: 1.17</div><div><br></div></div><div><br></div><div dir="ltr" data-setdir="false">http_access_log</div><div dir="ltr" data-setdir="false"><div><div>10.33.24.13 - - [12/Nov/2019:10:26:02 -0800] "GET /allocations/84ea2b90-06b2-489e-92ea-24b859b3c997 HTTP/1.1" 200 111 "-" "nova-scheduler keystoneauth1/3.4.0 python-requests/2.14.2 CPython/2.7.5"</div><div>10.33.24.13 - - [12/Nov/2019:10:26:02 -0800] "PUT /allocations/84ea2b90-06b2-489e-92ea-24b859b3c997 HTTP/1.1" 409 383 "-" "nova-scheduler keystoneauth1/3.4.0 python-requests/2.14.2 CPython/2.7.5"</div><div><br></div></div><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><br></div><div><br></div><div><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><div><div dir="ltr" data-setdir="false"><br></div></div><br></div><br></div><div dir="ltr" data-setdir="false"><br></div><div><br></div>
</div><div id="ydp59fdfe32yahoo_quoted_4086327416" class="ydp59fdfe32yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Wednesday, November 13, 2019, 11:36:35 AM PST, Albert Braden <albert.braden@synopsys.com> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div dir="ltr">Removing these 3 obsolete filters appears to have fixed the problem. Thank you for your advice!<br clear="none"><br clear="none">-----Original Message-----<br clear="none">From: Matt Riedemann <<a shape="rect" href="mailto:mriedemos@gmail.com" rel="nofollow" target="_blank">mriedemos@gmail.com</a>> <br clear="none">Sent: Tuesday, November 12, 2019 1:14 PM<br clear="none">To: <a shape="rect" href="mailto:openstack-discuss@lists.openstack.org" rel="nofollow" target="_blank">openstack-discuss@lists.openstack.org</a><br clear="none">Subject: Re: Scheduler sends VM to HV that lacks resources<br clear="none"><br clear="none">On 11/12/2019 2:47 PM, Albert Braden wrote:<br clear="none">> It's probably a config error. Where should I be looking? This is our nova config on the controllers:<br clear="none">> <br clear="none">> <a shape="rect" href="https://urldefense.proofpoint.com/v2/url?u=https-3A__paste.fedoraproject.org_paste_kNe1eRimk4ifrAuuN790bg&d=DwICaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=TZI4wT8_y-RAnwbbXaWBhdvAhhcbY1qymxKLRVpPt2U&s=3aQNqwtEMfOC7U_QUTqNqXiZv4yJy6ceB4kCuZKuL0o&e= " rel="nofollow" target="_blank">https://urldefense.proofpoint.com/v2/url?u=https-3A__paste.fedoraproject.org_paste_kNe1eRimk4ifrAuuN790bg&d=DwICaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=TZI4wT8_y-RAnwbbXaWBhdvAhhcbY1qymxKLRVpPt2U&s=3aQNqwtEMfOC7U_QUTqNqXiZv4yJy6ceB4kCuZKuL0o&e= </a><div class="ydp59fdfe32yqt4433243924" id="ydp59fdfe32yqtfd58035"><br clear="none"><br clear="none">If your deployment is pike or newer (I'm guessing rocky because your <br clear="none">other email says rocky), then you don't need these filters:<br clear="none"><br clear="none">RetryFilter - alternate hosts bp in queens release makes this moot<br clear="none">CoreFilter - placement filters on VCPU<br clear="none">RamFilter - placement filters on MEMORY_MB<br clear="none"><br clear="none">-- <br clear="none"><br clear="none">Thanks,<br clear="none"><br clear="none">Matt<br clear="none"><br clear="none"></div></div></div>
</div>
</div></body></html>