Open Stack

+1 on Mikhail's suggestion. You will want to enable debug logging for the
[nova]-scheduler (restart it of course after modifying the config) and then
look for errors related to Filter. You can also correlate the req-XXX... ID
with logs for nova-compute on your compute node(s) to see if that helps.

On Thu, Jul 28, 2016 at 4:50 PM, Mikhail Medvedev <mihailmed at gmail.com>
wrote:

> On Thu, Jul 28, 2016 at 2:34 PM, Ken D'Ambrosio <ken at jots.org> wrote:
> > Trying to orchestrate a bunch of VMs on our ~500 core cloud, and we're
> > getting the "not enough hosts" problem.  Only issue is that the dashboard
> > seems to imply we've got the elbow room needed to instantiate.
> Furthermore,
> > it would be *really handy* if I knew what the bottleneck it thought was:
> > disk, RAM, CPU, whatever.  Here's a sample of the error:
> >
> > ----------------------- begin -----------------------
> > 2016-07-28 18:40:47.961 1080 WARNING nova.scheduler.utils
> > [req-977badf4-a4a2-4766-87fd-c4e3ceb2e293
> a4c895a6fe294ae3a81e3c6bb9209f52
> > 33c2cf3cb61c425abcb0e0d35fdf21b7 - - -] [instance:
> > ee175a40-11d9-4a43-b6e1-c4d7935acff1] Setting instance to ERROR state.
> > 2016-07-28 18:40:51.892 1023 WARNING nova.scheduler.utils
> > [req-e2a65200-1662-4cd7-aa9e-db5e225adfa1
> a4c895a6fe294ae3a81e3c6bb9209f52
> > 33c2cf3cb61c425abcb0e0d35fdf21b7 - - -] Failed to
> > compute_task_build_instances: No valid host was found. There are not
> enough
> > hosts available.
> > Traceback (most recent call last):
> >
> >   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py",
> line
> > 142, in inner
> >     return func(*args, **kwargs)
> >
> >   File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line
> > 84, in select_destinations
> >     filter_properties)
> >
> >   File
> > "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py",
> line
> > 90, in select_destinations
> >     raise exception.NoValidHost(reason=reason)
> >
> > NoValidHost: No valid host was found. There are not enough hosts
> available.
> > ----------------------- begin -----------------------
> >
> > Any suggestions on how to track down exactly what's going wrong?
>
> Try taking a look at nova-scheduler log on your controller (usually in
> /var/log/nova/nova-scheduler.log). I do not remember any time I have
> seen the "not enough hosts" when the reason was not obvious from the
> scheduler log.
>
> >
> > Thanks!
> >
> > -Ken
> >
> > _______________________________________________
> > Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> > Post to     : openstack at lists.openstack.org
> > Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
> ---
> Mikhail Medvedev
> IBM
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160728/a74d7406/attachment.html>