[Openstack-operators] Launching new instances failing despite free resources

Lorin Hochstein lorin at nimbisservices.com
Thu Sep 6 21:14:59 UTC 2012

On Sep 6, 2012, at 9:18 AM, Jonathan Proulx <jon at jonproulx.com> wrote:

> On Thu, Sep 6, 2012 at 9:00 AM, Jonathan Proulx <jon at jonproulx.com> wrote:
>> running Essex (as packaged) on Ubuntu 12.04 launching new instances
>> has stopped working. They almost imediately enter Error Spawning state
>> which is consistent with my experience of resource starvation in the
>> cluster or scheduling errors.  Most recently I was /var/lib/nova on
>> one of the compute nodes filled but the schedule kept trying to run
>> new instances there (which is it's own issue, but I fixed that for me
>> simply by expanding that partition).  Though in that case they were
>> assigned to that node and then failing, in this case there is no
>> compute node associated with the failed instance.
> I tell a lie, there are all trying to run on the same compute node and
> in this case it has in sufficient memory to run the instance, so it
> shouldn't try to do so.  here are my scheduler options:
> --scheduler_available_filters=nova.scheduler.filters.standard_filters
> --scheduler_default_filters=AvailabilityZoneFilter,RamFilter,CoreFilter,ComputeFilter
> --least_cost_functions=nova.scheduler.least_cost.compute_fill_first_cost_fn
> --compute_fill_first_cost_fn_weight=1.0
> My intention (and what it seems to have been doing) is filling nodes
> based on RAM capacity rather than spreading as the default is.  I do
> believe it shouldn't try and run instances requiring 2G of ram when
> there are only 600M free.

By default, Essex tries to over provision based on memory. Try setting:

See http://docs.openstack.org/essex/openstack-compute/admin/content/scheduler-filters.html#ramfilter for more details (including why the over provisioning behavior is actually broken in Essex).

> I can't seem to figure out how to map from id in the compute_nodes
> table to host name or IP though there must be a way,
> hypervisor_hostname seems most likely but that is all NULL, any
> pointers there

Oooh, I'm pretty sure there's a way to do this, but don't remember offhand. Does "nova show <id>" give you this info? 

Take care,

Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.

> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20120906/1f930812/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4897 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20120906/1f930812/attachment-0001.bin>

More information about the OpenStack-operators mailing list