<font size=2 face="sans-serif">In fact, there is a blueprint which would
enable supporting this scenario without partitioning -- </font><a href="https://blueprints.launchpad.net/nova/+spec/cpu-entitlement"><font size=3 color=blue><u>https://blueprints.launchpad.net/nova/+spec/cpu-entitlement</u></font></a><font size=3>
</font>
<br><font size=2 face="sans-serif">The idea is to annotate flavors with
CPU allocation guarantees, and enable differentiation between instances,
potentially running on the same host.</font>
<br><font size=2 face="sans-serif">The implementation is augmenting the
CoreFilter code to factor in the differentiation. Hopefully this will be
out for review soon.</font>
<br>
<br><font size=2 face="sans-serif">Regards,</font>
<br><font size=2 face="sans-serif">Alex<br>
<br>
</font>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">John Garbutt <john@johngarbutt.com></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">"OpenStack Development
Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org>,
</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">14/11/2013 04:57 PM</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [openstack-dev]
[nova] Configure overcommit policy</font>
<br>
<hr noshade>
<br>
<br>
<br><tt><font size=2>On 13 November 2013 14:51, Khanh-Toan Tran<br>
<khanh-toan.tran@cloudwatt.com> wrote:<br>
> Well, I don't know what John means by "modify the over-commit
calculation in<br>
> the scheduler", so I cannot comment.<br>
<br>
I was talking about this code:<br>
</font></tt><a href=https://github.com/openstack/nova/blob/master/nova/scheduler/filters/core_filter.py#L64><tt><font size=2>https://github.com/openstack/nova/blob/master/nova/scheduler/filters/core_filter.py#L64</font></tt></a><tt><font size=2><br>
<br>
But I am not sure thats what you want.<br>
<br>
> The idea of choosing free host for Hadoop on the fly is rather complicated<br>
> and contains several operations, namely: (1) assuring the host never
get<br>
> pass 100% CPU load; (2) identifying a host that already has a Hadoop
VM<br>
> running on it, or already 100% CPU commitment; (3) releasing the host
from<br>
> 100% CPU commitment once the Hadoop VM stops; (4) possibly avoiding
other<br>
> applications to use the host (to economy the host resource).<br>
><br>
> - You'll need (1) because otherwise your Hadoop VM would come short
of<br>
> resources after the host gets overloaded.<br>
> - You'll need (2) because you don't want to restrict a new host while
one of<br>
> your 100% CPU commited hosts still has free resources.<br>
> - You'll need (3) because otherwise you host would be forerever restricted,<br>
> and that is no longer "on the fly".<br>
> - You'll may need (4) because otherwise it'd be a waste of resources.<br>
><br>
> The problem of changing CPU overcommit on the fly is that when your
Hadoop<br>
> VM is still running, someone else can add another VM in the same host
with a<br>
> higher CPU overcommit (e.g. 200%), (violating (1) ) thus effecting
your<br>
> Hadoop VM also.<br>
> The idea of putting the host in the aggregate can give you (1) and
(2). (4)<br>
> is done by AggregateInstanceExtraSpecsFilter. However, it does not
give you<br>
> (3); which can be done with pCloud.<br>
<br>
Step 1: use flavors so nova can tell between the two workloads, and<br>
configure them differently<br>
<br>
Step 2: find capacity for your workload given your current cloud usage<br>
<br>
At the moment, most of our solutions involve reserving bits of your<br>
cloud capacity for different workloads, generally using host<br>
aggregates.<br>
<br>
The issue with claiming back capacity from other workloads is a bit<br>
tricker. The issue is I don't think you have defined where you get<br>
that capacity back from? Maybe you want to look at giving some<br>
workloads a higher priority over the constrained CPU resources? But<br>
you will probably starve the little people out at random, which seems<br>
bad. Maybe you want to have a concept of "spot instances" where
they<br>
can use your "spare capacity" until you need it, and you can
just kill<br>
them?<br>
<br>
But maybe I am miss understanding your use case, its not totally clear
to me.<br>
<br>
John<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
OpenStack-dev@lists.openstack.org<br>
</font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"><tt><font size=2>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</font></tt></a><tt><font size=2><br>
<br>
</font></tt>
<br>