<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<br>
<div class="moz-cite-prefix">Le 17/02/2016 12:59, Chris Dent a
écrit :<br>
</div>
<blockquote cite="mid:alpine.OSX.2.20.1602171139160.6084@shine.home"
type="cite">On Wed, 17 Feb 2016, Cheng, Yingxin wrote:
<br>
<br>
<blockquote type="cite">To better illustrate the differences
between shared-state, resource-
<br>
provider and legacy scheduler, I've drew 3 simplified pictures
[1] in
<br>
emphasizing the location of resource view, the location of claim
and
<br>
resource consumption, and the resource update/refresh pattern in
three
<br>
kinds of schedulers. Hoping I'm correct in the
"resource-provider
<br>
scheduler" part.
<br>
</blockquote>
<br>
That's a useful visual aid, thank you. It aligns pretty well with
my
<br>
understanding of each idea.
<br>
<br>
A thing that may be missing, which may help in exploring the
usefulness
<br>
of each idea, is a representation of resources which are separate
<br>
from compute nodes and shared by them, such as shared disk or
pools
<br>
of network addresses. In addition some would argue that we need to
<br>
see bare-metal nodes for a complete picture.
<br>
<br>
One of the driving motivations of the resource-provider work is to
<br>
make it possible to adequately and accurately track and consume
the
<br>
shared resources. The legacy scheduler currently fails to do that
<br>
well. As you correctly points out it does this by having "strict
<br>
centralized consistency" as a design goal.
<br>
<br>
</blockquote>
<br>
So, to be clear, I'm really happy to see the resource-providers
series for many reasons :<br>
- it will help us getting a nice Facade for getting the resources
and attributing them<br>
- it will help a shared-storage deployment by making sure that we
don't have some resource problems when the resource is shared<br>
- it will create a possibility for external resource providers to
provide some resource types to Nova so the Nova scheduler could use
them (like Neutron related resources)<br>
<br>
That, I really want to have it implemented in Mitaka and Newton and
I'm totally on-board and supporting it.<br>
<br>
TBC, the only problem I see with the series is [2], not the whole,
please.<br>
<br>
<br>
<br>
<blockquote cite="mid:alpine.OSX.2.20.1602171139160.6084@shine.home"
type="cite">
<blockquote type="cite">As can be seen in the illustrations [1],
the main compatibility issue
<br>
between shared-state and resource-provider scheduler is caused
by the
<br>
different location of claim/consumption and the assumed
consistent
<br>
resource view. IMO unless the claims are allowed to happen in
both
<br>
places(resource tracker and resource-provider db), it seems
difficult
<br>
to make shared-state and resource-provider scheduler work
together.
<br>
</blockquote>
<br>
Yes, but doing claims twice feels intuitively redundant.
<br>
<br>
As I've explored this space I've often wondered why we feel it is
<br>
necessary to persist the resource data at all. Your shared-state
<br>
model is appealing because it lets the concrete
resource(-provider)
<br>
be the authority about its own resources. That is information
which
<br>
it can broadcast as it changes or on intervals (or both) to other
<br>
things which need that information. That feels like the correct
<br>
architecture in a massively distributed system, especially one
where
<br>
resources are not scarce.
<br>
</blockquote>
<br>
So, IMHO, we should only have the compute nodes being the authority
for allocating resources. They are many reasons for that I provided
in the spec review, but I can reply again :<br>
<br>
<ul class="com-google-gwtexpui-safehtml-client-SafeHtmlCss-wikiList">
<li>#1 If we consider that an external system, as a resource
provider, will provide a single resource class usage (like
network segment availability), it will still require the
instance to be spawned *for* consuming that resource class, even
if the scheduler accounts for it. That would mean that the
scheduler would have to manage a list of allocations with TTL,
and periodically verify that the allocation succeeded by asking
the external system (or getting feedback from the external
system). See, that's racy.</li>
<li>#2 the scheduler is just a decision maker, by any case it
doesn't account for the real instance creation (it doesn't hold
the ownership of the instance). Having it being accountable for
the instances usage is heavily difficult. Take for example a
request for CPU pinning or NUMA affinity. The user can't really
express which pin of the pCPU he will get, that's the compute
node which will do that for him. Of course, the scheduler will
help picking an host that can fit the request, but the real
pinning will happen in the compute node.</li>
</ul>
<br>
Also, I'm very interested in keeping an optimistic scheduler which
wouldn't lock the entire view of the world anytime a request comes
in. There are many papers showing different architectures and
benchmarks against different possibilities and TBH, I'm very
concerned by the scaling effect.<br>
Also, we should keep in mind our new paradigm called Cells V2, which
implies a global distributed scheduler for handling all requests.
Having it following the same design tenets of OpenStack [3] by
having a "eventually consistent shared-state" makes my guts saying
that I'd love to see that.<br>
<br>
<br>
<br>
<blockquote cite="mid:alpine.OSX.2.20.1602171139160.6084@shine.home"
type="cite">
<br>
The advantage of a centralized datastore for that information is
<br>
that it provides administrative control (e.g. reserving resources
for
<br>
other needs) and visibility. That level of command and control
seems
<br>
to be something people really want (unfortunately).
<br>
<br>
<br>
</blockquote>
<br>
My point is that while I truly understand the need of getting an API
resource like "scheduler, get me how much my cloud is free", that
shouldn't necessarly need to be accurate but rather eventually
consistent.<br>
If operators want to do capacity planning, they need trends and
thresholds, not exactly knowing the precise amounts that can change
everytime a request comes in.<br>
<br>
-Sylvain<br>
<br>
<br>
<br>
<br>
[2] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/271823/">https://review.openstack.org/#/c/271823/</a> <br>
[3] <a class="moz-txt-link-freetext" href="https://wiki.openstack.org/wiki/BasicDesignTenets">https://wiki.openstack.org/wiki/BasicDesignTenets</a><br>
<blockquote cite="mid:alpine.OSX.2.20.1602171139160.6084@shine.home"
type="cite">
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>