<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">Le 09/01/2015 15:07, Murray, Paul (HP
Cloud) a écrit :<br>
</div>
<blockquote
cite="mid:39E5672E03A1CB4B93936D1C4AA5E15D1D00137E@G6W2484.americas.hpqcorp.net"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas","serif";}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:976879973;
mso-list-type:hybrid;
mso-list-template-ids:1544333164 232298186 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";
color:windowtext;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<div>
<div>
<p class="MsoNormal">>There is bug when running nova with
ironic <a moz-do-not-send="true"
href="https://bugs.launchpad.net/nova/+bug/1402658"
target="_blank">https://bugs.launchpad.net/nova/+bug/1402658</a><span
style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">I filed this bug – it has been a
problem for us.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal">>The problem is at scheduler side
the IronicHostManager will consume all the resources for
that node whatever<o:p></o:p></p>
<p class="MsoNormal">>how much resource the instance
used. But at compute node side, the ResourceTracker won't
consume resources<o:p></o:p></p>
<p class="MsoNormal">>like that, just consume like normal
virtual instance. And ResourceTracker will update the
resource usage once the<o:p></o:p></p>
<p class="MsoNormal">>instance resource claimed, then
scheduler will know there are some free resource on that
node, then will try to<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">>schedule
other new instance to that node<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">You have summed up the problem
nicely – i.e.: the resource availability is calculated
incorrectly for ironic nodes.</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">>I take look at that, there is
NumInstanceFilter, it will limit how many instance can
schedule to one host. So can<o:p></o:p></p>
<p class="MsoNormal">>we just use this filter to finish
the goal? The max instance is configured by option
'max_instances_per_host', we<o:p></o:p></p>
<p class="MsoNormal">>can make the virt driver to report
how many instances it supported. The ironic driver can
just report max_instances_per_host=1.<o:p></o:p></p>
<p class="MsoNormal">>And libvirt driver can report
max_instance_per_host=-1, that means no limit. And then we
can just remove the<o:p></o:p></p>
<p class="MsoNormal">>IronicHostManager, then make the
scheduler side is more simpler. Does make sense? or there
are more trap?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">Makes sense, but solves the wrong
problem. The problem is what you said above – i.e.: the
resource availability is calculated incorrectly for
ironic nodes.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">The right solution would be to fix
the resource tracker. The ram resource on an ironic node
has different allocation behavior to a regular node. The
test to see if a new instance fits is the same, but
instead of deducting the requested amount to get the
remaining availability it should simply return 0. This
should be dealt with in the new resource objects ([2]
below) by either having different version of the
resource object for ironic nodes (certainly doable and
the most sensible option – resources should be presented
according to the resources on the host). Alternatively
the ram resource object should cater for the difference
in its calculations.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">I have a local fix for this that I
was too shy to propose upstream because it’s a bit hacky
and will hopefully be obsolete soon. I could share it if
you like.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">Paul<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">[2]
<a moz-do-not-send="true"
href="https://review.openstack.org/#/c/127609/">https://review.openstack.org/#/c/127609/</a>
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D"><o:p> </o:p></span></p>
</div>
</div>
</div>
</blockquote>
<br>
Agreed, I think that [2] will help a lot. Until it's done, are we
really sure we want to fix the bug ? It can be workarounded by
creating flavors being at least half the compute nodes and I really
would like adding more tech debt.<br>
<br>
-Sylvain<br>
<br>
<blockquote
cite="mid:39E5672E03A1CB4B93936D1C4AA5E15D1D00137E@G6W2484.americas.hpqcorp.net"
type="cite">
<div class="WordSection1">
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">From: <b>Sylvain
Bauza</b> <<a moz-do-not-send="true"
href="mailto:sbauza@redhat.com">sbauza@redhat.com</a>><br>
Date: 9 January 2015 at 09:17<br>
Subject: Re: [openstack-dev] [Nova][Ironic] Question about
scheduling two instances to same baremetal node<br>
To: "OpenStack Development Mailing List (not for usage
questions)" <<a moz-do-not-send="true"
href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Le 09/01/2015 09:01, Alex Xu a
écrit :<o:p></o:p></p>
</div>
<div>
<div>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Hi, All <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">There is bug when running
nova with ironic <a moz-do-not-send="true"
href="https://bugs.launchpad.net/nova/+bug/1402658"
target="_blank">https://bugs.launchpad.net/nova/+bug/1402658</a><span
style="color:#1F497D"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The case is simple: one
baremetal node with 1024MB ram, then boot two
instances with 512MB ram flavor.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Those two instances will be
scheduling to same baremetal node.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The problem is at scheduler
side the IronicHostManager will consume all
the resources for that node whatever<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">how much resource the
instance used. But at compute node side, the
ResourceTracker won't consume resources<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">like that, just consume
like normal virtual instance. And
ResourceTracker will update the resource usage
once the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">instance resource claimed,
then scheduler will know there are some free
resource on that node, then will try to<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">schedule other new instance
to that node.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I take look at that, there
is NumInstanceFilter, it will limit how many
instance can schedule to one host. So can<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">we just use this filter to
finish the goal? The max instance is
configured by option 'max_instances_per_host',
we<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">can make the virt driver to
report how many instances it supported. The
ironic driver can just report
max_instances_per_host=1.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">And libvirt driver can
report max_instance_per_host=-1, that means no
limit. And then we can just remove the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">IronicHostManager, then
make the scheduler side is more simpler. Does
make sense? or there are more trap?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks in advance for any
feedback and suggestion.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal">Mmm, I think I disagree with your
proposal. Let me explain by the best I can why :<br>
<br>
tl;dr: Any proposal unless claiming at the scheduler
level tends to be wrong<br>
<br>
The ResourceTracker should be only a module for
providing stats about compute nodes to the Scheduler.<br>
How the Scheduler is consuming these resources for
making a decision should only be a Scheduler thing.<br>
<br>
Here, the problem is that the decision making is also
shared with the ResourceTracker because of the claiming
system managed by the context manager when booting an
instance. It means that we have 2 distinct decision
makers for validating a resource.<br>
<br>
Let's stop to be realistic for a moment and discuss
about what could mean a decision for something else than
a compute node. Ok, let say a volume.<br>
Provided that *something* would report the volume
statistics to the Scheduler, that would be the Scheduler
which would manage if a volume manager could accept a
volume request. There is no sense to validate the
decision of the Scheduler on the volume manager, just
maybe doing some error management.<br>
<br>
We know that the current model is kinda racy with Ironic
because there is a 2-stage validation (see [1]). I'm not
in favor of complexifying the model, but rather put all
the claiming logic in the scheduler, which is a longer
path to win, but a safier one.<br>
<br>
-Sylvain<br>
<br>
[1] <a moz-do-not-send="true"
href="https://bugs.launchpad.net/nova/+bug/1341420"
target="_blank">https://bugs.launchpad.net/nova/+bug/1341420</a><br>
<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">Thanks<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Alex<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>OpenStack-dev mailing list<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>