<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hi all,
<div><br>
</div>
<div>Happy new year! (I added the openstack mailing list back into the thread.) Responses inline.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<div>
<div>On Jan 5, 2015, at 7:40 AM, <<a href="mailto:ruby.krishnaswamy@orange.com">ruby.krishnaswamy@orange.com</a>> <<a href="mailto:ruby.krishnaswamy@orange.com">ruby.krishnaswamy@orange.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Texte de bulles Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.TextedebullesCar
{mso-style-name:"Texte de bulles Car";
mso-style-priority:99;
mso-style-link:"Texte de bulles";
font-family:"Tahoma","sans-serif";}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Arial","sans-serif";
color:black;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:645475094;
mso-list-type:hybrid;
mso-list-template-ids:-864806962 -320332778 67895299 67895301 67895297 67895299 67895301 67895297 67895299 67895301;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1
{mso-list-id:806776346;
mso-list-type:hybrid;
mso-list-template-ids:267522898 788319200 67895299 67895301 67895297 67895299 67895301 67895297 67895299 67895301;}
@list l1:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial","sans-serif";
mso-fareast-font-family:Calibri;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2
{mso-list-id:1458135800;
mso-list-type:hybrid;
mso-list-template-ids:1940800560 2043564172 67895299 67895301 67895297 67895299 67895301 67895297 67895299 67895301;}
@list l2:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial","sans-serif";
mso-fareast-font-family:Calibri;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l3
{mso-list-id:2118404502;
mso-list-type:hybrid;
mso-list-template-ids:-2144175666 844914182 67895299 67895301 67895297 67895299 67895301 67895297 67895299 67895301;}
@list l3:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial","sans-serif";
mso-fareast-font-family:Calibri;}
@list l3:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l3:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l3:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l3:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l3:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l3:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l3:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l3:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Hello,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Happy New Year to all !<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">@Ramki: yes, please do send the original word document.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">@Tim from your mail:
</span><span lang="EN-US">My gut tells me that the placement-solver should basically say “I enforce policies having to do with the schema nova:location. This way the Congress policy engine knows to give it policies relevant to nova:location (placement). If
we do that, I believe we can carve off the right sub theory. ” </span><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"><o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo3"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Wingdings;"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">(for my understanding) Do you mean: The “solver” should take over derivation of ‘tables’ whose body contains an atom of one of the following?<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:72.0pt;text-indent:-18.0pt;mso-list:l0 level2 lfo3">
<!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: 'Courier New';"><span style="mso-list:Ignore">o<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> nova:location table,
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:72.0pt;text-indent:-18.0pt;mso-list:l0 level2 lfo3">
<!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: 'Courier New';"><span style="mso-list:Ignore">o<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">A table defined by other rules using nova:location (transitively)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:72.0pt;text-indent:-18.0pt;mso-list:l0 level2 lfo3">
<!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: 'Courier New';"><span style="mso-list:Ignore">o<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">A builtin using nova:location<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo3"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Wingdings;"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">So such (head) rules will get compiled into code to invoke the solver?
</span></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>In Congress the ‘error’ table is special. So conceptually yes—all errors that can be defined (transitively) in terms of nova:location and builtins like less-than, greater-than, etc. There is definitely a technical question to be answered here—what fragment
of Datalog can we send the solver? Can the solver handle negation? Which builtins can it handle? But I’d suggest we start with a slightly simpler case and work our way up from there. For example, here’s an anti-affinity policy.</div>
<div><br>
</div>
<div>// anti-affinity</div>
<div>error(vm) :-</div>
<div><span class="Apple-tab-span" style="white-space: pre;"></span>affinity_group(vm1, grp1),</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>affinity_group(vm2, grp2),</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>grp1 != grp2,</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>nova:location(vm1, loc1),</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>nova:location(vm2, loc2),</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>loc1 == loc2</div>
<div><br>
</div>
<div>This error is defined using nova:location and the builtins == and !=. But it also uses the affinity_group table, which would need to be communicated in some way to the solver.</div>
<div><br>
</div>
<blockquote type="cite">
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo3"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:54.0pt"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> Q: How to set the optimization goal?<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:0cm"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> What does “theory” correspond to?<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">@Tim: (for understanding): If we take only the anti-affinity “policy” enforcement, how can this be programmed as a policy using only Datalog?<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo3"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Wingdings;"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Place VM1 on hosts without breaching anti-affinity policy (for simplicity I am assuming that this is case of proactive placement without needing migration)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo3"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Wingdings;"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">ha_safe_assignments (VM1, host) :- nova:location(vm, host), not_same_ha_group(vm, VM1)<o:p></o:p></span></p>
<p class="MsoNormal"></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I think I answered this above (by happenstance, actually :)). </div>
<div><br>
</div>
<blockquote type="cite">
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> Then ha_safe_assignment will give table of possible (feasible) assignments?<o:p></o:p></span></p>
<p class="MsoNormal"></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>In terms of outputs...</div>
<div><br>
</div>
<div>If we’re integrating with the nova scheduler project, then we may not need to worry about this since once it’s given the right policy, it knows how to construct the appropriate linear program and utilize the outputs of that program to do VM placement.
I imagine they could enforce this policy proactively.</div>
<div>
<div><br>
</div>
<div>If we’re not using the Nova scheduler project, then we need to write an algorithm that takes the Datalog policy as input, constructs the linear program, and then utilizes the output to do VM placement. As a first cut I’d suggest focusing on reactive enforcement
since Ramki found the Nova API to migrate a VM.</div>
<div><br>
</div>
</div>
<blockquote type="cite">
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> <o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo3"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Wingdings;"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">In the case where an “optimal” (according to the goal policy) assignment is required<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo3"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Wingdings;"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">While we are talking of invoking solvers (lp, constraints etc), one may also want to implement some greedy algorithm to find the best assignment?<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Do we want to take this in consideration?</span></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I think we want to focus on getting correct answers first and worry about optimization second.</div>
<div><br>
</div>
<div><br>
</div>
<blockquote type="cite">
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">@Prabhakar: Did you mean that we need to first work out the following:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l2 level1 lfo2"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">INPUT : Goal to solve<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">How to set the goal (e.g. maximize servers occupancy) through Datalog?<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Then generate code (for goals and constraints below) to something like Pulp?
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l2 level1 lfo2"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">INPUT : Constraints to solver<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">How to identify solver variables (how to). E.g. for the ha assignment, we will want to set the following constraint:<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> Only one VM of a given group should be assigned to a host.<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> e.g. if Xij indicates that VM_i is assigned to host_j, and Gmn indicates that VM_m belongs to group_n, then we will want (for each group), the
sum of foreach j and each n, Xij*Gin = 1 for “I” in 1… max_vm”<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<div><br>
</div>
</div>
</div>
</blockquote>
<div>This is useful for me to see how you would create the matrix. The question is now how we write the algorithm that analyzes the Datalog above, identifies the solver variables, and generates the constraint. Once we’ve done that, we would also want to be
careful to ensure that if there are already VMs deployed, we try to fix a policy violation with a small number of migrations.</div>
<br>
<blockquote type="cite">
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l2 level1 lfo2"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">OUTPUT: matrix/table of assignment<o:p></o:p></span></p>
<p class="MsoListParagraph"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Will be “materialized” to a
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">BTW: Can we clarify: “</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">communicates with other domain specific policy engines</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">(such as solver)”<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l2 level1 lfo2"><!--[if !supportLists]--><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Is a solver considered as a domain specific policy engine? Or is a solver a mechanism/tool (providing algorithms) to calculate the values of variables
such that some configured “policies” are not violated and that some configured optimization goal is satisfied?</span></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I’d say…</div>
<div>“solver”: a tool/algorithm that assigns values to variables to achieve some goal (like a linear solver, SAT solver, SMT solver)</div>
<div>“policy engine”: a datacenter service that monitors/enforces/audits a given policy</div>
<br>
<blockquote type="cite">
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l2 level1 lfo2"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l2 level1 lfo2"><!--[if !supportLists]--><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span></span><!--[endif]--><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Instead: can the solver be considered as ‘extending’ the evaluation strategy used by the Datalog engine (by invoking a solver module)?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<div><br>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Conceptually, I’d say that “communicating with other domain specific policy engines” implies performing some analysis on the Datalog policy (i.e. not just evaluation) and invoking an API call that sets policy in the other engine. This could be completely
hidden from the policy writer—it’s just an implementation detail for how Congress monitors/enforces/audits policy. </div>
<div><br>
</div>
<div>“extending the evaluation strategy used by the Datalog engine” sounds more like we’re adding a new builtin to the policy language (like less-than, greater-than). This seems to require the policy-writer to learn about the extension and utilize it appropriately.</div>
<div><br>
</div>
<div>Tim</div>
<div><br>
</div>
<br>
<blockquote type="cite">
<div lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Regards<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;">Ruby<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif;"> </span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Ramki Krishnan [<a href="mailto:ramk@Brocade.com">mailto:ramk@Brocade.com</a>]
<br>
<b>Envoyé :</b> vendredi 19 décembre 2014 16:42<br>
<b>À :</b> Prabhakar Kudva; KRISHNASWAMY Ruby IMT/OLPS<br>
<b>Cc :</b> Gokul B Kandiraju; Norival Figueira; Tim Hinrichs<br>
<b>Objet :</b> RE: [Congress] Re: Placement and Scheduling via Policy<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Makes sense Prabhakar.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">We could capture Ruby’s additional use cases in the following document.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dkrishnan-2Dnfvrg-2Dpolicy-2Dbased-2Drm-2Dnfviaas_-3Finclude-5Ftext-3D1&d=AwMGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=Gy2MLgPshC-XbnttowqIfVvT5WQdEK6R6RGBCs6nkkQ&s=0zD4roZoYquRtqgb_SiSDTOIksVrxWwTMlU_qj3HNmk&e=">https://datatracker.ietf.org/doc/draft-krishnan-nfvrg-policy-based-rm-nfviaas/?include_text=1</a><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Ruby – I would be glad to send you the original word document for the same so that you can make your edits.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">We could kick off with an information model across congress and domain specific policy engines which could lead to a data model/protocol. Looks
like Nova will be our first victim -</span><span lang="EN-US" style="font-size:11.0pt;font-family:Wingdings;color:#1F497D">J</span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">. This could be another IETF draft.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Ramki<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Prabhakar Kudva [<a href="mailto:kudva@us.ibm.com">mailto:kudva@us.ibm.com</a>]
<br>
<b>Sent:</b> Friday, December 19, 2014 8:31 PM<br>
<b>To:</b> Ramki Krishnan<br>
<b>Cc:</b> Gokul B Kandiraju; Norival Figueira; <a href="mailto:ruby.krishnaswamy@orange.com">
ruby.krishnaswamy@orange.com</a>; Tim Hinrichs<br>
<b>Subject:</b> RE: [Congress] Re: Placement and Scheduling via Policy<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Since this is an example of how Congress communicates with other domain specific policy engines</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">(such as solver) perhaps we could spend time defining the exact apis and mechanisms of how we</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">would generally go about it. That is we could standardize how a Congress policy gets translated</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">to domain-specific ones in general? The path from the use case Ruby described can be a driver</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">for such an exercise?</span><span lang="EN-US">
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Prabhakar</span><span lang="EN-US">
<br>
<br>
<br>
<br>
<br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F">From: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">Ramki Krishnan <<a href="mailto:ramk@Brocade.com">ramk@Brocade.com</a>></span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F">To: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">"<a href="mailto:ruby.krishnaswamy@orange.com">ruby.krishnaswamy@orange.com</a>"
<<a href="mailto:ruby.krishnaswamy@orange.com">ruby.krishnaswamy@orange.com</a>>, "Tim Hinrichs" <<a href="mailto:thinrichs@vmware.com">thinrichs@vmware.com</a>>, Prabhakar Kudva/Watson/IBM@IBMUS</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F">Cc: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">Gokul B Kandiraju/Watson/IBM@IBMUS, Norival Figueira <<a href="mailto:nfigueir@Brocade.com">nfigueir@Brocade.com</a>></span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F">Date: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">12/19/2014 07:38 AM</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F">Subject: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">RE: [Congress] Re: Placement and Scheduling via Policy</span><span lang="EN-US">
<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-US">
<hr size="3" width="100%" noshade="" style="color:#A0A0A0" align="center">
</span></div>
<p class="MsoNormal"><span lang="EN-US"><br>
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080">Hi Ruby,</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080">Please find inline.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080">Thanks,</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080">Ramki</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"> </span><span lang="EN-US">
<br>
</span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
<a href="mailto:ruby.krishnaswamy@orange.com">ruby.krishnaswamy@orange.com</a> [</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">mailto:ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">]
<b><br>
Sent:</b> Friday, December 19, 2014 1:22 PM<b><br>
To:</b> Ramki Krishnan; Tim Hinrichs; Prabhakar Kudva<b><br>
Cc:</b> Gokul B Kandiraju; Norival Figueira<b><br>
Subject:</b> RE: [Congress] Re: Placement and Scheduling via Policy</span><span lang="EN-US">
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Hello Ramki</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">“Ramki: I believe nova-scheduler (or for that matter individual sub-system placement/scheduling engines) should still exist. The new placement/scheduling engine which works
across multiple sub-systems will over-ride the individual sub-system placement/scheduling engine as needed. These architectural aspects are captured in the following IETF draft”</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">OK, but I think that we should discuss further on what the “nova-scheduler” will be built on in the light of Congress policy engine.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">In the lines of this draft </span><span lang="EN-US"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dnorival-2Dnfvrg-2Dnfv-2Dpolicy-2Darch_-3Finclude-5Ftext-3D1&d=AwMGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=Gy2MLgPshC-XbnttowqIfVvT5WQdEK6R6RGBCs6nkkQ&s=GNziTPIS-pU6LtstkiIAfvuK0h46vnmbdyp36LM9bE0&e="><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">https://datatracker.ietf.org/doc/draft-norival-nfvrg-nfv-policy-arch/?include_text=1</span></a>,
it makes sense that the sub-systems (compute node) also rely on a policy engine. Taking your example (Platinum clients to be given platinum treatment):
<br>
</span><span lang="EN-US" style="font-family:Wingdings">ð </span><span lang="EN-US">Currently with Nova-scheduler, this lower policy will be implemented using the Host Aggregates mechanism.
<br>
</span><span lang="EN-US" style="font-family:Wingdings">ð </span><span lang="EN-US">But in going towards a convergent ‘policy’ architecture, it may make sense if subsystems also employ the same framework?
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080">Completely agree. As you mentioned, we should avoid point solutions like the host aggregates mechanism in Nova. I think this aspect is definitely worth bringing
up in our mailing list discussions.</span><span lang="EN-US"> <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Regards</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Ruby</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Ramki Krishnan [</span><span lang="EN-US"><a href="mailto:ramk@Brocade.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">mailto:ramk@Brocade.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">]
<b><br>
Envoyé :</b> vendredi 19 décembre 2014 08:40<b><br>
À :</b> KRISHNASWAMY Ruby IMT/OLPS; Tim Hinrichs; Prabhakar Kudva<b><br>
Cc :</b> Gokul B Kandiraju; Norival Figueira<b><br>
Objet :</b> RE: [Congress] Re: Placement and Scheduling via Policy</span><span lang="EN-US">
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Hi Ruby, Tim, All,</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Great discussions. Please find inline. I will be offline till next year and advanced happy new year to all.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Thanks,</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Ramki</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""> </span><span lang="EN-US">
<br>
</span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
[</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">mailto:ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">]
<b><br>
Sent:</b> Wednesday, December 17, 2014 12:28 PM<b><br>
To:</b> Tim Hinrichs; Prabhakar Kudva<b><br>
Cc:</b> Ramki Krishnan; Gokul B Kandiraju<b><br>
Subject:</b> RE: [Congress] Re: Placement and Scheduling via Policy</span><span lang="EN-US">
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Hi Tim & All</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">@Tim: I did not reply to openstack-dev. Do you think we could have an openstack list specific for “congress” to which anybody may subscribe?</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Ramki: I think we should include NFV and Congress in the openstack-dev list.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">1)</span><span lang="EN-US" style="font-size:7.5pt">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Enforcement:</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> By this we mean “how will the actions computed by the policy engine be executed by the concerned OpenStack functional module”.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> In this case, it is better to first work this out for a “simpler” case, e.g. your running example concerning the network/groups.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> Note: some actions concern only some data base (e.g. insert the user within some group).</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
2)</span><span lang="EN-US" style="font-size:7.5pt"> </span><span lang="EN-US">From Prabhakar’s mail
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">“Enforcement. That is with a large number of constraints in place for placement and</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
scheduling, how does the policy engine communicate and enforce the placement <br>
constraints to nova scheduler.</span><span lang="EN-US"> “ <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> Nova scheduler (current): It assigns VMs to servers based on the policy set by the administrator (through filters and host aggregates).
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> The administrator also configures a scheduling heuristic (implemented as a driver), for example “round-robin” driver.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> Then the computed assignment is sent back to the requestor (API server) that interacts with nova-compute to provision the VM.
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> The current nova-scheduler has another function: It updates the allocation status of each compute node on the DB (through another
indirection called nova-conductor)</span><span lang="EN-US"> <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> So it is correct to re-interpret your statement as follows:</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">-</span><span lang="EN-US" style="font-size:7.5pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">What is the entity with which
the policy engine interacts for either proactive or reactive placement management?</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">-</span><span lang="EN-US" style="font-size:7.5pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">How will the output from the policy
engine (for example the placement matrix) be communicated back?</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">o</span><span lang="EN-US" style="font-size:7.5pt">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Proactive: this gives the mapping of VM to host</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">o</span><span lang="EN-US" style="font-size:7.5pt">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Reactive: this gives the new mapping of running VMs to hosts</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">-</span><span lang="EN-US" style="font-size:7.5pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">How starting from the placement
matrix, the correct migration plan will be executed? (for reactive case)</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">3)</span><span lang="EN-US" style="font-size:7.5pt">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Currently openstack does not have “automated management of reactive placement”: Hence if the policy engine is used for reactive placement, then there is a need for another
“orchestrator” that can interpret the new proposed placement configuration (mapping of VM to servers) and execute the reconfiguration workflow.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Ramki: For arriving at the new proposed placement configuration, the policy engine needs to know the current resource mapping of all the sub-systems involved. Some ideas below
on division of labor – this definitely needs more discussion.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">-</span><span lang="EN-US" style="font-size:7.5pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Orchestrator (openstack heat
?) – proactive placement</span><span lang="EN-US"> <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">-</span><span lang="EN-US" style="font-size:7.5pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Congress – reactive placement</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">-</span><span lang="EN-US" style="font-size:7.5pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Resource manager (typically
distributed across individual sub-systems such as nova, neutron …) – global read/write access</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">For a proof of concept, the simple approach discussed in the IETF draft
</span><span lang="EN-US"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dkrishnan-2Dnfvrg-2Dpolicy-2Dbased-2Drm-2Dnfviaas_-3Finclude-5Ftext-3D1&d=AwMGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=Gy2MLgPshC-XbnttowqIfVvT5WQdEK6R6RGBCs6nkkQ&s=0zD4roZoYquRtqgb_SiSDTOIksVrxWwTMlU_qj3HNmk&e="><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">https://datatracker.ietf.org/doc/draft-krishnan-nfvrg-policy-based-rm-nfviaas/?include_text=1</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
should suffice I guess.</span><span lang="EN-US"> <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">4)</span><span lang="EN-US" style="font-size:7.5pt">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">So with a policy-based “placement engine” that is integrated with external solvers, then this engine will replace nova-scheduler?</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Could we converge on this?</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Ramki: I believe nova-scheduler (or for that matter individual sub-system placement/scheduling engines) should still exist. The new placement/scheduling engine which works
across multiple sub-systems will over-ride the individual sub-system placement/scheduling engine as needed. These architectural aspects are captured in the following IETF draft</span><span lang="EN-US">
<br>
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dnorival-2Dnfvrg-2Dnfv-2Dpolicy-2Darch_-3Finclude-5Ftext-3D1&d=AwMGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=Gy2MLgPshC-XbnttowqIfVvT5WQdEK6R6RGBCs6nkkQ&s=GNziTPIS-pU6LtstkiIAfvuK0h46vnmbdyp36LM9bE0&e="><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">https://datatracker.ietf.org/doc/draft-norival-nfvrg-nfv-policy-arch/?include_text=1</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
(copying Norival for the same).</span><span lang="EN-US"> <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Regards</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Ruby</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><span lang="EN-US">
<br>
</span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Tim Hinrichs [</span><span lang="EN-US"><a href="mailto:thinrichs@vmware.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">mailto:thinrichs@vmware.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">]
<b><br>
Envoyé :</b> mardi 16 décembre 2014 19:25<b><br>
À :</b> Prabhakar Kudva<b><br>
Cc :</b> KRISHNASWAMY Ruby IMT/OLPS; Ramki Krishnan (</span><span lang="EN-US"><a href="mailto:ramk@Brocade.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">ramk@Brocade.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">);
Gokul B Kandiraju; openstack-dev<b><br>
Objet :</b> [Congress] Re: Placement and Scheduling via Policy</span><span lang="EN-US">
<br>
<br>
[Adding openstack-dev to this thread. For those of you just joining… We started kicking around ideas for how we might integrate a special-purpose VM placement engine into Congress.]
<br>
<br>
Kudva: responses inline. <br>
<br>
<br>
On Dec 16, 2014, at 6:25 AM, Prabhakar Kudva <<a href="mailto:kudva@us.ibm.com">kudva@us.ibm.com</a>> wrote:
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Hi,
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
I am very interested in this.</span><span lang="EN-US"> <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
So, it looks like there are two parts to this:</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
1. Policy analysis when there are a significant mix of logical and builtin predicates (i.e.,</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
runtime should identify a solution space when there are arithmetic operators). This will</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
require linear programming/ILP type solvers. There might be a need to have a function</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
in runtime.py that specifically deals with this (Tim?)</span><span lang="EN-US"> <br>
<br>
I think it’s right that we expect there to be a mix of builtins and standard predicates. But what we’re considering here is having the linear solver be treated as if it were a domain-specific policy engine. So that solver wouldn’t be embedded into the runtime.py
necessarily. Rather, we’d delegate part of the policy to that domain-specific policy engine.
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">2. Enforcement. That is with a large number of constraints in place for placement and</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
scheduling, how does the policy engine communicate and enforce the placement <br>
constraints to nova scheduler.</span><span lang="EN-US"> <br>
<br>
I would imagine that we could delegate either enforcement or monitoring or both. Eventually we want enforcement here, but monitoring could be useful too.
<br>
<br>
And yes you’re asking the right questions. I was trying to break the problem down into pieces in my bullet (1) below. But I think there is significant overlap in the questions we need to answer whether we’re delegating monitoring or enforcement.
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Both of these require some form of mathematical analysis.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Would be happy and interested to discuss more on these lines.</span><span lang="EN-US">
<br>
<br>
Maybe take a look at how I tried to breakdown the problem into separate questions in bullet (1) below and see if that makes sense.
<br>
<br>
Tim <br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Prabhakar</span><span lang="EN-US">
<br>
<br>
<br>
<br>
<br>
<br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F"><br>
From: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">Tim Hinrichs <</span><span lang="EN-US"><a href="mailto:thinrichs@vmware.com"><span style="font-size:7.5pt;font-family:"Arial","sans-serif"">thinrichs@vmware.com</span></a></span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">></span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F"><br>
To: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">"</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:7.5pt;font-family:"Arial","sans-serif"">ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">"
<</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:7.5pt;font-family:"Arial","sans-serif"">ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">></span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F"><br>
Cc: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">"Ramki Krishnan (</span><span lang="EN-US"><a href="mailto:ramk@Brocade.com"><span style="font-size:7.5pt;font-family:"Arial","sans-serif"">ramk@Brocade.com</span></a></span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">)"
<</span><span lang="EN-US"><a href="mailto:ramk@Brocade.com"><span style="font-size:7.5pt;font-family:"Arial","sans-serif"">ramk@Brocade.com</span></a></span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">>, Gokul B Kandiraju/Watson/IBM@IBMUS,
Prabhakar Kudva/Watson/IBM@IBMUS</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F"><br>
Date: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">12/15/2014 12:09 PM</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#5F5F5F"><br>
Subject: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif"">Re: Placement and Scheduling via Policy</span><span lang="EN-US">
<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-US">
<hr size="3" width="100%" noshade="" style="color:#A0A0A0" align="center">
</span></div>
<p class="MsoNormal"><span lang="EN-US"><br>
<br>
<br>
<br>
[Adding Prabhakar and Gokul, in case they are interested.] <br>
<br>
1) Ruby, thinking about the solver as taking 1 matrix of [vm, server] and returning another matrix helps me understand what we’re talking about—thanks. I think you’re right that once we move from placement to optimization problems in general we’ll need to
figure out how to deal with actions. But if it’s a placement-specific policy engine, then we can build VM-migration into it.
<br>
<br>
It seems to me that the only part left is figuring out how to take an arbitrary policy, carve off the placement-relevant portion, and create the inputs the solver needs to generate that new matrix. Some thoughts...
<br>
<br>
- My gut tells me that the placement-solver should basically say “I enforce policies having to do with the schema nova:location.” This way the Congress policy engine knows to give it policies relevant to nova:location (placement). If we do that, I believe
we can carve off the right sub theory. <br>
<br>
- That leaves taking a Datalog policy where we know nova:location is important and converting it to the input language required by a linear solver. We need to remember that the Datalog rules may reference tables from other services like Neutron, Ceilometer,
etc. I think the key will be figuring out what class of policies we can actually do that for reliably. Cool—a concrete question.
<br>
<br>
<br>
2) We can definitely wait until January on this. I’ll be out of touch starting Friday too; it seems we all get back early January, which seems like the right time to resume our discussions. We have some concrete questions to answer, which was what I was hoping
to accomplish before we all went on holiday. <br>
<br>
Happy Holidays! <br>
Tim <br>
<br>
<br>
On Dec 15, 2014, at 5:53 AM, <<a href="mailto:ruby.krishnaswamy@orange.com">ruby.krishnaswamy@orange.com</a>> <<a href="mailto:ruby.krishnaswamy@orange.com">ruby.krishnaswamy@orange.com</a>> wrote:
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Hi Tim</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
“Questions:</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
1) Is there any more data the solver needs? Seems like it needs something about CPU-load for each VM.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
2) Which solver should we be using? What does the linear program that we feed it look like? How do we translate the results of the linear solver into a collection of ‘migrate_VM’ API calls?”</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Question (2) seems to me the first to address, in particular:</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
“how to prepare the input (variables, constraints, goal) and invoke the solver” </span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:Wingdings"><br>
ð </span><span lang="EN-US" style="font-size:10.0pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">We need rules that represent constraints to give the solver (e.g. a technical constraint that a VM should not be assigned
to more than one server or that more than maximum resource (cpu / mem …) of a server cannot be assigned.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
“how to translate the results of the linear solver into a collection of API calls”:</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:Wingdings"><br>
ð </span><span lang="EN-US" style="font-size:10.0pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">The output from the “solver” will give the new placement plan (respecting the constraints in input)?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
o </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">E.g. a table of [vm, server, true/false]</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:Wingdings"><br>
ð </span><span lang="EN-US" style="font-size:10.0pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Then this depends on how “action” is going to be implemented in Congress (whether an external solver is used or not)</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
o </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Is the action presented as the “final” DB rows that the system must produce as a result of the actions?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
o </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">E.g. if current vm table is [vm3, host4] and the recomputed row says [vm3, host6], then the action is to move vm3 to host6?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
“how will the solver be invoked”?</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:Wingdings"><br>
ð </span><span lang="EN-US" style="font-size:10.0pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">When will the optimization call be invoked?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:Wingdings"><br>
ð </span><span lang="EN-US" style="font-size:10.0pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Is it “batched”, e.g. periodically invoke Congress to compute new assignments?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
<br>
Which solver to use: <br>
</span><span lang="EN-US"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__www.coin-2Dor.org_projects_&d=AAMFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=3lvgeryw4T-aWafrSZZG96NcydtHt6HnT_6vKookx6U&s=01_9grcy8VGwbKRXcqhFRex3N0XIoCBzOimWFwXYI58&e="><span style="font-size:10.0pt;font-family:"Arial","sans-serif"">http://www.coin-or.org/projects/</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">
and </span><span lang="EN-US"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__www.coin-2Dor.org_projects_PuLP.xml&d=AAMFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=3lvgeryw4T-aWafrSZZG96NcydtHt6HnT_6vKookx6U&s=RRiv5ZWCQwWguBZIsIXzCA4_otY4Gr7aeFmFMRB4ZZQ&e="><span style="font-size:10.0pt;font-family:"Arial","sans-serif"">http://www.coin-or.org/projects/PuLP.xml</span></a>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
I think it may be useful to pass through an interface (e.g. LP modeler to generate LP files in standard formats accepted by prevalent solvers)</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
The mathematical program:</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
We can (Orange) contribute to writing down in an informal way the program for this precise use case, if this can wait until January.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Perhaps the objective is to may be “minimize the number of servers whose usage is less than 50%”, since the original policy “Not more than 1 server of type1 to have a load under 50%” need not necessarily have a solution.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
<br>
This may help to derive the “mappings” from Congress (rules to program equations, intermediary tables to program variables)?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
For “migration” use case: it may be useful to add some constraint representing cost of migration, such that the solver computes the new assignment plan such that the maximum migration cost is not exceeded. To start with, perhaps number of migrations?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
I will be away from the end of the week until 5<sup>th</sup> January. I will also discuss with colleagues to see how we can formalize contribution (congress+nfv poc).</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Rgds</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Ruby</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><br>
De :</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Tim Hinrichs [</span><span lang="EN-US"><a href="mailto:thinrichs@vmware.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">mailto:thinrichs@vmware.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">]
<b><br>
Envoyé :</b> vendredi 12 décembre 2014 19:41<b><br>
À :</b> KRISHNASWAMY Ruby IMT/OLPS<b><br>
Cc :</b> Ramki Krishnan (</span><span lang="EN-US"><a href="mailto:ramk@Brocade.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">ramk@Brocade.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">)<b><br>
Objet :</b> Re: Placement and Scheduling via Policy</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
There’s a ton of good stuff here! <br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
So if we took Ramki’s initial use case and combined it with Ruby’s HA constraint, we’d have something like the following policy.
<br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
// anti-affinity</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
error (server, VM1, VM2) :-</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
same_ha_group(VM1, VM2),</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
nova:location(VM1, server),</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
nova:location(VM2, server)</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
// server-utilization <br>
error(server) :-</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
type1_server(server),</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
ceilometer:average_utilization(server, “cpu-util”, avg),</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
avg < 50</span><span lang="EN-US"> <br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
As a start, this seems plenty complex to me. anti-affinity is great b/c it DOES NOT require a sophisticated solver; server-utilization is great because it DOES require a linear solver.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Data the solver needs:</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
- Ceilometer: cpu-utilization for all the servers</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
- Nova: data as to where each VM is located</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
- Policy: high-availability groups</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Questions:</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
1) Is there any more data the solver needs? Seems like it needs something about CPU-load for each VM.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
2) Which solver should we be using? What does the linear program that we feed it look like? How do we translate the results of the linear solver into a collection of ‘migrate_VM’ API calls?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Maybe another few emails and then we set up a phone call.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Tim</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
On Dec 11, 2014, at 1:33 AM, <</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">>
<</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">>
wrote:</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Hello</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
A) First a small extension to the use case that Ramki proposes <br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Add high availability constraint.</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Assuming server-a and server-b are of same size and same failure model.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
[Later: Assumption of identical failure rates can be loosened.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Instead of considering only servers as failure domains, can introduce other failure domains
</span><span lang="EN-US" style="font-size:10.0pt;font-family:Wingdings">è</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> not just an anti-affinity policy but a calculation from 99,99.. requirement to VM placements, e.g.
<br>
]</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- For an exemplary maximum usage scenario, 53 physical servers could be under peak utilization (100%), 1 server (server-a) could be under partial utilization (50%) with 2 instances of type large.3 and 1 instance of type large.2, and 1 server (server-b) could
be under partial utilization (37.5%) with 3 instances of type large.2.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Call VM.one.large2 as the large2 VM in server-a</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Call VM.two.large2 as one of the large2 VM in server-b</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- VM.one.large2 and VM.two.large2 <br>
- When one of the large.3 instances mapped to server-a is deleted from physical server type 1, Policy 1 will be violated, since the overall utilization of server-a falls to 37,5%.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Various new placements(s) are described below</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
VM.two.large2 must not be moved. Moving VM.two.large2 breaks non-affinity constraint.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
error (server, VM1, VM2) :-</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
node (VM1, server1),</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
node (VM2, server2),</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
same_ha_group(VM1, VM2),</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
equal(server1, server2);</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
<br>
1) New placement 1: Move 2 instances of large.2 to server-a. Overall</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
utilization of server-a - 50%. Overall utilization of server-b -</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
12.5%.</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
2) New placement 2: Move 1 instance of large.3 to server-b. Overall</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
utilization of server-a - 0%. Overall utilization of server-b -</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
62.5%.</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
3) New placement 3: Move 3 instances of large.2 to server-a. Overall</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
utilization of server-a - 62.5%. Overall utilization of server-b -</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
0%.</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
New placements 2 and 3 could be considered optimal, since they</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
achieve maximal bin packing and open up the door for turning off</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
server-a or server-b and maximizing energy efficiency.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
But new placement 3 breaks client policy.</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
BTW: what happens if a given situation does not allow the policy violation to be removed? </span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
B) Ramki’s original use case can itself be extended:</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Adding additional constraints to the previous use case due to cases such as:</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Server heterogeneity <br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- CPU “pinning” <br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- “VM groups” (and allocation <br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Application interference <br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Refining on the statement “i</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">nstantaneous energy consumption can be approximately measured using an overall utilization metric, which is a combination of CPU utilization, memory
usage, I/O usage, and network usage”</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Let me know if this will interest you. Some (e.g. application interference) will need some time. E.G; benchmarking / profiling to class VMs etc.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
C) New placement plan execution</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- In Ramki’s original use case, violation is detected at events such as VM delete.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
While certainly this by itself is sufficiently complex, we may need to consider other triggering cases (periodic or when multiple VMs are deleted/added)</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- In this case, it may not be sufficient to compute the new placement plan that brings the system to a configuration that does not break policy, but also add other goals</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
D) Let me know if a use case such as placing “video conferencing servers” (geographically distributed clients) would suit you (multi site scenario)</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:Wingdings"><br>
ð </span><span lang="EN-US" style="font-size:10.0pt"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Or is it too premature?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Ruby</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><br>
De :</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Tim Hinrichs [</span><span lang="EN-US"><a href="mailto:thinrichs@vmware.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">mailto:thinrichs@vmware.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">]
<b><br>
Envoyé :</b> mercredi 10 décembre 2014 19:44<b><br>
À :</b> KRISHNASWAMY Ruby IMT/OLPS<b><br>
Cc :</b> Ramki Krishnan (</span><span lang="EN-US"><a href="mailto:ramk@Brocade.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">ramk@Brocade.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">)<b><br>
Objet :</b> Re: Placement and Scheduling via Policy</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Hi Ruby, <br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Whatever information you think is important for the use case is good. Section 3 from one of the docs Ramki sent you covers his use case.</span><span lang="EN-US">
<u><span style="color:blue"><br>
</span></u><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dkrishnan-2Dnfvrg-2Dpolicy-2Dbased-2Drm-2Dnfviaas_-3Finclude-5Ftext-3D1&d=AAMFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=R82SMwEX_3O32-8F5eqMQ8Y6wuHt9WhjmMg6rr-4gWs&s=nE7Xheq0TcCDN98mFIOG_VvMsmfBeIDNDVVFV1HpJx0&e="><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">https://datatracker.ietf.org/doc/draft-krishnan-nfvrg-policy-based-rm-nfviaas/?include_text=1</span></a>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
From my point of view, the keys things for the use case are…</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
- The placement policy (i.e. the conditions under which VMs require migration).</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
- A description of how we want to compute what specific migrations should be performed (a sketch of (i) the information that we need about current placements, policy violations, etc., (2) what systems/algorithms/etc. can utilize that input to figure out what
migrations to perform.</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
I think we want to focus on the end-user/customer experience (write a policy, and watch the VMs move around to obey that policy in response to environment changes) and then work out the details of how to implement that experience. That’s why I didn’t include
things like delays, asynchronous/synchronous, architecture, applications, etc. in my 2 bullets above.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Tim</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
On Dec 10, 2014, at 8:55 AM, <</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">>
<</span><span lang="EN-US"><a href="mailto:ruby.krishnaswamy@orange.com"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">ruby.krishnaswamy@orange.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif"">>
wrote:</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
<br>
</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Hi Ramki, Tim</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
By a “format” for describing use cases, I meant to ask what sets of information to provide, for example,
<br>
- what granularity in description of use case?</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- a specific placement policy (and perhaps citing reasons for needing such policy)?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Specific applications</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Requirements on the placement manager itself (delay, …)?</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
o </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Architecture as well</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Specific services from the placement manager (using Congress), such as,</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
o </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Violation detection (load, security, …)</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
- Adapting (e.g. context-aware) of policies used</span><span lang="EN-US"> </span>
<span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
In any case I will read the documents that Ramki has sent to not resend similar things.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Regards</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
Ruby</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""><br>
</span><span lang="EN-US"> </span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><br>
De :</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Ramki Krishnan [</span><span lang="EN-US"><a href="mailto:ramk@Brocade.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">mailto:ramk@Brocade.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">]
<b><br>
Envoyé :</b> mercredi 10 décembre 2014 16:59<b><br>
À :</b> Tim Hinrichs; KRISHNASWAMY Ruby IMT/OLPS<b><br>
Cc :</b> Norival Figueira; Pierre Ettori; Alex Yip; </span><span lang="EN-US"><a href="mailto:dilikris@in.ibm.com"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">dilikris@in.ibm.com</span></a></span><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><br>
Objet :</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> RE: Placement and Scheduling via Policy</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
Hi Tim,</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
This sounds like a plan. It would be great if you could add the links below to the Congress wiki. I am all for discussing this in the openstack-dev mailing list and at this point this discussion is completely open.</span><span lang="EN-US">
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
IRTF NFVRG Research Group: </span><span lang="EN-US"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__trac.tools.ietf.org_group_irtf_trac_wiki_nfvrg&d=AAMFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=R82SMwEX_3O32-8F5eqMQ8Y6wuHt9WhjmMg6rr-4gWs&s=X---GnOf7YwhOGKMWYa8Mh52VtmO-2imfuZdKLEY39M&e="><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">https://trac.tools.ietf.org/group/irtf/trac/wiki/nfvrg</span></a>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
IRTF NFVRG draft on NFVIaaS placement/scheduling (includes system analysis for the PoC we are thinking):
</span><span lang="EN-US"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dkrishnan-2Dnfvrg-2Dpolicy-2Dbased-2Drm-2Dnfviaas_-3Finclude-5Ftext-3D1&d=AAMFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=R82SMwEX_3O32-8F5eqMQ8Y6wuHt9WhjmMg6rr-4gWs&s=nE7Xheq0TcCDN98mFIOG_VvMsmfBeIDNDVVFV1HpJx0&e="><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">https://datatracker.ietf.org/doc/draft-krishnan-nfvrg-policy-based-rm-nfviaas/?include_text=1</span></a>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
IRTF NFVRG draft on Policy Architecture and Framework (looking forward to your comments and thoughts):
</span><span lang="EN-US"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dnorival-2Dnfvrg-2Dnfv-2Dpolicy-2Darch_-3Finclude-5Ftext-3D1&d=AAMFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=R82SMwEX_3O32-8F5eqMQ8Y6wuHt9WhjmMg6rr-4gWs&s=lBet00H8iO1igDZNEMUGaryHWutkg8abBbL5VG8pjyk&e="><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">https://datatracker.ietf.org/doc/draft-norival-nfvrg-nfv-policy-arch/?include_text=1</span></a>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
Hi Ruby,</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
Looking forward to your use cases.</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
Thanks,</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
Ramki</span><span lang="EN-US"> </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#004080"><br>
</span><span lang="EN-US"> <br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">_________________________________________________________________________________________________________________________</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">This message and its attachments may contain confidential or privileged information that may be protected by law;</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">they should not be distributed, used or copied without authorisation.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">If you have received this email in error, please notify the sender and delete this message and its attachments.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">Thank you.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">_________________________________________________________________________________________________________________________</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""> </span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">This message and its attachments may contain confidential or privileged information that may be protected by law;</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">they should not be distributed, used or copied without authorisation.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">If you have received this email in error, please notify the sender and delete this message and its attachments.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.</span><span lang="EN-US">
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">Thank you.</span><span lang="EN-US">
<o:p></o:p></span></p>
</div>
<pre>_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
</pre>
</div>
</blockquote>
</div>
<br>
</div>
</body>
</html>