<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Texte de bulles Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.hoenzb
{mso-style-name:hoenzb;}
span.TextedebullesCar
{mso-style-name:"Texte de bulles Car";
mso-style-priority:99;
mso-style-link:"Texte de bulles";
font-family:"Tahoma","sans-serif";}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Arial","sans-serif";
color:black;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:609240378;
mso-list-template-ids:136239948;}
@list l1
{mso-list-id:1456680915;
mso-list-type:hybrid;
mso-list-template-ids:864421662 -849994714 67895299 67895301 67895297 67895299 67895301 67895297 67895299 67895301;}
@list l1:level1
{mso-level-start-at:15;
mso-level-number-format:bullet;
mso-level-text:\F0F0;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;
mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman";}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hi Tim<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">What I’d like to see is the best of both worlds. Users write Datalog policies describing whatever VM-placement policy they want. If the policy they’ve written is on the solver-scheduler’s list of options, we use the
hard-coded implementation, but if the policy isn’t on that list we translate directly to LP.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l1 level1 lfo2"><![if !supportLists]><span lang="EN-US" style="font-family:Wingdings"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US">How (calling the hard-coded implementation) ?<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:72.0pt;text-indent:-18.0pt;mso-list:l1 level2 lfo2">
<![if !supportLists]><span lang="EN-US" style="font-family:"Courier New""><span style="mso-list:Ignore">o<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US">Through the message bus?<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l1 level1 lfo2"><![if !supportLists]><span lang="EN-US" style="font-family:Wingdings"><span style="mso-list:Ignore">ð<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US">Is it Congress that will send out the data or should each implementation (of a policy) read it in directly?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Ruby<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Tim Hinrichs [mailto:thinrichs@vmware.com]
<br>
<b>Envoyé :</b> jeudi 12 février 2015 19:03<br>
<b>À :</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Objet :</b> Re: [openstack-dev] [Congress][Delegation] Google doc for working notes<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi Debo and Yathiraj,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">I took a third look at the solver-scheduler docs and code with your comments in mind. A few things jumped out.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">1) Choice of LP solver.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I see solver-scheduler uses Pulp, which was on the Congress short list as well. So we’re highly aligned on the choice of underlying solver.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">2) User control over VM-placement.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">To choose the criteria for VM-placement, the solver-scheduler user picks from a list of predefined options, e.g. ActiveHostConstraint, MaxRamAllocationPerHostConstraint.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">We’re investigating a slightly different approach, where the user defines the criteria for VM-placement by writing any policy they like in Datalog. Under the hood we then convert that Datalog to an LP problem. From the developer’s perspective,
with the Congress approach we don’t attempt to anticipate the different policies the user might want and write code for each policy; instead, we as developers write a translator from Datalog to LP. From the user’s perspective, the difference is that if the
option they want isn’t on the solver-scheduler's list, they’re out of luck or need to write the code themselves. But with the Congress approach, they can write any VM-placement policy they like.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">What I’d like to see is the best of both worlds. Users write Datalog policies describing whatever VM-placement policy they want. If the policy they’ve written is on the solver-scheduler’s list of options, we use the hard-coded implementation,
but if the policy isn’t on that list we translate directly to LP. This approach gives us the ability to write custom code to handle common cases while at the same time letting users write whatever policy they like.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">3) API and architecture.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Today the solver-scheduler's VM-placement policy is defined at config-time (i.e. not run-time). Am I correct that this limitation is only because there’s no API call to set the solver-scheduler’s policy? Or is there some other reason
the policy is set at config-time?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Congress policies change at runtime, so we’ll definitely need a VM-placement engine whose policy can be changed at run-time as well.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If we focus on just migration (and not provisioning), we can build a VM-placement engine that sits outside of Nova that has an API call that allows us to set policy at runtime. We can also set up that engine to get data updates that influence
the policy. We were planning on creating this kind of VM-placement engine within Congress as a node on the DSE (our message bus). This is convenient because all nodes on the DSE run in their own thread, any node on the DSE can subscribe to any data from
any other node (e.g. ceilometer’s data), and the algorithms for translating Datalog to LP look to be quite similar to the algorithms we’re using in our domain-agnostic policy engine.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Tim<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Feb 11, 2015, at 4:50 PM, Debojyoti Dutta <<a href="mailto:ddutta@gmail.com">ddutta@gmail.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Tim: moving our thread to the mailer. Excited to collaborate!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From: </span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Debo~ Dutta <<a href="mailto:dedutta@cisco.com"><span style="color:purple">dedutta@cisco.com</span></a>><br>
<b>Date: </b>Wednesday, February 11, 2015 at 4:48 PM<br>
<b>To: </b>Tim Hinrichs <<a href="mailto:thinrichs@vmware.com"><span style="color:purple">thinrichs@vmware.com</span></a>><br>
<b>Cc: </b>"Yathiraj Udupi (yudupi)" <<a href="mailto:yudupi@cisco.com"><span style="color:purple">yudupi@cisco.com</span></a>>, Gokul B Kandiraju <<a href="mailto:gokul@us.ibm.com"><span style="color:purple">gokul@us.ibm.com</span></a>>, Prabhakar Kudva <<a href="mailto:kudva@us.ibm.com"><span style="color:purple">kudva@us.ibm.com</span></a>>,
"<a href="mailto:ruby.krishnaswamy@orange.com"><span style="color:purple">ruby.krishnaswamy@orange.com</span></a>" <<a href="mailto:ruby.krishnaswamy@orange.com"><span style="color:purple">ruby.krishnaswamy@orange.com</span></a>>, "<a href="mailto:dilikris@in.ibm.com"><span style="color:purple">dilikris@in.ibm.com</span></a>"
<<a href="mailto:dilikris@in.ibm.com"><span style="color:purple">dilikris@in.ibm.com</span></a>>, Norival Figueira <<a href="mailto:nfigueir@Brocade.com"><span style="color:purple">nfigueir@Brocade.com</span></a>>, Ramki Krishnan <<a href="mailto:ramk@Brocade.com"><span style="color:purple">ramk@Brocade.com</span></a>>,
"Xinyuan Huang (xinyuahu)" <<a href="mailto:xinyuahu@cisco.com"><span style="color:purple">xinyuahu@cisco.com</span></a>>, "Rishabh Jain -X (rishabja - AAP3 INC at Cisco)" <<a href="mailto:rishabja@cisco.com"><span style="color:purple">rishabja@cisco.com</span></a>><br>
<b>Subject: </b>Re: Nova solver scheduler and Congress<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Tim <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">To address your particular questions: <o:p></o:p></span></p>
</div>
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">translate some policy language into constraints for the LP/CVP and we had left that to congress hoping to integrate when the policy efforts in openstack were ready (our initial effort was pre
congress) <o:p></o:p></span></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">For migrations: we are currently doing that – its about incremental constraints into the same solver. Hence its a small deal …. <o:p></o:p></span></li></ol>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Joining forces is a terrific idea. Would love to join the IRC call and see how we can build cool stuff in the community together. I hope we don’t have to replicate the vm
placement engine while the work that was done in the community does something very similar (and be adapted)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">debo<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From: </span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Tim Hinrichs <<a href="mailto:thinrichs@vmware.com"><span style="color:purple">thinrichs@vmware.com</span></a>><br>
<b>Date: </b>Wednesday, February 11, 2015 at 4:43 PM<br>
<b>To: </b>Debo~ Dutta <<a href="mailto:dedutta@cisco.com"><span style="color:purple">dedutta@cisco.com</span></a>><br>
<b>Cc: </b>"Yathiraj Udupi (yudupi)" <<a href="mailto:yudupi@cisco.com"><span style="color:purple">yudupi@cisco.com</span></a>>, Gokul B Kandiraju <<a href="mailto:gokul@us.ibm.com"><span style="color:purple">gokul@us.ibm.com</span></a>>, Prabhakar Kudva <<a href="mailto:kudva@us.ibm.com"><span style="color:purple">kudva@us.ibm.com</span></a>>,
"<a href="mailto:ruby.krishnaswamy@orange.com"><span style="color:purple">ruby.krishnaswamy@orange.com</span></a>" <<a href="mailto:ruby.krishnaswamy@orange.com"><span style="color:purple">ruby.krishnaswamy@orange.com</span></a>>, "<a href="mailto:dilikris@in.ibm.com"><span style="color:purple">dilikris@in.ibm.com</span></a>"
<<a href="mailto:dilikris@in.ibm.com"><span style="color:purple">dilikris@in.ibm.com</span></a>>, Norival Figueira <<a href="mailto:nfigueir@Brocade.com"><span style="color:purple">nfigueir@Brocade.com</span></a>>, Ramki Krishnan <<a href="mailto:ramk@Brocade.com"><span style="color:purple">ramk@Brocade.com</span></a>>,
"Xinyuan Huang (xinyuahu)" <<a href="mailto:xinyuahu@cisco.com"><span style="color:purple">xinyuahu@cisco.com</span></a>>, "Rishabh Jain -X (rishabja - AAP3 INC at Cisco)" <<a href="mailto:rishabja@cisco.com"><span style="color:purple">rishabja@cisco.com</span></a>><br>
<b>Subject: </b>Re: Nova solver scheduler and Congress<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Debo,
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">The 2 efforts share great similarities, which was why we investigated the state of solver-scheduler. From what I understand, (i) solver-scheduler doesn’t currently have
a policy language and (ii) it doesn’t do migrations. (I realize these are both in the works.) We needed both and wanted to make progress before those were complete.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">In the long run, it may make perfect sense to replace our vm-placement engine with yours. So joining forces sounds like a good idea. At the very *least* we ought to keep
up to date with each other’s progress. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">I’m starting to wonder if we ought to schedule a (bi-) weekly IRC for this topic.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Tim<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">On Feb 11, 2015, at 4:35 PM, Debo Dutta (dedutta) <<a href="mailto:dedutta@cisco.com"><span style="color:purple">dedutta@cisco.com</span></a>> wrote:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><br>
<br>
<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Tim <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">This looks awesome. Trying to figure out how this approach is different from the solver scheduler effort we did? We would be happy to fold our solver scheduler effort into
this (that way you also get code up and running)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Will also respond on the thread. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">thx<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">debo<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm;z-index:auto">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From: </span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Tim Hinrichs <<a href="mailto:thinrichs@vmware.com"><span style="color:purple">thinrichs@vmware.com</span></a>><br>
<b>Date: </b>Wednesday, February 11, 2015 at 4:11 PM<br>
<b>To: </b>"Yathiraj Udupi (yudupi)" <<a href="mailto:yudupi@cisco.com"><span style="color:purple">yudupi@cisco.com</span></a>><br>
<b>Cc: </b>Gokul B Kandiraju <<a href="mailto:gokul@us.ibm.com"><span style="color:purple">gokul@us.ibm.com</span></a>>, Prabhakar Kudva <<a href="mailto:kudva@us.ibm.com"><span style="color:purple">kudva@us.ibm.com</span></a>>, "<a href="mailto:ruby.krishnaswamy@orange.com"><span style="color:purple">ruby.krishnaswamy@orange.com</span></a>"
<<a href="mailto:ruby.krishnaswamy@orange.com"><span style="color:purple">ruby.krishnaswamy@orange.com</span></a>>, "<a href="mailto:dilikris@in.ibm.com"><span style="color:purple">dilikris@in.ibm.com</span></a>" <<a href="mailto:dilikris@in.ibm.com"><span style="color:purple">dilikris@in.ibm.com</span></a>>,
Norival Figueira <<a href="mailto:nfigueir@Brocade.com"><span style="color:purple">nfigueir@Brocade.com</span></a>>, Ramki Krishnan <<a href="mailto:ramk@Brocade.com"><span style="color:purple">ramk@Brocade.com</span></a>>, "Xinyuan Huang (xinyuahu)" <<a href="mailto:xinyuahu@cisco.com"><span style="color:purple">xinyuahu@cisco.com</span></a>>,
"Rishabh Jain -X (rishabja - AAP3 INC at Cisco)" <<a href="mailto:rishabja@cisco.com"><span style="color:purple">rishabja@cisco.com</span></a>>, Debo~ Dutta <<a href="mailto:dedutta@cisco.com"><span style="color:purple">dedutta@cisco.com</span></a>><br>
<b>Subject: </b>Re: Nova solver scheduler and Congress<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Yathiraj,
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">The group is getting big enough that we’ve decided to move the entire discussion to the openstack-dev mailing list. I sent a note today with the google doc we’re working
on. We’re trying to include [Congress][Delegation] in the subject line of relevant posts. Here’s the gdoc.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1ksDilJYXV-2D5AXWON8PLMedDKr9NpS8VbT0jIy-5FMIEtI_edit-3Fusp-3Dsharing&d=AwMF-g&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=no-emyiErtYa3_zneDNhY78LG0mCHc0bgMpXi1StZ7A&s=rLY2ACQqD5EQn3MgOnoX8M_zr9254v-FqhF56wfGpic&e="><span style="color:#954F72">https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit?usp=sharing</span></a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Tim<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">On Feb 10, 2015, at 11:10 AM, Yathiraj Udupi (yudupi) <<a href="mailto:yudupi@cisco.com"><span style="color:purple">yudupi@cisco.com</span></a>> wrote:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><br>
<br>
<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Tim, <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Thanks for your response. I think Congress will have to appreciate different APIs interacting with multiple components in the OpenStack ecosystem. So I will be happy to
help figure out the integration plan in general from the Congress side. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">I will be very interested and glad to participate in the discussions of designing these interfaces in Congress. Please share any preliminary designs you may have. I
had participated in one of the Congress mid-cycle meet ups, and I am interested in the upcoming work on these kind of enforcement aspects (reactive, proactive) of Congress. In terms of Nova scheduling via Solver scheduler, it will also help us be ready with
the integration points. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Let’s be in sync. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Yathi. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">On 2/10/15, 11:03 AM, "Tim Hinrichs" <<a href="mailto:thinrichs@vmware.com"><span style="color:purple">thinrichs@vmware.com</span></a>> wrote:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Yathiraj,
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Thanks for the help!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">The reason I asked is that we’re trying to figure out the basic interface for how two policy engines (in general) ought to interact. We were hoping Congress and solver-scheduler
had very similar APIs, which would make that interface relatively simple. But it sounds like the two systems have pretty different APIs. So for now we’ll keep working on that interface, and once we have something worked out we’ll touch base with you to think
through how an integration might work. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Tim<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">On Feb 6, 2015, at 11:45 AM, Yathiraj Udupi (yudupi) <<a href="mailto:yudupi@cisco.com"><span style="color:purple">yudupi@cisco.com</span></a>> wrote:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><br>
<br>
<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Tim, <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Thanks for reaching out. Currently available documentation is all in the project README - <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_stackforge_nova-2Dsolver-2Dscheduler_blob_master_README.md&d=AwMF-g&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=4W62lNEl8JKjwUKKASTM0fjHUVJATmMpEsCoYaeA_OQ&s=1rPI8hfdD9f8_B9_14xALPUjAuJJnWVJRhugMJ16IIk&e="><span style="color:purple">https://github.com/stackforge/nova-solver-scheduler/blob/master/README.md</span></a>
Towards the end there are some examples. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">To answer your specific questions:<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">1) How do we change the Nova server-placement policy? An API call? A file on the Nova server? <o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">- Currently the server placement policies are all defined in solver scheduler in the form of pluggable constraint or cost modules. So imagine there will be a policy to place
a compute VM close to on a storage volume host, there is a volume affinity cost module - <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_stackforge_nova-2Dsolver-2Dscheduler_blob_master_nova_scheduler_solvers_costs_volume-5Faffinity-5Fcost.py&d=AwMF-g&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=4W62lNEl8JKjwUKKASTM0fjHUVJATmMpEsCoYaeA_OQ&s=T34XhGj816dbMWcnLKU0Oda99ku1twIhUWN_SLTHPkQ&e="><span style="color:purple">https://github.com/stackforge/nova-solver-scheduler/blob/master/nova/scheduler/solvers/costs/volume_affinity_cost.py</span></a> and
you will feed the necessary inputs for the policy via scheduler hints that are induced during the nova API calls, in this case of the volume cost, the scheduler hint will be to provide the specific volume id. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">- So all these constraint and cost classes are defined the nova conf file, so all applicable policies are included statically at the moment, and inputs to the policies are
via scheduler hints. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">- We are yet to commit the code to dynamically set these policies at run-time, so we can choose what policies to include. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">2) What policy language does solver-scheduler use?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">- We haven’t yet added support for a formal policy language, but everything is implemented as pluggable modules now, and a policy will be simply defining what constraints
or costs to use, for what scenarios. We plan to use simple policy rules to define this, and will be added soon in the roadmap. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">3) I presume solver-scheduler *provisions* servers according to policy. Does it also *migrate* servers when they begin to violate policy?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Yes for provisioning according to policy, Support for migration is coming soon, and will utilize the nova migration workflow based on a policy violation detected by a constraint
validator. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Congress could feed some of these policies in terms of selecting our solver scheduler constraints and costs for specific scenarios, and also feed the necessary scheduler
hints. So based on the Congress policy, we can select our constraints and costs, and enable nova server placement accordingly. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Including our small solver scheduler team here from Cisco, we will be glad to work with the community. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Yathi. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">On 2/5/15, 4:22 PM, "Tim Hinrichs" <<a href="mailto:thinrichs@vmware.com"><span style="color:purple">thinrichs@vmware.com</span></a>> wrote:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hi Yathiraj,
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">I had a couple of questions about the nova solver scheduler. We’re looking into driving server-placement via policy through Congress and wanted to understand more about
how solver-scheduler works. If it’s easier to just point me to docs or examples, that’d be great!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Thanks!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Tim<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
</div>
</blockquote>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
</div>
</blockquote>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Feb 11, 2015 at 9:01 AM, Tim Hinrichs <<a href="mailto:thinrichs@vmware.com" target="_blank">thinrichs@vmware.com</a>> wrote:<o:p></o:p></p>
<div>
<p class="MsoNormal">Hi all, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">A (growing) group of folks are interested in working on the problem of delegating policy from Congress to domain-specific policy engines. We started looking at an NFV use case: migrating VMs to reduce energy consumption. In particular
we’re looking into building a VM-placement policy engine built on top of a linear programming solver. Here’s a doc with some working notes where we’re trying to figure out how to do the translation from Congress policy to the linear program.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1ksDilJYXV-2D5AXWON8PLMedDKr9NpS8VbT0jIy-5FMIEtI_edit-3Fusp-3Dsharing&d=AwMFaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=ZN1RTPgf15rmqr46xMdX8HUY-fmlaOfIY3TWbQlfgBk&s=zdYLSRitftQH2q_b1lhY80vIsECD9-xW6LZhcwJrRZA&e=" target="_blank"><span style="color:#954F72">https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit?usp=sharing</span></a><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888">Tim<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal">-Debo~<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org">OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<PRE>_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
</PRE></body>
</html>