<html><body><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><div><style><!--
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:1;
mso-generic-font-family:roman;
mso-font-format:other;
mso-font-pitch:variable;
mso-font-signature:0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536870145 1073786111 1 0 415 0;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-520081665 -1073717157 41 0 66047 0;}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman","serif";
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;}
a:link, span.MsoHyperlink
{mso-style-noshow:yes;
mso-style-priority:99;
color:blue;
text-decoration:underline;
text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:purple;
text-decoration:underline;
text-underline:single;}
tt
{mso-style-noshow:yes;
mso-style-priority:99;
font-family:"Courier New";
mso-ascii-font-family:"Courier New";
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:"Courier New";
mso-bidi-font-family:"Courier New";}
span.EmailStyle18
{mso-style-type:personal-reply;
mso-style-noshow:yes;
mso-style-unhide:no;
mso-ansi-font-size:11.0pt;
mso-bidi-font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
color:#1F497D;
mso-themecolor:dark2;}
span.SpellE
{mso-style-name:"";
mso-spl-e:yes;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;
mso-header-margin:36.0pt;
mso-footer-margin:36.0pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
--></style></div><div class="WordSection1"><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:minor-latin;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";mso-bidi-theme-font:minor-bidi;color:#1F497D;mso-themecolor:dark2;mso-ansi-language:EN-US" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; mso-ascii-theme-font: minor-latin; mso-hansi-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-bidi-theme-font: minor-bidi; color: #1f497d; mso-themecolor: dark2; mso-ansi-language: EN-US;">Dear all,</span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:minor-latin;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";mso-bidi-theme-font:minor-bidi;color:#1F497D;mso-themecolor:dark2;mso-ansi-language:EN-US" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; mso-ascii-theme-font: minor-latin; mso-hansi-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-bidi-theme-font: minor-bidi; color: #1f497d; mso-themecolor: dark2; mso-ansi-language: EN-US;"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:minor-latin;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";mso-bidi-theme-font:minor-bidi;color:#1F497D;mso-themecolor:dark2;mso-ansi-language:EN-US" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; mso-ascii-theme-font: minor-latin; mso-hansi-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-bidi-theme-font: minor-bidi; color: #1f497d; mso-themecolor: dark2; mso-ansi-language: EN-US;">There is a need for a scheduler that can scheduling a group of resources as a whole, which is difficult to realize due to the separation of Nova, Cinder –scheduler. Thus I’m in favor of a dedicated scheduler component.</span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:minor-latin;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";mso-bidi-theme-font:minor-bidi;color:#1F497D;mso-themecolor:dark2;mso-ansi-language:EN-US" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; mso-ascii-theme-font: minor-latin; mso-hansi-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-bidi-theme-font: minor-bidi; color: #1f497d; mso-themecolor: dark2; mso-ansi-language: EN-US;"><br></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri, sans-serif; font-size: 11pt;">However, before talking about API or implementation, wouldn't it be better to see if the nova/cinder scheduler is independent enough to be separated from the core, in particular the data that the require to make a proper scheduling decision? It is reasonable to look again at the current architecture of Nova and Cinder to see which relation that</span><span data-mce-style="mso-spacerun: yes;" style="color: rgb(31, 73, 125); font-family: Calibri, sans-serif; font-size: 11pt;"> </span><span style="color: rgb(31, 73, 125); font-family: Calibri, sans-serif; font-size: 11pt;">nova-scheduler and cinder-scheduler have with the rest of nova/cinder components, and which data they take from nova/cinder DB and whether these data can be separated from Nova/Cinder. If we’re trying to separate Nova/Cinder-scheduler from the rest, better to cut all ties between them and the rest. </span></p><p class="MsoNormal"><tt><span lang="EN-US" style="font-size:10.0pt;mso-ansi-language:EN-US" data-mce-style="font-size: 10.0pt; mso-ansi-language: EN-US;">> 4) Drop cinder, and nova tables that store host states (as we don't </span></tt><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-ansi-language:EN-US" data-mce-style="font-size: 10.0pt; font-family: 'Courier New'; mso-ansi-language: EN-US;"><br><tt>> need them) </tt></span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:minor-latin;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";mso-bidi-theme-font:minor-bidi;color:#1F497D;mso-themecolor:dark2;mso-ansi-language:EN-US" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; mso-ascii-theme-font: minor-latin; mso-hansi-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-bidi-theme-font: minor-bidi; color: #1f497d; mso-themecolor: dark2; mso-ansi-language: EN-US;">Is it really OK to drop these tables? Could Nova can work without them (e.g. rollback)? And if Ceilometer is about to ask nova for host state metrics ?</span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:minor-latin;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";mso-bidi-theme-font:minor-bidi;color:#1F497D;mso-themecolor:dark2;mso-ansi-language:EN-US" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; mso-ascii-theme-font: minor-latin; mso-hansi-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-bidi-theme-font: minor-bidi; color: #1f497d; mso-themecolor: dark2; mso-ansi-language: EN-US;"> </span></p><p class="MsoNormal"><a name="_MailOriginal" class="mceItemAnchor"></a><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-font-family:"Times New Roman"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif'; mso-fareast-font-family: 'Times New Roman';">De :</span></b><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-font-family:"Times New Roman"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif'; mso-fareast-font-family: 'Times New Roman';"> Boris <span class="SpellE">Pavlovic</span> [mailto:bpavlovic@mirantis.com] <br><b>Envoyé :</b> vendredi 15 novembre 2013 16:57<br><b>À :</b> OpenStack Development Mailing List (not for usage questions)<br><b>Objet :</b> Re: [openstack-dev] [nova][cinder][oslo][scheduler] How to leverage oslo schduler/filters for nova and cinder</span></span></p><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">Alex,</span></p><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">(*sorry for long reply from vocation*)</span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Arial','sans-serif';">>> How do you envision the life cycle of such a scheduler in terms of code repository, build, test, etc? </span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">As a first step we could just make it inside nova, when we finish and prove that this approach works well we could split it out the nova in separated project and integrate with devstack and so on so on...</span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Arial','sans-serif';">>> What kind of changes to provisioning APIs do you envision to 'feed' such a scheduler? </span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">At this moment nova.scheduler is already separated service with amqp queue, what we need at this moment is to add 1 new rpc method to it. That will update state of some host. </span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Arial','sans-serif';">>> Any particular reason you didn't mention Neutron? </span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Yes, I am not familiar with Neutron architecture =)</span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Arial','sans-serif';">>> Also, there are some interesting technical challenges (e.g., state management across potentially large number of instances of memcached). </span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">10-100k keys-values is nothing for memcached. So what kind of instances?</span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Best regards,</span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Boris Pavlovic</span></span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div></div><div><p class="MsoNormal" style="margin-bottom:12.0pt" data-mce-style="margin-bottom: 12.0pt;"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">On Sun, Nov 10, 2013 at 4:20 PM, Alex Glikson <</span><a href="mailto:GLIKSON@il.ibm.com" target="_blank" data-mce-href="mailto:GLIKSON@il.ibm.com"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">GLIKSON@il.ibm.com</span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">> wrote:</span></p><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Hi Boris,</span> <br></span></p><div><br></div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">This is a very interesting approach.</span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">How do you envision the life cycle of such a scheduler in terms of code repository, build, test, etc?</span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">What kind of changes to provisioning APIs do you envision to 'feed' such a scheduler?</span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Any particular reason you didn't mention Neutron?</span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Also, there are some interesting technical challenges (e.g., state management across potentially large number of instances of memcached).</span> <br></span></p><div><br></div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Thanks,</span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-family:"Arial","sans-serif"" data-mce-style="font-family: 'Arial','sans-serif';">Alex</span> <br></span></p><div><br></div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">Boris Pavlovic <</span></tt></span><a href="mailto:bpavlovic@mirantis.com" target="_blank" data-mce-href="mailto:bpavlovic@mirantis.com"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';">bpavlovic@mirantis.com</span></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> wrote on 10/11/2013 07:05:42 PM:</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br></span></span></p><div><br></div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><tt>> From: Boris Pavlovic <</tt></span></span><a href="mailto:bpavlovic@mirantis.com" target="_blank" data-mce-href="mailto:bpavlovic@mirantis.com"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';">bpavlovic@mirantis.com</span></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">></span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> To: "OpenStack Development Mailing List (not for usage questions)" </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> <</tt></span></span><a href="mailto:openstack-dev@lists.openstack.org" target="_blank" data-mce-href="mailto:openstack-dev@lists.openstack.org"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';">openstack-dev@lists.openstack.org</span></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">>, </span></tt><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> Date: 10/11/2013 07:07 PM</span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> Subject: Re: [openstack-dev] [nova][cinder][oslo][scheduler] How to </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> leverage oslo schduler/filters for nova and cinder</tt></span> </span></p><div><div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Jay, </tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Hi Jay, yes we were working about putting all common stuff in oslo-</tt><br><tt>> scheduler. (not only filters)</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> As a result of this work we understood, that this is wrong approach.</tt><br><tt>> Because it makes result code very complex and unclear. And actually </tt><br><tt>> we didn't find the way to put all common stuff inside oslo. Instead </tt><br><tt>> of trying to make life too complex we found better approach. </tt><br><tt>> Implement scheduler aaS that can scale (current solution has some </tt><br><tt>> scale issues) & store all data from nova, cinder & probably other places.</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> To implement such approach we should change a bit current architecture: </tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 1) Scheduler should store all his data (not nova.db & cinder.db)</span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 2) Scheduler should always have own snapshot of "wold" state, and </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> sync it with another schedulers using something that is quite fast </tt><br><tt>> (e.g. memcached) </tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 3) Merge schedulers rpc methods from nova & cinder in one scheduler </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> (it is possbile if we store all data from cinder & nova in one sceduler). </tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 4) Drop cinder, and nova tables that store host states (as we don't </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> need them) </tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> We implemented already base start (mechanism that store snapshot of </tt><br><tt>> world state & sync it between different schedulers): </tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> </tt></span></span><a href="https://review.openstack.org/#/c/45867/" target="_blank" data-mce-href="https://review.openstack.org/#/c/45867/"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">https://review.openstack.org/#/c/45867/</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;"> (it is still bit in WIP)</span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Best regards, </tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> Boris Pavlovic</span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> ---</span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> Mirantis Inc. </span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> </tt><br></span><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> On Sun, Nov 10, 2013 at 1:59 PM, Jay Lau <</span></tt></span><a href="mailto:jay.lau.513@gmail.com" target="_blank" data-mce-href="mailto:jay.lau.513@gmail.com"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';">jay.lau.513@gmail.com</span></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> wrote:</span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> I noticed that there is already a bp in oslo tracing what I want to do: </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> </tt></span></span><a href="https://blueprints.launchpad.net/oslo/+spec/oslo-scheduler" target="_blank" data-mce-href="https://blueprints.launchpad.net/oslo/+spec/oslo-scheduler"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">https://blueprints.launchpad.net/oslo/+spec/oslo-scheduler</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br></span><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> Thanks,</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br></span><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> Jay</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br></span><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br></span><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 2013/11/9 Jay Lau <</span></tt></span><a href="mailto:jay.lau.513@gmail.com" target="_blank" data-mce-href="mailto:jay.lau.513@gmail.com"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';">jay.lau.513@gmail.com</span></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">></span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> Greetings,</span></tt> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Now in oslo, we already put some scheduler filters/weights logic </tt><br><tt>> there and cinder is using oslo scheduler filters/weights logic, </tt><br><tt>> seems we want both nova&cinder use this logic in future.</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Found some problems as following:</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 1) In cinder, some filters/weight logic reside in cinder/openstack/</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> common/scheduler and some filter/weight logic in cinder/scheduler, </tt><br><tt>> this is not consistent and also will make some cinder hackers </tt><br><tt>> confused: where shall I put the scheduler filter/weight.</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 2) Nova is not using filter/weight from oslo and also not using </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> entry point to handle all filter/weight.</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 3) There is not enough filters in oslo, we may need to add more </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> there: such as same host filter, different host filter, retry filter etc.</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> So my proposal is as following:</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 1) Add more filters to oslo, such as same host filter, different </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> host filter, retry filter etc.</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 2) Move all filters/weight logic in cinder from cinder/scheduler to </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> cinder/openstack/common/scheduler</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> 3) Enable nova use filter/weight logic from oslo (Move all filter </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> logic to nova/openstack/common/scheduler) and also use entry point </tt><br><tt>> to handle all filters/weight logic.</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Comments?</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Thanks,</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> Jay</tt></span> <br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> </span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> </tt><br><tt>> _______________________________________________</tt><br><tt>> OpenStack-dev mailing list</tt><br><tt>> </tt></span></span><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank" data-mce-href="mailto:OpenStack-dev@lists.openstack.org"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';">OpenStack-dev@lists.openstack.org</span></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> </tt></span></span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" data-mce-href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br></span><br></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">> _______________________________________________</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> OpenStack-dev mailing list</tt><br><tt>> </tt></span></span><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank" data-mce-href="mailto:OpenStack-dev@lists.openstack.org"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';">OpenStack-dev@lists.openstack.org</span></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><span style="font-size:10.0pt;font-family:"Courier New"" data-mce-style="font-size: 10.0pt; font-family: 'Courier New';"><br><tt>> </tt></span></span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" data-mce-href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><tt><span style="font-size:10.0pt" data-mce-style="font-size: 10.0pt;">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</span></tt></span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></p></div></div><p class="MsoNormal" style="margin-bottom:12.0pt" data-mce-style="margin-bottom: 12.0pt;"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><br>_______________________________________________<br>OpenStack-dev mailing list<br></span><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank" data-mce-href="mailto:OpenStack-dev@lists.openstack.org"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">OpenStack-dev@lists.openstack.org</span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><br></span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" data-mce-href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></p></div><p class="MsoNormal"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"> </span></p></div><div class="MsoNormal" align="center" style="text-align:center" data-mce-style="text-align: center;"><hr size="1" width="100%" noshade="noshade" style="color:#A0A0A0" align="center" data-mce-style="color: #a0a0a0;"></div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto" data-mce-style="mso-margin-top-alt: auto; mso-margin-bottom-alt: auto;"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">Aucun virus trouvé dans ce message.<br>Analyse effectuée par AVG - </span><a href="http://www.avg.fr" target="_blank" data-mce-href="http://www.avg.fr"><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;">www.avg.fr</span><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"></span></a><span style="mso-bookmark:_MailOriginal" data-mce-style="mso-bookmark: _MailOriginal;"><br>Version: 2014.0.4158 / Base de données virale: 3629/6837 - Date: 14/11/2013</span></p></div></div></body></html>