Hi, <div><br></div><div>my answers below,<br><br><div class="gmail_quote">On Thu, Oct 20, 2011 at 3:47 PM, Boris-Michel Deschenes <span dir="ltr"><<a href="mailto:boris-michel.deschenes@ubisoft.com">boris-michel.deschenes@ubisoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="FR-CA" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Hi guys,<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D">Just a quick note, I had this setup at some point (NFS-mounted /var/lib/nova/instances) which is essential to get live VM migrations if I’m not mistaken (live migration was working perfectly). The problem I had with this setup was that the VM startup time was considerably slower than when the images were residing on a local disk (and I mean, even after all images are “cached”).</span></p>
</div></div></blockquote><div><br></div><div>It's true. The fastest disk and the closest to the drive the better.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="FR-CA" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D">Basically an image will start the fastest when it is cached locally (local drive)</span></p></div></div></blockquote><div><br></div><div>Correct.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div lang="FR-CA" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D">Then, not quite as fast when cached but on a NFS-mounted directory</span></p></div></div></blockquote><div><br></div><div>Correct. It takes some time to create the local disks. It's very important to have a good connection to the shared file system (it's not mandatory to use NFS).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div lang="FR-CA" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D">Then really slowly when residing entirely on another disk and needed to be written locally to be cached.</span></p></div></div></blockquote><div>
<br></div><div>Right, it can take several minutes on a 1Gb.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div lang="FR-CA" link="blue" vlink="purple">
<div><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal">
<span lang="EN-CA" style="font-size:11.0pt;color:#1F497D">These are the observations I made but I realize other factors weigh in (SAS vs SATA disk, network speed, etc.) Please advise if you get the same speed in NFS-cached vs local-cached setup as it might convince me to go back to an NFS share (also were you using SAS disks to serve the NFS?).</span></p>
</div></div></blockquote><div><br></div><div>No, the performance on local disk is much higher than running a NFS on a 1Gb. For my perspective not only live migration is a must for our customers, but also the local virtual disks must persists a catastrophic failure of a nova-compute. That's the reason why recommend 10Gb and a good performant NFS file server connected. 15K or 10K SAS is not so relevant, the bottleneck is the network (speed and latency). There are also good solutions combining 10Gb + SSD Cache disks + 7.2KRPM SAS/SATA disks. </div>
<div><br></div><div>I would like to know what the people are using in real life deployments. Any more thoughts?</div><div><br></div><div>Regards</div><div>Diego</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="FR-CA" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D">Thanks<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-CA" style="font-size:11.0pt;color:#1F497D"><u></u> <u></u></span></p><p class="MsoNormal">
<b><span lang="FR" style="font-size:10.0pt">De :</span></b><span lang="FR" style="font-size:10.0pt"> <a href="mailto:openstack-operators-bounces@lists.openstack.org" target="_blank">openstack-operators-bounces@lists.openstack.org</a> [mailto:<a href="mailto:openstack-operators-bounces@lists.openstack.org" target="_blank">openstack-operators-bounces@lists.openstack.org</a>] <b>De la part de</b> Diego Parrilla<br>
<b>Envoyé :</b> 20 octobre 2011 04:54<br><b>À :</b> Till Mossakowski<br><b>Cc :</b> <a href="mailto:openstack-operators@lists.openstack.org" target="_blank">openstack-operators@lists.openstack.org</a><br><b>Objet :</b> Re: [Openstack-operators] Starting large VMs takes quite long<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">Hi, <u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">my answers below.<u></u><u></u></p></div><div><div><div></div>
<div class="h5"><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal">On Wed, Oct 19, 2011 at 8:12 PM, Till Mossakowski <<a href="mailto:Till.Mossakowski@dfki.de" target="_blank">Till.Mossakowski@dfki.de</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi,<u></u><u></u></p><p class="MsoNormal">my answers below,<u></u><u></u></p><p class="MsoNormal"><br>many thanks for your quick answer.<u></u><u></u></p><div><blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p><p class="MsoNormal"> I have set up openstack using stackops.<br><br><br>Good choice ;-)<u></u><u></u></p></blockquote><p class="MsoNormal"><u></u> <u></u></p>
</div><p class="MsoNormal">Yes, the stackops GUI is very nice. However, stackops is based on cactus, right? Is there a way of using diablo with stackops? Perhaps it is possible to upgrade the Ubuntu lucid distro that is coming with stackops to natty or oneiric and then upgrade to diablo using the source ppa:openstack-release/2011.3 for openstack?<u></u><u></u></p>
<div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Yes, the 0.3 version with Diablo release is coming. We detected some QA issues. But things are working much better now.<u></u><u></u></p></div>
<div>
<p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm"><div><blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p><p class="MsoNormal">5GB image it's not too big... we use NFS to share instances among nodes<br>to help with the live migration and performance it's acceptable. How<br>
much is 'quite a while' in seconds?<u></u><u></u></p></blockquote><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal">between half a minute and a minute (I haven't taken the exact time...).<br>
This is too long for our users.<u></u><u></u></p>
</blockquote><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">If the virtual disks are cached, launching a 40GB virtual machine takes less than 5 seconds in our test platform (IBM x3550M3 Dual Xeon 5620 64GB with NFS as shared storage on 1Gb)<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm"><div><blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p><p class="MsoNormal">If you share the /var/lib/nova/instances with NFS, during the 'launch'<br>process the base virtual image is copied to '_base'. Depending on the<br>
size of this file it will take longer. Once it's copied next time you<br>use this image it should go much faster.<br><br>Note: I have tested right now with a 1Gb launching a >25GB Windows VM<br>and it took 3-4 minutes the first time. New Windows images, it took only<br>
a few seconds.<u></u><u></u></p></blockquote><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal">This is interesting. Is there a way of telling the scheduler to prefer a compute node that already has copied the needed image?<u></u><u></u></p>
</blockquote><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Try this:<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">1) Configure the compute nodes to use a shared directory with NFS on /var/lib/nova/instances<u></u><u></u></p>
</div><div><p class="MsoNormal">2) Launch ALL the virtual disks you need at runtime. It will take a while the first time.<u></u><u></u></p></div><div><p class="MsoNormal">3) Virtual disks are now cached in /var/lib/nova/instances/_base<u></u><u></u></p>
</div><div><p class="MsoNormal">4) Try to launch now the virtual disks again. They should start very fast.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">If you need some kind of assistance, please let me know.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Regards<u></u><u></u></p></div><div><p class="MsoNormal">Diego<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div><div><p class="MsoNormal"><br><br>Best, Till<br><br>-- <br>Prof. Dr. Till Mossakowski Cartesium, room 2.51 Phone <a href="tel:%2B49-421-218-64226" target="_blank">+49-421-218-64226</a><br>DFKI GmbH Bremen Fax <a href="tel:%2B49-421-218-9864226" target="_blank">+49-421-218-9864226</a><br>
Safe & Secure Cognitive Systems <a href="mailto:Till.Mossakowski@dfki.de" target="_blank">Till.Mossakowski@dfki.de</a><br>Enrique-Schmidt-Str. 5, D-28359 Bremen <a href="http://www.dfki.de/sks/till" target="_blank">http://www.dfki.de/sks/till</a><br>
<br>Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH<br>principal office, *not* the address for mail etc.!!!:<br>Trippstadter Str. 122, D-67663 Kaiserslautern<br>management board: Prof. Wolfgang Wahlster (chair), Dr. Walter Olthoff<br>
supervisory board: Prof. Hans A. Aukes (chair)<br>Amtsgericht Kaiserslautern, HRB 2313<u></u><u></u></p></div></div></blockquote></div><p class="MsoNormal"><u></u> <u></u></p></div></div><div><div><div></div><div class="h5">
<p class="MsoNormal"><br clear="all">-- <span style="font-size:10.0pt;color:black"><u></u><u></u></span></p><div><p class="MsoNormal"><span lang="ES" style="font-size:10.0pt;color:black">Diego Parrilla<br></span><b><span style="font-size:10.0pt;color:black">CEO</span></b><span style="font-size:7.5pt;color:black"><br>
</span><b><span style="font-size:10.0pt;color:black"><a href="http://www.stackops.com/" target="_blank">www.stackops.com</a> | </span></b><span style="font-size:7.5pt;color:black"> <a href="mailto:diego.parrilla@stackops.com" target="_blank">diego.parrilla@stackops.com</a> | <a href="tel:%2B34%20649%2094%2043%2029" value="+34649944329" target="_blank">+34 649 94 43 29</a> | skype:diegoparrilla</span><span style="font-size:10.0pt;color:black"><a href="http://www.stackops.com/" target="_blank"><b><br>
</b></a><u></u><u></u></span></p></div></div></div><div><p><b><span style="font-size:10.0pt;color:#004438;border:solid windowtext 1.0pt;padding:0cm"><img border="0" width="100" height="100" alt="Description : Image supprimée par l'expéditeur."></span></b><b><span style="font-size:10.0pt;color:#004438"><u></u><u></u></span></b></p>
</div><div class="im"><div><p><span style="font-size:7.5pt;color:black">******************** ADVERTENCIA LEGAL ******************** <br>Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley. <br>
<br>***************** PRIVILEGED AND CONFIDENTIAL **************** <br>We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law.</span><span style="font-size:10.0pt;color:black"><u></u><u></u></span></p>
<p><span style="font-size:10.0pt;color:black"><u></u> <u></u></span></p></div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div></blockquote></div><br><div><br clear="all">-- <br><span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Times; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; font-size: medium; "><span style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; "><div align="left">
<div><font size="2"><span lang="ES" style="font-family: Arial; ">Diego Parrilla<br><a href="http://www.stackops.com/" title="file:///C:/Documents%20and%20Settings/carolina.capsir.per1/Application%20Data/Microsoft/Signatures/www.garrigues.com
www.garrigues.com" target="_blank" style="color: rgb(7, 77, 143); "><span title="file:///C:/Documents%20and%20Settings/carolina.capsir.per1/Application%20Data/Microsoft/Signatures/www.garrigues.com"></span></a></span></font><font face="Arial" size="2" style="font-family: arial, helvetica, sans-serif; "><b>CEO</b><font size="1"><br>
</font></font><span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Times; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; font-size: medium; "><span style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; "></span></span><b><font face="Arial" size="2" style="font-family: arial, helvetica, sans-serif; "><a href="http://www.stackops.com/" target="_blank"><b>www.stackops.com</b></a> | </font></b><font face="Arial" size="2" style="font-family: arial, helvetica, sans-serif; "><font size="1"> <a href="mailto:diego.parrilla@stackops.com" target="_blank">diego.parrilla@stackops.com</a></font></font><span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Times; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; font-size: medium; "><span style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; "><font color="#004438" face="Arial" size="2"><b><b><span lang="EN-GB" style="font-size: 10pt; color: rgb(0, 68, 56); font-family: Arial; "></span></b></b></font></span></span><font face="Arial" size="2" style="font-family: arial, helvetica, sans-serif; "><font size="1"> | </font></font><span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Times; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; font-size: medium; "><span style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; "><font face="Arial" size="2" style="font-family: arial, helvetica, sans-serif; "><font size="1">+34 649 94 43 29 | <a>skype:diegoparrilla</a></font></font></span></span><font face="Arial" size="2" style="font-family: arial, helvetica, sans-serif; "><a href="http://www.stackops.com/" target="_blank"><b><br>
</b></a></font></div></div><div><font color="#004438" face="Arial" size="2"><b><p><span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Times; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; font-size: medium; "><span style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; "><span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Times; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; font-size: medium; "><span style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; "><font color="#004438" face="Arial" size="2"><b><b><span lang="EN-GB" style="font-size: 10pt; color: rgb(0, 68, 56); font-family: Arial; "><img src="http://stackops.s3-external-3.amazonaws.com/STACKOPSLOGO-ICON.png"></span></b></b></font></span></span></span></span></p>
</b></font></div><div><p><font color="#000000" size="1">******************** ADVERTENCIA LEGAL ********************<span> </span><br>Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley.<span> </span><br>
<br>***************** PRIVILEGED AND CONFIDENTIAL ****************<span> </span><br>We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law.</font></p>
<p><font color="#000000" size="1"><br></font></p></div></span></span><br></div></div>