<div dir="ltr">Tim, <div><br></div><div>First of all we should finish pure OpenStack profiling system. Soon I am going to raise another thread about it. </div><div>This will allow us not only to detect that we have some issues in some OpenStack API, but also help us to find the real reason. </div>
<div><br></div><div>Secondly we should cover all main functionality with benchmarks. (I think this will be done pretty soon)</div><div><br></div><div><br></div><div>Thirdly we (openstack community) should start running benchmark against different clouds against real customers installations & dev. environments, investigate profiling/benchmark results and as a result we will be able to:</div>
<div>1) Find the best/optimal hardware set for OS</div><div>2) Tune code</div><div>3) Find optimal deployment arch in case of specific load/hardware</div><div><br></div><div>So we will be able to test different arch/code/hardware and collect all this information on some OpenStack wiki pages. </div>
<div>But this will require a lot of work of whole community.. I hope Rally team will get it..</div><div><br></div><div><br></div><div>Best regards,</div><div>Boris Pavlovic</div><div><br></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Sun, Dec 29, 2013 at 12:08 AM, Tim Bell <span dir="ltr"><<a href="mailto:Tim.Bell@cern.ch" target="_blank">Tim.Bell@cern.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks.. can you advise where the accumulated experience from Rally will be assembled ?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Rally gives me the method to test my cloud but we also need to have a set of documentation on how to build clouds for scale so we
don’t all have to tune (and end up with different approaches)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Tim<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif""> <a href="mailto:boris@pavlovic.ru" target="_blank">boris@pavlovic.ru</a> [mailto:<a href="mailto:boris@pavlovic.ru" target="_blank">boris@pavlovic.ru</a>]
<b>On Behalf Of </b>Boris Pavlovic<br>
<b>Sent:</b> 28 December 2013 21:02<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Cc:</b> Ali Beddah; Tim Bell</span></p><div><div class="h5"><br>
<b>Subject:</b> Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack<u></u><u></u></div></div><p></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Ali Gamal,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">Tim, <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Yes it fits. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">There are couple of use cases that should be covered by Rally:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">1) Easy way to find & fix bottlenecks/scale issues & improve performance of OS (without having tons of servers)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2) Find the best Arch for your hardware and your typical loads<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">3) Ensure that existing installation pass SLA<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">4) Ensure that OpenStack VMs have their resources and work as expected (already started discussion)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Best regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Boris Pavlovic <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Sat, Dec 28, 2013 at 11:40 PM, Tim Bell <<a href="mailto:Tim.Bell@cern.ch" target="_blank">Tim.Bell@cern.ch</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I think there also needs to be a scalability best practise and reference architecture.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Benchmarking allows us to identify problems with the code but we also need some community wisdom
on how to deploy at scale.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Does this fit within Rally or can you advise where this community wisdom should be accumulated ?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Tim</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">
Ali Gamal [mailto:<a href="mailto:agamal@itsyn.com" target="_blank">agamal@itsyn.com</a>]
<br>
<b>Sent:</b> 28 December 2013 20:31</span><u></u><u></u></p>
<div>
<p class="MsoNormal"><br>
<b>To:</b> OpenStack Development Mailing List<u></u><u></u></p>
</div>
<p class="MsoNormal"><b>Cc:</b> Ali Beddah<br>
<b>Subject:</b> Re: [openstack-dev] Announce of Rally - benchmarking system for OpenStack<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">On Oct 17, 2013 12:45 AM, "Boris Pavlovic" <<a href="mailto:bpavlovic@mirantis.com" target="_blank">bpavlovic@mirantis.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Hi Stackers,<br>
<br>
<br>
We are thrilled to present to you Rally, the benchmarking system for OpenStack.<br>
<br>
<br>
It is not a secret that we have performance & scaling issues and that OpenStack won’t scale out of box. It is also well known that if you get your super big DC (5k-15k servers) you are able to find & fix all OpenStack issues in few months (like Rackspace, BlueHost
& others have proved). So the problem with performance at scale is solvable.<br>
<br>
<br>
The main blocker to fix such issues in community is that there is no simple way to get relevant and repeatable “numbers” that represent OpenStack performance at scale. It is not enough to tune an individual OpenStack component, because its performance at scale
is no guarantee that it will not introduce a bottleneck somewhere else.<br>
<br>
<br>
The correct approach to comprehensively test OpenStack scalability, in our opinion, consists of the following four steps:<br>
<br>
1) Deploy OpenStack<br>
2) Create load by simultaneously making OpenStack API calls<br>
3) Collect performance and profile data<br>
4) Make data easy to consume by presenting it in a humanly readable form<br>
<br>
<br>
Rally is the system that implements all the steps above plus it maintains an extendable repository of standard performance tests. To use Rally, a user has to specify where to deploy OS, select the deployment mechanism (DevStack, Triple-O, Fuel, Etc.) and the
set of benchmarking tests to run. <br>
<br>
For more details and how to use it take a look at our wiki <a href="https://wiki.openstack.org/wiki/Rally" target="_blank">
https://wiki.openstack.org/wiki/Rally</a> it should already work out of box.<br>
<br>
<br>
Happy hunting!<br>
<br>
<br>
Links:<br>
<br>
1. Code: <a href="https://github.com/stackforge/rally" target="_blank">https://github.com/stackforge/rally</a><u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">2. Wiki:
<a href="https://wiki.openstack.org/wiki/Rally" target="_blank">https://wiki.openstack.org/wiki/Rally</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">2. Launchpad:
<a href="https://launchpad.net/rally" target="_blank">https://launchpad.net/rally</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">3. Statistics:
<a href="http://stackalytics.com/?release=havana&project_type=All&module=rally" target="_blank">
http://stackalytics.com/?release=havana&project_type=All&module=rally</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">4. RoadMap:
<a href="https://wiki.openstack.org/wiki/Rally/RoadMap" target="_blank">https://wiki.openstack.org/wiki/Rally/RoadMap</a><br>
<br>
<br>
Best regards,<br>
Boris Pavlovic<br>
---<br>
Mirantis Inc.<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br></div>