<div dir="ltr"><div class="gmail_default" style="font-family:courier new,monospace"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Oct 18, 2013 at 10:17 AM, Boris Pavlovic <span dir="ltr"><<a href="mailto:boris@pavlovic.me" target="_blank">boris@pavlovic.me</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">John, <div class="im"><div><br></div><div><div style="font-size:13px;font-family:'courier new',monospace">
Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than "that's not what tempest is", maybe it's something tempest "could do". Don't know, not saying one way or the other, just wondering if it's worth some investigation or thought.</div>
</div><div><br></div><div><br></div></div><div>These investigations I made before start working around Rally. It was about 3 months ago. </div><div>It is not "quickly discounting" I didn't have yesterday time to make long response, so I will write it today: </div>
<div><br></div><div>I really don't like to make a copy of another projects, so I tried to reuse all projects & libs that we already have. <br></div><div><br></div><div>To explain why we shouldn't merge Rally & Tempest in one project (and should keep both) we should analyze their use cases. </div>
<div><br></div><div><br></div><div>1. DevStack - one "click" and get your OpenStack cloud from sources</div><div><br></div><div>2. Tempest - one "click" and get your OpenStack Cloud verified </div><div>
<br></div><div>Both of these projects are great, because they are very useful and solve complicated tasks without "pain" for end user. (and I like them)</div><div><br></div><div>3. Rally is also one "click" system that solve OpenStack benchmarking.</div>
<div><br></div><div>To clarify situation. We should analyze what I mean by one "click" benchmarking and what are the use cases.</div><div><br></div><div>Use Cases: </div><div>1. Investigate how deployments influence on OS performance (find the set of good OpenStack deployment architectures)</div>
<div>2. Automatically get numbers & profiling info about how your changes influence on OS performance</div><div>3. Using Rally profiler detect scale & performance issues.</div><div>Like here when we are trying to delete 3 VMs by one request they are deleted one by one because of DB lock on quotas table <a href="http://37.58.79.43:8080/traces/0011f252c9d98e31" target="_blank">http://37.58.79.43:8080/traces/0011f252c9d98e31</a></div>
<div>4. Determine maximal load that could handle production cloud</div><div><br></div><div>To cover these cases we should actually test OpenStack deployments making simultaneously OpenStack API calls. </div>
<div><br></div><div>So to get results we have to: </div><div>1. Deploy OpenStack cloud somewhere. (Or get existing cloud)</div><div>2. Verify It</div><div>3. Run Benchmarks </div><div>4. Collect all results & present it in human readable form. </div>
<div><br></div><div><br></div><div>The goal of Rally was designed to automate these steps:</div><div>1.a Use existing cloud. </div><div>1.b.1 Automatically get (virtual) Servers from (soft layer, Amazon, RackSpace or you private public cloud, or OpenStack cloud)</div>
<div>1.b.2 DeployOpenStack on these servers from source (using Devstack, Anvli, Fuel or TrippleO...). </div><div>1.b.3 Patch this OpenStack with tomograph to get profiling information (I hope we will merge these patches into upstream). </div>
<div>2. Using tempest verify this cloud (we are going to switch from fuel-ostf-tests)</div><div>3. Run specified parametrized (to be able to make different load) benchmark scenarios </div><div>4. Collect all information about execution & present it in human readable form. (Tomograph, Zipking, matplotlib...)</div>
<div><br></div><div><br></div><div>So I am not sure that we should put inside Tempest Rally, because Rally use tempest. It is something like putting Nova into Cinder =)</div><div>Also putting Tempest into Rally is not a good idea. (same as putting Cinder back to Nova).</div>
<div class="im">
<div><br></div><div><br></div><div>Best regards,</div><div>Boris Pavlovic</div><div>---</div><div>Mirantis Inc. </div><div><br></div></div><div><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 17, 2013 at 11:56 PM, John Griffith <span dir="ltr"><<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div><div style="font-family:'courier new',monospace"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Oct 17, 2013 at 1:44 PM, Jay Pipes <span dir="ltr"><<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div>On 10/17/2013 03:32 PM, Boris Pavlovic wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Jay,<br>
<br>
<br>
Or, alternately, just have Rally as part of Tempest.<br>
<br>
<br>
Actually, tempest is used only to verify that cloud works properly.<br>
And verification is only small part of the Rally.<br>
<br>
At this moment we are using fuel-ostf-tests, but we are going to use<br>
tempest to verify cloud.<br>
</blockquote>
<br></div>
OK, cool... was just a suggestion :) Tempest has a set of stress tests [1] which are kind of related, which is the only reason I brought it up.<br>
<br>
Best,<br>
-jay<br>
<br>
[1] <a href="https://github.com/openstack/tempest/tree/master/tempest/stress" target="_blank">https://github.com/openstack/<u></u>tempest/tree/master/tempest/<u></u>stress</a><div><div><br>
<br>
______________________________<u></u>_________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.<u></u>org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
</div></div></blockquote></div><br></div></div></div><div class="gmail_extra"><div style="font-family:'courier new',monospace">Actually seems like a pretty good suggestion IMO, at least something worth some investigation and consideration before quickly discounting it. Rather than "that's not what tempest is", maybe it's something tempest "could do". Don't know, not saying one way or the other, just wondering if it's worth some investigation or thought.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">By the way, VERY COOL!!</div><br></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><div class="gmail_default" style="font-family:'courier new',monospace">Boris,</div><div class="gmail_default" style="font-family:'courier new',monospace"><br></div><div class="gmail_default" style="font-family:'courier new',monospace">
Great, thanks for the further explanation. I was in no way saying that *you* discounted it quickly, but at first glance I personally thought it was worth looking at. I mostly wanted to point out the consideration here and your detailed response here pretty much answers the questions that I had. I appreciate the details.</div>
<div class="gmail_default" style="font-family:'courier new',monospace"><br></div><div class="gmail_default" style="font-family:'courier new',monospace">John</div><br></div></div>