Open Stack

David,


Yes, that is the goal.
We should be able to get information about "how" it works, not only that it
works.

And get such interesting results:
https://wiki.openstack.org/wiki/Rally#How_influence_amqp_rpc_single_reply_queue_on_performance



Best regards,
Boris Pavlovic
---
Mirantis Inc.



On Fri, Oct 18, 2013 at 8:57 PM, David Kranz <dkranz at redhat.com> wrote:

>  On 10/18/2013 12:17 PM, Boris Pavlovic wrote:
>
> John,
>
>  Actually seems like a pretty good suggestion IMO, at least something
> worth some investigation and consideration before quickly discounting it.
>  Rather than "that's not what tempest is", maybe it's something tempest
> "could do".  Don't know, not saying one way or the other, just wondering if
> it's worth some investigation or thought.
>
>
>  These investigations I made before start working around Rally. It was
> about 3 months ago.
> It is not "quickly discounting" I didn't have yesterday time to make long
> response, so I will write it today:
>
>  I really don't like to make a copy of another projects, so I tried to
> reuse all projects & libs that we already have.
>
>  To explain why we shouldn't merge Rally & Tempest in one project (and
> should keep both)  we should analyze their use cases.
>
>
>  1. DevStack - one "click" and get your OpenStack cloud from sources
>
>  2. Tempest - one "click" and get your OpenStack Cloud verified
>
>  Both of these projects are great, because they are very useful and solve
> complicated tasks without "pain" for end user. (and I like them)
>
>  3. Rally is also one "click" system that solve OpenStack benchmarking.
>
>  To clarify situation. We should analyze what I mean by one "click"
> benchmarking and what are the use cases.
>
>  Use Cases:
> 1. Investigate how deployments influence on OS performance (find the set
> of good OpenStack deployment architectures)
> 2. Automatically get numbers & profiling info about how your changes
> influence on OS performance
> 3. Using Rally profiler detect scale & performance issues.
> Like here when we are trying to delete 3 VMs by one request they are
> deleted one by one because of DB lock on quotas table
> http://37.58.79.43:8080/traces/0011f252c9d98e31
> 4. Determine maximal load that could handle production cloud
>
>  To cover these cases we should actually test OpenStack deployments
> making simultaneously OpenStack API calls.
>
>  So to get results we have to:
> 1. Deploy OpenStack cloud somewhere. (Or get existing cloud)
> 2. Verify It
> 3. Run Benchmarks
> 4. Collect all results & present it in human readable form.
>
>
>  The goal of Rally was designed to automate these steps:
> 1.a Use existing cloud.
> 1.b.1 Automatically get (virtual) Servers from (soft layer, Amazon,
> RackSpace or you private public cloud, or OpenStack cloud)
> 1.b.2 DeployOpenStack on these servers from source (using Devstack, Anvli,
> Fuel or TrippleO...).
> 1.b.3 Patch this OpenStack with tomograph to get profiling information (I
> hope we will merge these patches into upstream).
> 2. Using tempest verify this cloud (we are going to switch from
> fuel-ostf-tests)
> 3. Run specified parametrized (to be able to make different load)
> benchmark scenarios
> 4. Collect all information about execution & present it in human readable
> form. (Tomograph, Zipking, matplotlib...)
>
>
>  So I am not sure that we should put inside Tempest Rally, because Rally
> use tempest. It is something like putting Nova into Cinder =)
> Also putting Tempest into Rally is not a good idea. (same as putting
> Cinder back to Nova).
>
>
>  Best regards,
> Boris Pavlovic
> ---
> Mirantis Inc.
>
>
> On Thu, Oct 17, 2013 at 11:56 PM, John Griffith <
> john.griffith at solidfire.com> wrote:
>
>>
>>
>>
>> On Thu, Oct 17, 2013 at 1:44 PM, Jay Pipes <jaypipes at gmail.com> wrote:
>>
>>> On 10/17/2013 03:32 PM, Boris Pavlovic wrote:
>>>
>>>> Jay,
>>>>
>>>>
>>>> Or, alternately, just have Rally as part of Tempest.
>>>>
>>>>
>>>> Actually, tempest is used only to verify that cloud works properly.
>>>> And verification is only small part of the Rally.
>>>>
>>>> At this moment we are using fuel-ostf-tests, but we are going to use
>>>> tempest to verify cloud.
>>>>
>>>
>>>  OK, cool... was just a suggestion :) Tempest has a set of stress tests
>>> [1] which are kind of related, which is the only reason I brought it up.
>>>
>>> Best,
>>> -jay
>>>
>>> [1] https://github.com/openstack/tempest/tree/master/tempest/stress
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>   Actually seems like a pretty good suggestion IMO, at least something
>> worth some investigation and consideration before quickly discounting it.
>>  Rather than "that's not what tempest is", maybe it's something tempest
>> "could do".  Don't know, not saying one way or the other, just wondering if
>> it's worth some investigation or thought.
>>
>>  By the way, VERY COOL!!
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> _______________________________________________
> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>  Thanks, Boris. This is really great. I took a look and there is a lot of
> similarity between the tempest stress framework and the rally benchmark
> driver and some code could be shared. But that code is only a very small
> part of each of these projects and IMO there is no reason to try and
> integrate the two more deeply. Rally will be beneficial to the OpenStack QA
> community which needs to grow beyond "OpenStackQA == tempest".
>
>  -David
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131019/e934140f/attachment.html>