Open Stack

On Fri, Oct 18, 2013 at 10:17 AM, Boris Pavlovic <boris at pavlovic.me> wrote:

> John,
>
> Actually seems like a pretty good suggestion IMO, at least something worth
> some investigation and consideration before quickly discounting it.  Rather
> than "that's not what tempest is", maybe it's something tempest "could do".
>  Don't know, not saying one way or the other, just wondering if it's worth
> some investigation or thought.
>
>
> These investigations I made before start working around Rally. It was
> about 3 months ago.
> It is not "quickly discounting" I didn't have yesterday time to make long
> response, so I will write it today:
>
> I really don't like to make a copy of another projects, so I tried to
> reuse all projects & libs that we already have.
>
> To explain why we shouldn't merge Rally & Tempest in one project (and
> should keep both)  we should analyze their use cases.
>
>
> 1. DevStack - one "click" and get your OpenStack cloud from sources
>
> 2. Tempest - one "click" and get your OpenStack Cloud verified
>
> Both of these projects are great, because they are very useful and solve
> complicated tasks without "pain" for end user. (and I like them)
>
> 3. Rally is also one "click" system that solve OpenStack benchmarking.
>
> To clarify situation. We should analyze what I mean by one "click"
> benchmarking and what are the use cases.
>
> Use Cases:
> 1. Investigate how deployments influence on OS performance (find the set
> of good OpenStack deployment architectures)
> 2. Automatically get numbers & profiling info about how your changes
> influence on OS performance
> 3. Using Rally profiler detect scale & performance issues.
> Like here when we are trying to delete 3 VMs by one request they are
> deleted one by one because of DB lock on quotas table
> http://37.58.79.43:8080/traces/0011f252c9d98e31
> 4. Determine maximal load that could handle production cloud
>
> To cover these cases we should actually test OpenStack deployments making
> simultaneously OpenStack API calls.
>
> So to get results we have to:
> 1. Deploy OpenStack cloud somewhere. (Or get existing cloud)
> 2. Verify It
> 3. Run Benchmarks
> 4. Collect all results & present it in human readable form.
>
>
> The goal of Rally was designed to automate these steps:
> 1.a Use existing cloud.
> 1.b.1 Automatically get (virtual) Servers from (soft layer, Amazon,
> RackSpace or you private public cloud, or OpenStack cloud)
> 1.b.2 DeployOpenStack on these servers from source (using Devstack, Anvli,
> Fuel or TrippleO...).
> 1.b.3 Patch this OpenStack with tomograph to get profiling information (I
> hope we will merge these patches into upstream).
> 2. Using tempest verify this cloud (we are going to switch from
> fuel-ostf-tests)
> 3. Run specified parametrized (to be able to make different load)
> benchmark scenarios
> 4. Collect all information about execution & present it in human readable
> form. (Tomograph, Zipking, matplotlib...)
>
>
> So I am not sure that we should put inside Tempest Rally, because Rally
> use tempest. It is something like putting Nova into Cinder =)
> Also putting Tempest into Rally is not a good idea. (same as putting
> Cinder back to Nova).
>
>
> Best regards,
> Boris Pavlovic
> ---
> Mirantis Inc.
>
>
> On Thu, Oct 17, 2013 at 11:56 PM, John Griffith <
> john.griffith at solidfire.com> wrote:
>
>>
>>
>>
>> On Thu, Oct 17, 2013 at 1:44 PM, Jay Pipes <jaypipes at gmail.com> wrote:
>>
>>> On 10/17/2013 03:32 PM, Boris Pavlovic wrote:
>>>
>>>> Jay,
>>>>
>>>>
>>>> Or, alternately, just have Rally as part of Tempest.
>>>>
>>>>
>>>> Actually, tempest is used only to verify that cloud works properly.
>>>> And verification is only small part of the Rally.
>>>>
>>>> At this moment we are using fuel-ostf-tests, but we are going to use
>>>> tempest to verify cloud.
>>>>
>>>
>>> OK, cool... was just a suggestion :) Tempest has a set of stress tests
>>> [1] which are kind of related, which is the only reason I brought it up.
>>>
>>> Best,
>>> -jay
>>>
>>> [1] https://github.com/openstack/**tempest/tree/master/tempest/**stress<https://github.com/openstack/tempest/tree/master/tempest/stress>
>>>
>>>
>>> ______________________________**_________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.**org <OpenStack-dev at lists.openstack.org>
>>> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>>
>>
>> Actually seems like a pretty good suggestion IMO, at least something
>> worth some investigation and consideration before quickly discounting it.
>>  Rather than "that's not what tempest is", maybe it's something tempest
>> "could do".  Don't know, not saying one way or the other, just wondering if
>> it's worth some investigation or thought.
>>
>> By the way, VERY COOL!!
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> Boris,

Great, thanks for the further explanation.  I was in no way saying that
*you* discounted it quickly, but at first glance I personally thought it
was worth looking at.  I mostly wanted to point out the consideration here
and your detailed response here pretty much answers the questions that I
had.  I appreciate the details.

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131018/230afdec/attachment.html>