Open Stack

Igor,


I wonder what are the benefits of using Rally then? We can run whatever we
> want by means of MCollective or Ansible. Is there something that could help
> us? I don't know, maybe some dashboard integration with per test results or
> running by tests by tag?


Benefits are in the Rally framework, engine and integrated tooling, that
are doing very hard things to provide simple interfaces for writing simple
plugins that are emulating complicated test cases.

The major benefits are next:

*1) Generalization*
1.1) One tool with one reporting system and one output format for all kinds
of testing strategies (functional, load, perf, scale, ...)
1.2) One set of plugins (code) that can be used to generate all kinds of
testing strategies
1.3.) One API for all kinds of testing strategies

*2) Simplicity *
2.1) Plugins are really simple to write, usually requires one method to be
implemented
2.2) Auto discovery: adding plugins == add code in special (or specified)
directory

*3) Reusability & Data Driven approach: *
3.1) Split code (plugins) & tests cases (yaml files)
3.2) Test cases is mixture of different plugins
3.3) Plugins accept arguments

*4) Integrated tooling*
4.1) All results are persisted in Rally DB and you can access it in any
moment
4.2) Results can be exported in different formats (you can write even own
plugins for simplifying integration)
4.3) Detailed HMTL reports with task results overview and trends




Before switching Fuel from ostf to rally, I would like to see feature
> parity comparison. It's very necessary to understand how much work we need
> to spend to rewrite all our tests in rally way.


Totally agree, let's do it.


Best regards,
Boris Pavlovic



On Mon, Jun 27, 2016 at 8:58 AM, Vladimir Kuklin <vkuklin at mirantis.com>
wrote:

> +1 to initial suggestion, but I guess we need to have a full feature
> equality (e.g. HA tests for mysql and rabbitmq replication) before
> switching to Rally.
>
> On Mon, Jun 27, 2016 at 6:17 PM, Sergii Golovatiuk <
> sgolovatiuk at mirantis.com> wrote:
>
>> Hi,
>>
>> Before switching Fuel from ostf to rally, I would like to see feature
>> parity comparison. It's very necessary to understand how much work we need
>> to spend to rewrite all our tests in rally way.
>>
>>
>>
>> --
>> Best regards,
>> Sergii Golovatiuk,
>> Skype #golserge
>> IRC #holser
>>
>> On Mon, Jun 27, 2016 at 4:32 PM, Alexander Kostrikov <
>> akostrikov at mirantis.com> wrote:
>>
>>> Hello, everybody!
>>> Hello, Alex!
>>> >I thought Rally was more for benchmarking.  Wouldn't Tempest make more
>>> sense?
>>> Rally is a good tool with nice api/usage/extensibility.
>>> I really liked "up and running tests in 5 minutes" in Rally with clear
>>> picture of what I am doing.
>>> So, I 100% for a Rally as a QA.
>>>
>>> Another note:
>>> We will need to implement some HA tests, probably not in Rally.
>>>
>>> On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin <akurilin at mirantis.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky <
>>>> ikalnitsky at mirantis.com> wrote:
>>>>
>>>>>
>>>>> > On Jun 27, 2016, at 16:23, Alex Schultz <aschultz at mirantis.com>
>>>>> wrote:
>>>>> >
>>>>> > I thought Rally was more for benchmarking.  Wouldn't Tempest make
>>>>> more sense?
>>>>>
>>>>> According to Rally wiki page [1], it seems they have a verification
>>>>> layer (Tempest so far). Hm, I wonder does it mean we will need to rewrite
>>>>> our scenarios for Tempest?
>>>>>
>>>>>
>>>> Rally consists of two main components: Rally Task and Rally
>>>> Verification. They are totally separated.
>>>> Task component is fully pluggable and you can run there whatever you
>>>> want in whatever you want way.
>>>> Verification component is hardcoded now. It was designed for
>>>> managing(install, configure) and launching(execute and store results)
>>>> Tempest. But we have a spec to make this component pluggable too.
>>>>
>>>>
>>>>> - igor
>>>>>
>>>>>
>>>>> [1] https://wiki.openstack.org/wiki/Rally
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey Kurilin.
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Kind Regards,
>>>
>>> Alexandr Kostrikov,
>>>
>>> Mirantis, Inc.
>>>
>>> 35b/3, Vorontsovskaya St., 109147, Moscow, Russia
>>>
>>>
>>> Tel.: +7 (495) 640-49-04
>>> Tel.: +7 (925) 716-64-52 <%2B7%20%28906%29%20740-64-79>
>>>
>>> Skype: akostrikov_mirantis
>>>
>>> E-mail: akostrikov at mirantis.com <elogutova at mirantis.com>
>>>
>>> *www.mirantis.com <http://www.mirantis.ru/>*
>>> *www.mirantis.ru <http://www.mirantis.ru/>*
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com <http://www.mirantis.ru/>
> www.mirantis.ru
> vkuklin at mirantis.com
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160627/9502fa6b/attachment.html>