<div dir="ltr">Boris, can You take a look on HA scenarios [0] to give a quick feedback on how it should be implemented with Rally in right way? Plugins/templates/how yaml should look like.<div>To make correct comparison of ostf and rally-based version.</div><div><div><br></div><div>Thanks!</div><div>[0] <a href="http://paste.openstack.org/show/523983/">http://paste.openstack.org/show/523983/</a><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 28, 2016 at 3:34 AM, Boris Pavlovic <span dir="ltr"><<a href="mailto:bpavlovic@mirantis.com" target="_blank">bpavlovic@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><span style="font-size:12.8px">Igor, </span></div><span class=""><span style="font-size:12.8px"><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px"><br></span></div></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span style="font-size:12.8px">I wonder what are the benefits of using </span><span style="font-size:12.8px;background-color:rgb(255,255,255)">Rally</span><span style="font-size:12.8px"> then? We can run whatever we want by means of MCollective or Ansible. Is there something that could help us? I don't know, maybe some dashboard integration with per test results or running by tests by tag?</span></blockquote><div><br></div></span><div>Benefits are in the Rally framework, engine and integrated tooling, that are doing very hard things to provide simple interfaces for writing simple plugins that are emulating complicated test cases.</div><div><br></div><div>The major benefits are next: </div><div><br></div><div><b>1) Generalization</b></div><div>1.1) One tool with one reporting system and one output format for all kinds of testing strategies (functional, load, perf, scale, ...)</div><div>1.2) One set of plugins (code) that can be used to generate all kinds of testing strategies </div><div>1.3.) One API for all kinds of testing strategies </div><div><br></div><div><b>2) Simplicity </b></div><div>2.1) Plugins are really simple to write, usually requires one method to be implemented </div><div>2.2) Auto discovery: adding plugins == add code in special (or specified) directory </div><div><br></div><div><b>3) Reusability & Data Driven approach: </b></div><div>3.1) Split code (plugins) & tests cases (yaml files)</div><div>3.2) Test cases is mixture of different plugins </div><div>3.3) Plugins accept arguments </div><div><b><br></b></div><div><b>4) Integrated tooling</b><br></div><div>4.1) All results are persisted in Rally DB and you can access it in any moment</div><div>4.2) Results can be exported in different formats (you can write even own plugins for simplifying integration)</div><div>4.3) Detailed HMTL reports with task results overview and trends <br></div><span class=""><div><br></div><div><br></div><div><br></div><div><br></div><div><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex" class="gmail_quote">Before switching Fuel from ostf to rally, I would like to see feature parity comparison. It's very necessary to understand how much work we need to spend to rewrite all our tests in rally way.</blockquote></div><div><br></div></span><div>Totally agree, let's do it. </div><div><br></div><div><br></div><div>Best regards,</div><div>Boris Pavlovic </div><div><div class="h5"><div><br></div><div> </div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 27, 2016 at 8:58 AM, Vladimir Kuklin <span dir="ltr"><<a href="mailto:vkuklin@mirantis.com" target="_blank">vkuklin@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">+1 to initial suggestion, but I guess we need to have a full feature equality (e.g. HA tests for mysql and rabbitmq replication) before switching to Rally.</div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Mon, Jun 27, 2016 at 6:17 PM, Sergii Golovatiuk <span dir="ltr"><<a href="mailto:sgolovatiuk@mirantis.com" target="_blank">sgolovatiuk@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>Before switching Fuel from ostf to rally, I would like to see feature parity comparison. It's very necessary to understand how much work we need to spend to rewrite all our tests in rally way.</div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div data-smartmail="gmail_signature"><div dir="ltr">--<br>
Best regards,<br>
Sergii Golovatiuk,<br>
Skype #golserge<br>
IRC #holser<br></div></div></div><div><div>
<br><div class="gmail_quote">On Mon, Jun 27, 2016 at 4:32 PM, Alexander Kostrikov <span dir="ltr"><<a href="mailto:akostrikov@mirantis.com" target="_blank">akostrikov@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hello, everybody!</div><div>Hello, Alex!</div><span>><span style="color:rgb(0,0,0);font-size:12.8px">I thought Rally was more for benchmarking. Wouldn't Tempest make more sense? </span></span><div><font color="#000000"><span style="font-size:12.8px">Rally is a good tool with nice api/usage/extensibility.<br></span></font><div><span style="font-size:12.8px;color:rgb(0,0,0)">I really liked "up and running tests in 5 minutes" in Rally with clear picture of what I am doing.</span><br></div></div><div><span style="font-size:12.8px;color:rgb(0,0,0)">So, I 100% for a Rally as a QA.</span><br></div><div><br></div><div>Another note:</div><div>We will need to implement some HA tests, probably not in Rally.</div></div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin <span dir="ltr"><<a href="mailto:akurilin@mirantis.com" target="_blank">akurilin@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky <span dir="ltr"><<a href="mailto:ikalnitsky@mirantis.com" target="_blank">ikalnitsky@mirantis.com</a>></span> wrote:<br></span><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span><br>
> On Jun 27, 2016, at 16:23, Alex Schultz <<a href="mailto:aschultz@mirantis.com" target="_blank">aschultz@mirantis.com</a>> wrote:<br>
><br>
> I thought Rally was more for benchmarking. Wouldn't Tempest make more sense?<br>
<br>
</span>According to Rally wiki page [1], it seems they have a verification layer (Tempest so far). Hm, I wonder does it mean we will need to rewrite our scenarios for Tempest?<br>
<br></blockquote><div><br></div></span><div>Rally consists of two main components: Rally Task and Rally Verification. They are totally separated.<br>Task component is fully pluggable and you can run there whatever you want in whatever you want way.<br></div><div>Verification component is hardcoded now. It was designed for managing(install, configure) and launching(execute and store results) Tempest. But we have a spec to make this component pluggable too.<br></div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
- igor<br>
<br>
<br>
[1] <a href="https://wiki.openstack.org/wiki/Rally" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/Rally</a><br>
<div><div>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></span></div><br><br clear="all"><span><br>-- <br><div data-smartmail="gmail_signature"><div dir="ltr">Best regards,<br>Andrey Kurilin.<br></div></div>
</span></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br></div></div><div data-smartmail="gmail_signature"><div dir="ltr"><p style="margin-bottom:0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">Kind Regards,<br></span></p><p style="margin-bottom:0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">Alexandr Kostrikov,<br></span></p><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"><br>Mirantis, Inc.</span><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)" lang="EN-US"></span>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">35b/3, Vorontsovskaya
St., 109147, Moscow, Russia</span><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)" lang="EN-US"></span></p>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"><br>
Tel.: <a href="tel:%2B7%20%28495%29%20640-49-04" value="+74956404904" target="_blank">+7 (495) 640-49-04</a><br>
Tel.: <a href="tel:%2B7%20%28906%29%20740-64-79" value="+79067406479" target="_blank">+7 (925) 716-64-52</a></span><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"></span></p>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">Skype: akostrikov_mirantis</span></p>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">E-mail:<span> </span></span><span style="font-size:10pt;font-family:Arial,sans-serif"><a href="mailto:elogutova@mirantis.com" target="_blank"><span style="color:rgb(17,85,204)" lang="EN-US"><span><span><span>akostrikov@mirantis.com</span></span></span></span></a></span><span style="font-family:Arial,sans-serif" lang="EN-US"></span></p>
<p style="margin-bottom:0.0001pt"><u><span style="font-size:10pt;font-family:Arial,sans-serif"><a href="http://www.mirantis.ru/" target="_blank"><span style="color:rgb(17,85,204)" lang="EN-US">www.mirantis.com</span></a></span></u><u><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"><br>
</span></u><u><span style="font-size:10pt;font-family:Arial,sans-serif"><a href="http://www.mirantis.ru/" target="_blank"><span style="color:rgb(17,85,204)" lang="EN-US">www.mirantis.ru</span></a></span></u><span style="font-family:Arial,sans-serif" lang="EN-US"></span></p></div></div>
</div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br></div></div><div data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Yours Faithfully,<br>Vladimir Kuklin,<br>Fuel Library Tech Lead,<br>Mirantis, Inc.<br><a href="tel:%2B7%20%28495%29%20640-49-04" value="+74956404904" target="_blank">+7 (495) 640-49-04</a><br><a href="tel:%2B7%20%28926%29%20702-39-68" value="+79267023968" target="_blank">+7 (926) 702-39-68</a><br>Skype kuklinvv<br>35bk3, Vorontsovskaya Str.<br>Moscow, Russia,<br><a href="http://www.mirantis.ru/" target="_blank">www.mirantis.com</a><br><a href="http://www.mirantis.ru/" target="_blank">www.mirantis.ru</a><br><a href="mailto:vkuklin@mirantis.com" target="_blank">vkuklin@mirantis.com</a></div></div></div></div>
</div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><p style="margin-bottom:0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">Kind Regards,<br></span></p><p style="margin-bottom:0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">Alexandr Kostrikov,<br></span></p><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"><br>Mirantis, Inc.</span><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)" lang="EN-US"></span>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">35b/3, Vorontsovskaya
St., 109147, Moscow, Russia</span><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)" lang="EN-US"></span></p>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"><br>
Tel.: <a href="tel:%2B7%20%28495%29%20640-49-04" value="+74956404904" target="_blank">+7 (495) 640-49-04</a><br>
Tel.: <a href="tel:%2B7%20%28906%29%20740-64-79" value="+79067406479" target="_blank">+7 (925) 716-64-52</a></span><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"></span></p>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">Skype: akostrikov_mirantis</span></p>
<p style="margin:0cm 0cm 0.0001pt"><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US">E-mail:<span> </span></span><span style="font-size:10pt;font-family:Arial,sans-serif"><a href="mailto:elogutova@mirantis.com" target="_blank"><span style="color:rgb(17,85,204)" lang="EN-US"><span><span><span>akostrikov@mirantis.com</span></span></span></span></a></span><span style="font-family:Arial,sans-serif" lang="EN-US"></span></p>
<p style="margin-bottom:0.0001pt"><u><span style="font-size:10pt;font-family:Arial,sans-serif"><a href="http://www.mirantis.ru/" target="_blank"><span style="color:rgb(17,85,204)" lang="EN-US">www.mirantis.com</span></a></span></u><u><span style="font-size:10pt;font-family:Arial,sans-serif" lang="EN-US"><br>
</span></u><u><span style="font-size:10pt;font-family:Arial,sans-serif"><a href="http://www.mirantis.ru/" target="_blank"><span style="color:rgb(17,85,204)" lang="EN-US">www.mirantis.ru</span></a></span></u><span style="font-family:Arial,sans-serif" lang="EN-US"></span></p></div></div>
</div>