<div dir="ltr">Guys, <div><br></div><div>I added patch to infra:</div><div><a href="https://review.openstack.org/#/c/143879/">https://review.openstack.org/#/c/143879/</a><br></div><div><br></div><div>That allows to run Rally against Mistral in gates.</div><div><br></div><div>Best regards,</div><div>Boris Pavlovic </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Dec 22, 2014 at 4:25 PM, Anastasia Kuznetsova <span dir="ltr"><<a href="mailto:akuznetsova@mirantis.com" target="_blank">akuznetsova@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dmitry,<div><br></div><div>Now I see that my comments are not so informative, I will try to describe environment and scenarios in more details.</div><div><br></div><div>1) <b style="margin:0px;padding:0px;font-family:Arial,sans-serif;font-size:11.8181819915771px;line-height:16px">1 api 1 engine 1 executor </b><span style="margin:0px;padding:0px;font-family:Arial,sans-serif;line-height:16px">it means that there were 3 Mistral processes running on the same box</span></div><div><span style="margin:0px;padding:0px;font-family:Arial,sans-serif;line-height:16px">2) list-workbooks scenario was run when there were no workflow executions at the same time, I will notice this your comment and I will measure time in such situation, but I guess that it will take more time, the question is as far as.</span></div><div><span style="margin:0px;padding:0px"><font face="Arial, sans-serif"><span style="line-height:16px">3) 60 % of success means that only 60 % of number of times execution of scenario 'list-workbooks' were successful, at the moment I have observed only one type of error: </span></font><br><font face="monospace, monospace">error connection to Rabbit : Error ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))<br></font>4) we don't know the </span><span style="font-size:13px">durability criteria of Mistral and under what load Mistral will 'die', we want to define the </span>threshold.</div><div><br></div><div>P.S. Dmitry, if you have any ideas/scenarios which you want to test, please share them.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Dec 20, 2014 at 9:35 AM, Dmitri Zimine <span dir="ltr"><<a href="mailto:dzimine@stackstorm.com" target="_blank">dzimine@stackstorm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>Anastasia, any start is a good start. </div><div><br></div><div><div><b style="margin:0px;padding:0px;font-family:Arial,sans-serif;font-size:11.8181819915771px;line-height:16px">> 1 api 1 engine 1 executor, list-workbooks</b></div></div><div><br></div><div>what exactly doest it mean: 1) is mistral deployed on 3 boxes with component per box, or all three are processes on the same box? 2) is list-workbooks test running while workflow executions going on? How many? what’s the character of the load 3) when it says 60% success what exactly does it mean, what kind of failures? 4) what is the durability criteria, how long do we expect Mistral to withstand the load. </div><div><br></div><div>Let’s discuss this in details on the next IRC meeting? </div><div><br></div><div>Thanks again for getting this started. </div><span><font color="#888888"><div><br></div><div>DZ.</div></font></span><div><div><div><br></div><div><div><font face="Arial, sans-serif"><span style="line-height:16px"><b><br></b></span></font><div><div>On Dec 19, 2014, at 7:44 AM, Anastasia Kuznetsova <<a href="mailto:akuznetsova@mirantis.com" target="_blank">akuznetsova@mirantis.com</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr">Boris,<div><br></div><div>Thanks for feedback! </div><div><font color="#741b47"><br></font></div><div><font color="#741b47"><span style="font-size:13px">> But I belive that you should put bigger load here: </span><a href="https://etherpad.openstack.org/p/mistral-rally-testing-results" style="font-size:13px" target="_blank">https://etherpad.openstack.org/p/mistral-rally-testing-results</a></font><br></div><div><br></div><div>As I said it is only beginning and I will increase the load and change its type. </div><div><br></div><div><div style="font-size:13px"><font color="#741b47">>As well concurrency should be at least 2-3 times bigger than times otherwise it won't generate proper load and you won't collect >enough data for statistical analyze. </font></div><div style="font-size:13px"><font color="#741b47">></font></div><div style="font-size:13px"><font color="#741b47">>As well use "rps" runner that generates more real life load.</font></div><div style="font-size:13px"><font color="#741b47">>Plus it will be nice to share as well output of "rally task report" command.</font></div></div><div style="font-size:13px"><br></div><div style="font-size:13px">Thanks for the advice, I will consider it in further testing and reporting.</div><div style="font-size:13px"><br></div><div style="font-size:13px">Answering to your question about using Rally for integration testing, as I mentioned in our load testing plan published on wiki page, one of our final goals is to have a Rally gate in one of Mistral repositories, so we are interested in it and I already prepare first commits to Rally.</div><div style="font-size:13px"><br></div><div style="font-size:13px">Thanks,</div><div style="font-size:13px">Anastasia Kuznetsova</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Dec 19, 2014 at 4:51 PM, Boris Pavlovic <span dir="ltr"><<a href="mailto:bpavlovic@mirantis.com" target="_blank">bpavlovic@mirantis.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Anastasia, <div><br></div><div>Nice work on this. But I belive that you should put bigger load here: <a href="https://etherpad.openstack.org/p/mistral-rally-testing-results" style="font-size:13px" target="_blank">https://etherpad.openstack.org/p/mistral-rally-testing-results</a></div><div><br></div><div>As well concurrency should be at least 2-3 times bigger than times otherwise it won't generate proper load and you won't collect enough data for statistical analyze. </div><div><br></div><div>As well use "rps" runner that generates more real life load.</div><div>Plus it will be nice to share as well output of "rally task report" command.</div><div><br></div><div><br></div><div>By the way what do you think about using Rally scenarios (that you already wrote) for integration testing as well? </div><div><br></div><div><br></div><div>Best regards,</div><div>Boris Pavlovic </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Dec 19, 2014 at 2:39 PM, Anastasia Kuznetsova <span dir="ltr"><<a href="mailto:akuznetsova@mirantis.com" target="_blank">akuznetsova@mirantis.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Hello everyone,</div><div><br></div><div>I want to announce that Mistral team has started work on load and performance testing in this release cycle.</div><div><br></div><div>Brief information about scope of our work can be found here: </div><div><a href="https://wiki.openstack.org/wiki/Mistral/Testing#Load_and_Performance_Testing" target="_blank">https://wiki.openstack.org/wiki/Mistral/Testing#Load_and_Performance_Testing</a></div><div><br></div><div>First results are published here:</div><div><a href="https://etherpad.openstack.org/p/mistral-rally-testing-results" target="_blank">https://etherpad.openstack.org/p/mistral-rally-testing-results</a></div><div><br></div><div>Thanks,</div><div>Anastasia Kuznetsova</div><div>@ Mirantis Inc.</div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div></div>
_______________________________________________<br>OpenStack-dev mailing list<br><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br></blockquote></div><br></div></div></div></div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>