<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Thanks Boris!<div class=""><br class=""><div apple-content-edited="true" class="">
<div class="">Renat Akhmerov</div><div class="">@ Mirantis Inc.</div><div class=""><br class=""></div>
</div>
<br class=""><div><blockquote type="cite" class=""><div class="">On 24 Dec 2014, at 23:54, Boris Pavlovic <<a href="mailto:bpavlovic@mirantis.com" class="">bpavlovic@mirantis.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Guys, <div class=""><br class=""></div><div class="">I added patch to infra:</div><div class=""><a href="https://review.openstack.org/#/c/143879/" class="">https://review.openstack.org/#/c/143879/</a><br class=""></div><div class=""><br class=""></div><div class="">That allows to run Rally against Mistral in gates.</div><div class=""><br class=""></div><div class="">Best regards,</div><div class="">Boris Pavlovic </div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Dec 22, 2014 at 4:25 PM, Anastasia Kuznetsova <span dir="ltr" class=""><<a href="mailto:akuznetsova@mirantis.com" target="_blank" class="">akuznetsova@mirantis.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="">Dmitry,<div class=""><br class=""></div><div class="">Now I see that my comments are not so informative, I will try to describe environment and scenarios in more details.</div><div class=""><br class=""></div><div class="">1) <b style="margin:0px;padding:0px;font-family:Arial,sans-serif;font-size:11.8181819915771px;line-height:16px" class="">1 api 1 engine 1 executor </b><span style="margin:0px;padding:0px;font-family:Arial,sans-serif;line-height:16px" class="">it means that there were 3 Mistral processes running on the same box</span></div><div class=""><span style="margin:0px;padding:0px;font-family:Arial,sans-serif;line-height:16px" class="">2) list-workbooks scenario was run when there were no workflow executions at the same time, I will notice this your comment and I will measure time in such situation, but I guess that it will take more time, the question is as far as.</span></div><div class=""><span style="margin:0px;padding:0px" class=""><font face="Arial, sans-serif" class=""><span style="line-height:16px" class="">3) 60 % of success means that only 60 % of number of times execution of scenario 'list-workbooks' were successful, at the moment I have observed only one type of error: </span></font><br class=""><font face="monospace, monospace" class="">error connection to Rabbit : Error ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))<br class=""></font>4) we don't know the </span><span style="font-size:13px" class="">durability criteria of Mistral and under what load Mistral will 'die', we want to define the </span>threshold.</div><div class=""><br class=""></div><div class="">P.S. Dmitry, if you have any ideas/scenarios which you want to test, please share them.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br class=""><div class="gmail_quote">On Sat, Dec 20, 2014 at 9:35 AM, Dmitri Zimine <span dir="ltr" class=""><<a href="mailto:dzimine@stackstorm.com" target="_blank" class="">dzimine@stackstorm.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class="">Anastasia, any start is a good start. </div><div class=""><br class=""></div><div class=""><div class=""><b style="margin:0px;padding:0px;font-family:Arial,sans-serif;font-size:11.8181819915771px;line-height:16px" class="">> 1 api 1 engine 1 executor, list-workbooks</b></div></div><div class=""><br class=""></div><div class="">what exactly doest it mean: 1) is mistral deployed on 3 boxes with component per box, or all three are processes on the same box? 2) is list-workbooks test running while workflow executions going on? How many? what’s the character of the load 3) when it says 60% success what exactly does it mean, what kind of failures? 4) what is the durability criteria, how long do we expect Mistral to withstand the load. </div><div class=""><br class=""></div><div class="">Let’s discuss this in details on the next IRC meeting? </div><div class=""><br class=""></div><div class="">Thanks again for getting this started. </div><span class=""><font color="#888888" class=""><div class=""><br class=""></div><div class="">DZ.</div></font></span><div class=""><div class=""><div class=""><br class=""></div><div class=""><div class=""><font face="Arial, sans-serif" class=""><span style="line-height:16px" class=""><b class=""><br class=""></b></span></font><div class=""><div class="">On Dec 19, 2014, at 7:44 AM, Anastasia Kuznetsova <<a href="mailto:akuznetsova@mirantis.com" target="_blank" class="">akuznetsova@mirantis.com</a>> wrote:</div><br class=""><blockquote type="cite" class=""><div dir="ltr" class="">Boris,<div class=""><br class=""></div><div class="">Thanks for feedback! </div><div class=""><font color="#741b47" class=""><br class=""></font></div><div class=""><font color="#741b47" class=""><span style="font-size:13px" class="">> But I belive that you should put bigger load here: </span><a href="https://etherpad.openstack.org/p/mistral-rally-testing-results" style="font-size:13px" target="_blank" class="">https://etherpad.openstack.org/p/mistral-rally-testing-results</a></font><br class=""></div><div class=""><br class=""></div><div class="">As I said it is only beginning and I will increase the load and change its type. </div><div class=""><br class=""></div><div class=""><div style="font-size:13px" class=""><font color="#741b47" class="">>As well concurrency should be at least 2-3 times bigger than times otherwise it won't generate proper load and you won't collect >enough data for statistical analyze. </font></div><div style="font-size:13px" class=""><font color="#741b47" class="">></font></div><div style="font-size:13px" class=""><font color="#741b47" class="">>As well use "rps" runner that generates more real life load.</font></div><div style="font-size:13px" class=""><font color="#741b47" class="">>Plus it will be nice to share as well output of "rally task report" command.</font></div></div><div style="font-size:13px" class=""><br class=""></div><div style="font-size:13px" class="">Thanks for the advice, I will consider it in further testing and reporting.</div><div style="font-size:13px" class=""><br class=""></div><div style="font-size:13px" class="">Answering to your question about using Rally for integration testing, as I mentioned in our load testing plan published on wiki page, one of our final goals is to have a Rally gate in one of Mistral repositories, so we are interested in it and I already prepare first commits to Rally.</div><div style="font-size:13px" class=""><br class=""></div><div style="font-size:13px" class="">Thanks,</div><div style="font-size:13px" class="">Anastasia Kuznetsova</div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, Dec 19, 2014 at 4:51 PM, Boris Pavlovic <span dir="ltr" class=""><<a href="mailto:bpavlovic@mirantis.com" target="_blank" class="">bpavlovic@mirantis.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="">Anastasia, <div class=""><br class=""></div><div class="">Nice work on this. But I belive that you should put bigger load here: <a href="https://etherpad.openstack.org/p/mistral-rally-testing-results" style="font-size:13px" target="_blank" class="">https://etherpad.openstack.org/p/mistral-rally-testing-results</a></div><div class=""><br class=""></div><div class="">As well concurrency should be at least 2-3 times bigger than times otherwise it won't generate proper load and you won't collect enough data for statistical analyze. </div><div class=""><br class=""></div><div class="">As well use "rps" runner that generates more real life load.</div><div class="">Plus it will be nice to share as well output of "rally task report" command.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">By the way what do you think about using Rally scenarios (that you already wrote) for integration testing as well? </div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Best regards,</div><div class="">Boris Pavlovic </div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, Dec 19, 2014 at 2:39 PM, Anastasia Kuznetsova <span dir="ltr" class=""><<a href="mailto:akuznetsova@mirantis.com" target="_blank" class="">akuznetsova@mirantis.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr" class=""><div class="">Hello everyone,</div><div class=""><br class=""></div><div class="">I want to announce that Mistral team has started work on load and performance testing in this release cycle.</div><div class=""><br class=""></div><div class="">Brief information about scope of our work can be found here: </div><div class=""><a href="https://wiki.openstack.org/wiki/Mistral/Testing#Load_and_Performance_Testing" target="_blank" class="">https://wiki.openstack.org/wiki/Mistral/Testing#Load_and_Performance_Testing</a></div><div class=""><br class=""></div><div class="">First results are published here:</div><div class=""><a href="https://etherpad.openstack.org/p/mistral-rally-testing-results" target="_blank" class="">https://etherpad.openstack.org/p/mistral-rally-testing-results</a></div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Anastasia Kuznetsova</div><div class="">@ Mirantis Inc.</div></div>
<br class="">_______________________________________________<br class="">
OpenStack-dev mailing list<br class="">
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank" class="">OpenStack-dev@lists.openstack.org</a><br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
<br class=""></blockquote></div></div>
<br class="">_______________________________________________<br class="">
OpenStack-dev mailing list<br class="">
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank" class="">OpenStack-dev@lists.openstack.org</a><br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
<br class=""></blockquote></div></div>
_______________________________________________<br class="">OpenStack-dev mailing list<br class=""><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank" class="">OpenStack-dev@lists.openstack.org</a><br class=""><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class=""></blockquote></div><br class=""></div></div></div></div></div><br class="">_______________________________________________<br class="">
OpenStack-dev mailing list<br class="">
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank" class="">OpenStack-dev@lists.openstack.org</a><br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
<br class=""></blockquote></div><br class=""></div>
</div></div><br class="">_______________________________________________<br class="">
OpenStack-dev mailing list<br class="">
<a href="mailto:OpenStack-dev@lists.openstack.org" class="">OpenStack-dev@lists.openstack.org</a><br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
<br class=""></blockquote></div><br class=""></div>
_______________________________________________<br class="">OpenStack-dev mailing list<br class=""><a href="mailto:OpenStack-dev@lists.openstack.org" class="">OpenStack-dev@lists.openstack.org</a><br class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br class=""></div></blockquote></div><br class=""></div></body></html>