<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style>
<!--
@font-face
{font-family:Calibri}
@font-face
{font-family:Tahoma}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
p
{margin-right:0in;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif"}
span.BalloonTextChar
{font-family:"Tahoma","sans-serif"}
p.msochpdefault, li.msochpdefault, div.msochpdefault
{margin-right:0in;
margin-left:0in;
font-size:12.0pt;
font-family:"Calibri","sans-serif"}
span.emailstyle18
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.balloontextchar0
{font-family:"Tahoma","sans-serif"}
span.EmailStyle23
{font-family:"Calibri","sans-serif";
color:#1F497D}
.MsoChpDefault
{font-size:10.0pt;
font-family:"Calibri","sans-serif"}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.WordSection1
{}
-->
</style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Hello Boris,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Rally seemes to be a really interesting tool and I will eventually using it for full scale benchmarking.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">However I will focus on tempest based tests for the time being in order to have these tests replayed regularly during the integration process.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">I believe that my needs are covered by both Tempest and Rally and if time allow me to do so, I will submit the same kind of scenario for full scale benchmarking.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Best Regards,
</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="FR" style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Julien LELOUP</span></p>
<p class="MsoNormal"><span lang="FR" style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">julien.leloup@3ds.com</span></p>
<p class="MsoNormal"><span lang="FR" style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> LELOUP Julien [mailto:Julien.LELOUP@3ds.com]
<br>
<b>Sent:</b> Tuesday, January 21, 2014 11:34 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [Tempest - Stress test] : cleanup() removing resources for all tenants with an admin_manager</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Hello Boris,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">I’ll check Rally in order to see what tool is the best for my tests.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Best Regards,
</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="FR" style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Julien LELOUP</span></p>
<p class="MsoNormal"><span lang="FR" style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"><a href="mailto:julien.leloup@3ds.com"><span lang="EN-US">julien.leloup@3ds.com</span></a></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> Boris Pavlovic [<a href="mailto:bpavlovic@mirantis.com">mailto:bpavlovic@mirantis.com</a>]
<br>
<b>Sent:</b> Monday, January 20, 2014 5:52 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [Tempest - Stress test] : cleanup() removing resources for all tenants with an admin_manager</span></p>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">Julien,</p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Probably you should try to use Rally for benchmarking.</p>
</div>
<div>
<p class="MsoNormal"><a href="https://wiki.openstack.org/wiki/Rally">https://wiki.openstack.org/wiki/Rally</a></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">There is already working generic cleanup...</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">There is already implemented framework that allows parametrized benchmarks:</p>
</div>
<div>
<p class="MsoNormal"><a href="https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L32-L39">https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L32-L39</a></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Simple way to configure load using json (load will be created from real users, no admin, that will be pre created for each benchmark):</p>
</div>
<div>
<p class="MsoNormal"><a href="https://github.com/stackforge/rally/blob/master/doc/samples/tasks/nova/boot-and-delete.json">https://github.com/stackforge/rally/blob/master/doc/samples/tasks/nova/boot-and-delete.json</a></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">And simple CLI interface (now we are working around Web UI)</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Best regards,</p>
</div>
<div>
<p class="MsoNormal">Boris Pavlovic </p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"> </p>
<div>
<p class="MsoNormal">On Mon, Jan 20, 2014 at 8:32 PM, LELOUP Julien <<a href="mailto:Julien.LELOUP@3ds.com" target="_blank">Julien.LELOUP@3ds.com</a>> wrote:</p>
<div>
<div>
<p class="MsoNormal">Hi everyone,</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I’m forwarding my own email previously posted on the QA list.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I would like to discuss about the cleanup() process used right after a stress test run in Tempest.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">For what I see now by using it and by reading the code, the cleanup() seems a bit rough since it is using an “admin_manager” in order to get all kind of test resources actually available : servers, key pairs, volumes, .etc…</p>
<p class="MsoNormal">More precisely, when it comes to clean servers, it is searching for servers on all tenants. I find this behavior a little rough since it will blow all objects on the target OpenStack, even object unrelated to the stress tests that just
ran.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Actually before reading the cleanup() I had a problem when one of my stress test erased all the servers and volumes on another tenant, which impaired other people working on our OpenStack.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I can imagine that for some scenarios, using an admin user to deeply clean an OpenStack is required, but I believe that most of the time the cleanup() process should focus only on the tenant used during the stress test and leave the other
tenants alone.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Am I doing something wrong ? Is there a way to restrain the cleanup() process ?</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">If no parameters or configuration allows me to do so, should I improve the cleanup() code in order to allow it to remove only the test resources created for the test?</p>
<p class="MsoNormal">I do not wish to make this kind of code if the OpenStack community believe that the present behavior is totally intended and should not be modified.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Best Regards, </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span lang="FR">Julien LELOUP</span></p>
<p class="MsoNormal"><span lang="FR"><a href="mailto:julien.leloup@3ds.com" target="_blank">julien.leloup@3ds.com</a></span></p>
</div>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">If you are not one of the named recipients or have received this email in error,
</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">(i) you should not read, disclose, or copy it,</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">(ii) please notify sender of your receipt by reply email and delete this email and all attachments,</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">For other languages, go to
<a href="http://www.3ds.com/terms/email-disclaimer" target="_blank">http://www.3ds.com/terms/email-disclaimer</a>
</span></i></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></p>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">If you are not one of the named recipients or have received this email in error,
</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">(i) you should not read, disclose, or copy it,</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">(ii) please notify sender of your receipt by reply email and delete this email and all attachments,</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.</span></i></p>
<p style="margin-right:0in; margin-bottom:0in; margin-left:35.4pt; margin-bottom:.0001pt">
<i><span style="font-size:9.0pt; font-family:"Arial","sans-serif"; color:#9D9D9D">For other languages, go to
<a href="http://www.3ds.com/terms/email-disclaimer">http://www.3ds.com/terms/email-disclaimer</a>
</span></i></p>
</div>
<p style="font-size:9pt; margin:0px 0px 0px 35.4pt; color:#9d9d9d; font-style:italic; font-family:Arial,Helvetica,sans-serif">
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.</p>
<p style="font-size:9pt; margin:0px 0px 0px 35.4pt; color:#9d9d9d; font-style:italic; font-family:Arial,Helvetica,sans-serif">
If you are not one of the named recipients or have received this email in error, </p>
<p style="font-size:9pt; margin:0px 0px 0px 35.4pt; color:#9d9d9d; font-style:italic; font-family:Arial,Helvetica,sans-serif">
(i) you should not read, disclose, or copy it,</p>
<p style="font-size:9pt; margin:0px 0px 0px 35.4pt; color:#9d9d9d; font-style:italic; font-family:Arial,Helvetica,sans-serif">
(ii) please notify sender of your receipt by reply email and delete this email and all attachments,</p>
<p style="font-size:9pt; margin:0px 0px 0px 35.4pt; color:#9d9d9d; font-style:italic; font-family:Arial,Helvetica,sans-serif">
(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.</p>
<p style="font-style:italic; margin:0px 0px 0px 35.4pt; font-family:Arial,Helvetica,sans-serif; color:#9d9d9d; font-size:9pt">
</p>
<p style="font-size:9pt; margin:0px 0px 0px 35.4pt; color:#9d9d9d; font-style:italic; font-family:Arial,Helvetica,sans-serif">
For other languages, go to http://www.3ds.com/terms/email-disclaimer </p>
</body>
</html>