Open Stack

Hello Boris,

Rally seemes to be a really interesting tool and I will eventually using it for full scale benchmarking.

However I will focus on tempest based tests for the time being in order to have these tests replayed regularly during the integration process.
I believe that my needs are covered by both Tempest and Rally and if time allow me to do so, I will submit the same kind of scenario for full scale benchmarking.

Best Regards,

Julien LELOUP
julien.leloup at 3ds.com

From: LELOUP Julien [mailto:Julien.LELOUP at 3ds.com]
Sent: Tuesday, January 21, 2014 11:34 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Tempest - Stress test] : cleanup() removing resources for all tenants with an admin_manager

Hello Boris,

I'll check Rally in order to see what tool is the best for my tests.

Best Regards,

Julien LELOUP
julien.leloup at 3ds.com<mailto:julien.leloup at 3ds.com>


From: Boris Pavlovic [mailto:bpavlovic at mirantis.com]
Sent: Monday, January 20, 2014 5:52 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Tempest - Stress test] : cleanup() removing resources for all tenants with an admin_manager

Julien,

Probably you should try to use Rally for benchmarking.
https://wiki.openstack.org/wiki/Rally

There is already working generic cleanup...

There is already implemented framework that allows parametrized benchmarks:
https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L32-L39

Simple way to configure load using json (load will be created from real users, no admin, that will be pre created for each benchmark):
https://github.com/stackforge/rally/blob/master/doc/samples/tasks/nova/boot-and-delete.json


And simple CLI interface (now we are working around Web UI)


Best regards,
Boris Pavlovic

On Mon, Jan 20, 2014 at 8:32 PM, LELOUP Julien <Julien.LELOUP at 3ds.com<mailto:Julien.LELOUP at 3ds.com>> wrote:
Hi everyone,

I'm forwarding my own email previously posted on the QA list.

I would like to discuss about the cleanup() process used right after a stress test run in Tempest.

For what I see now by using it and by reading the code, the cleanup() seems a bit rough since it is using an "admin_manager" in order to get all kind of test resources actually available : servers, key pairs, volumes, .etc...
More precisely, when it comes to clean servers, it is searching for servers on all tenants. I find this behavior a little rough since it will blow all objects on the target OpenStack, even object unrelated to the stress tests that just ran.

Actually before reading the cleanup() I had a problem when one of my stress test erased all the servers and volumes on another tenant, which impaired other people working on our OpenStack.

I can imagine that for some scenarios, using an admin user to deeply clean an OpenStack is required, but I believe that most of the time the cleanup() process should focus only on the tenant used during the stress test and leave the other tenants alone.

Am I doing something wrong ? Is there a way to restrain the cleanup() process ?

If no parameters or configuration allows me to do so, should I improve the cleanup() code in order to allow it to remove only the test resources created for the test?
I do not wish to make this kind of code if the OpenStack community believe that the present behavior is totally intended and should not be modified.


Best Regards,

Julien LELOUP
julien.leloup at 3ds.com<mailto:julien.leloup at 3ds.com>

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140127/2d9ff69d/attachment.html>