Open Stack

On 08/19/2013 10:28 AM, Tal Kammer wrote:
> I wanted to start building some scenario based testing and noticed that there are some overlap of code where some scenarios re-implement code that is also present in the API.
> For example, some of the scenarios (test_server_basic_ops.py for example) re-implement methods that were written in the corresponding REST client in the API.
> I was wondering if there is some guidelines of what is a resalable overlap of code (though I'm not sure it really is an overlap when scenario uses the python clients and the API the REST client), for example, say I want to write a scenario for nova that tests something with the quota, I can see from the code of the API (tempest/tempest/api/compute/admin/test_quotas.py) that it has a lot of the quota related code already implemented with the REST client, so when I'm building a scenario, how much "freedom" do I have to write methods that uses the python clients instead of the API methods?

there is an attempt to describe the scope of the scenario tests in the 
doc[1]

my understanding is that scenario tests, by using the official API 
clients (and not the tempest clients), will be a tool for testing the 
official clients too and not only the target services, so some code 
duplication in the test cases is acceptable

in addition to that, scenario tests should check for integration between 
projects and try to reproduce some sequence of steps possibly performed 
by end users so, in contrast with an API suite which tries to test 
extensively all the quota calls (eventually with negative tests too), a 
scenario test should probably just create some resource until a quota 
enforcement is hit and then adjust the quota setting and maybe try again 
the initial operation to ensure it succeeds

1. http://docs.openstack.org/developer/tempest/field_guide/scenario.html
-- 
Giulio Fidente
GPG KEY: 08D733BA | IRC: giulivo