Open Stack

On 03/26/2013 09:27 PM, Yaniv Kaul wrote:
> Without the ability to configure OpenStack,  it's not going to fly.  Configure means:
> 0. Installation scenarios (installation of different components on different hosts) 
> 1. Ini files manipulation (nova.conf and friends)  and restarting services
> 2. JSON files manipulation (policy files)
> 3. Configuration via the API (and/or nova-manage)
> 
> The above are basic blocks to any true scenario (and end to end) testing.
> Y. 

Tempest does not need to do any of the above. Tempest assumes that
environment has been set up already, and executes its tests for expected
behaviour regardless of the configuration of the environment.

Projects like devstack-gate (which runs devstack and configures the
environments used in continuous integration testing) also write
Tempest's configuration file, which tells Tempest certain key details
about the environment it will execute against, including capabilities
like ability to resize images, perform migrations, etc.

Now what *would* enable better support for scenario testing would be
API-based discovery of an OpenStack environment's capabilities and
configuration settings... that would enable Tempest to customize some of
its settings based on what it can query the OpenStack endpoints for.

Best,
-jay

> Masayuki Igawa <igawa at mxs.nes.nec.co.jp> wrote:
> 
> Hi,
> 
> Here is a blueprint I've registered.
>   https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests
> 
> This blueprint proposes to add a new category "scenario tests" in Tempest.
> Scenario tests are for testing across the components such as Nova, Keystone, Glance and so on.
> By scenario tests, developers will be able to check whether a new code (bug-fixes or features) will cause a side effect to the other components.
> We will be able to check OpenStack basic functions by common use case scenarios.
> I'd like to propose a minimum basic scenario test that must work at any OpenStack environments.
> 
> Implementation
> ===============
> * add tempest/tests/scenario directory
> * add scenario tests in the directory like following
> 
> """
> Scenario-1: very small cloud, basic life-cycle scenario, single-flavor, single-image, single-tenant...
> - for admin user
> 1. Create a flavor
> 2. Create a image
> 3. Configure network settings
> 4. Create & configure a member role, a project, a quota, a user
> - for regular user
> 5. Create a key-pair
> 6. Boot a instance. List & show the instance
> 7. Create a volume. List & show the volume
> 8. Attach the volume
> 9. Detach the volume
> 10. Delete the instance
> - for admin user
> 11. Delete the user, the role, the network, the image, the tenant, and the flavor
> """
> 
> We think that the scenario tests can raise the use case coverage that is important for user's viewpoint.
> 
> We'd like to discuss the following points, so any comments are welcome.
>  * Points of the implementation
>    * How do we implement the scenario tests?
>      * via CLI/RestClient/client library...
>      * Where do we implement in the directory?
>    * Points of Scenarios
>      * scenario categories: Private cloud, Public cloud, VPC
>      * the scale: # of tenants, users, networks, and so on
>      * the way of verifying: RESTful APIs, ping, ssh, and so on
> 
> 
> Thanks,
> Masayuki Igawa
> 
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> 
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>