Open Stack

On 27/03/13 13:37, Sean Dague wrote:
> I think there is value here, though tests like this are long enough 
> that we need to think really hard about if we'd run it in the gate. 
> And if not, what the value of having them in tempest is.

If we won't have them in Tempest, we'll have to have them somewhere else.
I completely agree they are not suitable nor intended for gating.

>
> As we've seen with the stress tests, they bit rot really quickly. So 
> if there aren't regular runs of these tests being reported back to the 
> community, their value is quite minimal (if not completely counter 
> productive, because they don't work).
>
> It might also be worth putting these in a fully separate project tree, 
> because I can see the value in running these tests with the official 
> clients.

Possibly, and using Tempest as a git sub-module of that project (to 
benefit from the code that's already in Tempest).
Y.

>
>     -Sean
>
> On 03/26/2013 08:33 PM, Masayuki Igawa wrote:
>> Hi,
>>
>> Here is a blueprint I've registered.
>> https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests
>>
>> This blueprint proposes to add a new category "scenario tests" in 
>> Tempest.
>> Scenario tests are for testing across the components such as Nova, 
>> Keystone, Glance and so on.
>> By scenario tests, developers will be able to check whether a new 
>> code (bug-fixes or features) will cause a side effect to the other 
>> components.
>> We will be able to check OpenStack basic functions by common use case 
>> scenarios.
>> I'd like to propose a minimum basic scenario test that must work at 
>> any OpenStack environments.
>>
>> Implementation
>> ===============
>> * add tempest/tests/scenario directory
>> * add scenario tests in the directory like following
>>
>> """
>> Scenario-1: very small cloud, basic life-cycle scenario, 
>> single-flavor, single-image, single-tenant...
>> - for admin user
>> 1. Create a flavor
>> 2. Create a image
>> 3. Configure network settings
>> 4. Create & configure a member role, a project, a quota, a user
>> - for regular user
>> 5. Create a key-pair
>> 6. Boot a instance. List & show the instance
>> 7. Create a volume. List & show the volume
>> 8. Attach the volume
>> 9. Detach the volume
>> 10. Delete the instance
>> - for admin user
>> 11. Delete the user, the role, the network, the image, the tenant, 
>> and the flavor
>> """
>>
>> We think that the scenario tests can raise the use case coverage that 
>> is important for user's viewpoint.
>>
>> We'd like to discuss the following points, so any comments are welcome.
>>   * Points of the implementation
>>     * How do we implement the scenario tests?
>>       * via CLI/RestClient/client library...
>>       * Where do we implement in the directory?
>>     * Points of Scenarios
>>       * scenario categories: Private cloud, Public cloud, VPC
>>       * the scale: # of tenants, users, networks, and so on
>>       * the way of verifying: RESTful APIs, ping, ssh, and so on
>>
>>
>> Thanks,
>> Masayuki Igawa
>>
>> _______________________________________________
>> openstack-qa mailing list
>> openstack-qa at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>
>
>