Open Stack

Hi, Sean.

Thank you for your comments.

On Wed, 27 Mar 2013 07:37:53 -0400
Sean Dague <sean at dague.net> wrote:
> I think there is value here, though tests like this are long enough that 
> we need to think really hard about if we'd run it in the gate. And if 
> not, what the value of having them in tempest is.
> 
> As we've seen with the stress tests, they bit rot really quickly. So if 
> there aren't regular runs of these tests being reported back to the 
> community, their value is quite minimal (if not completely counter 
> productive, because they don't work).

I agree. I also think scenario tests should be run regularly.

> 
> It might also be worth putting these in a fully separate project tree, 
> because I can see the value in running these tests with the official 
> clients.

I think that when the tests will be so long, these should be in a
separate project tree. And it's good that these are in Tempest while
the tests are not so long.


Regards,
Masayuki Igawa



> 
> 	-Sean
> 
> On 03/26/2013 08:33 PM, Masayuki Igawa wrote:
> > Hi,
> >
> > Here is a blueprint I've registered.
> >    https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests
> >
> > This blueprint proposes to add a new category "scenario tests" in Tempest.
> > Scenario tests are for testing across the components such as Nova, Keystone, Glance and so on.
> > By scenario tests, developers will be able to check whether a new code (bug-fixes or features) will cause a side effect to the other components.
> > We will be able to check OpenStack basic functions by common use case scenarios.
> > I'd like to propose a minimum basic scenario test that must work at any OpenStack environments.
> >
> > Implementation
> > ===============
> > * add tempest/tests/scenario directory
> > * add scenario tests in the directory like following
> >
> > """
> > Scenario-1: very small cloud, basic life-cycle scenario, single-flavor, single-image, single-tenant...
> > - for admin user
> > 1. Create a flavor
> > 2. Create a image
> > 3. Configure network settings
> > 4. Create & configure a member role, a project, a quota, a user
> > - for regular user
> > 5. Create a key-pair
> > 6. Boot a instance. List & show the instance
> > 7. Create a volume. List & show the volume
> > 8. Attach the volume
> > 9. Detach the volume
> > 10. Delete the instance
> > - for admin user
> > 11. Delete the user, the role, the network, the image, the tenant, and the flavor
> > """
> >
> > We think that the scenario tests can raise the use case coverage that is important for user's viewpoint.
> >
> > We'd like to discuss the following points, so any comments are welcome.
> >   * Points of the implementation
> >     * How do we implement the scenario tests?
> >       * via CLI/RestClient/client library...
> >       * Where do we implement in the directory?
> >     * Points of Scenarios
> >       * scenario categories: Private cloud, Public cloud, VPC
> >       * the scale: # of tenants, users, networks, and so on
> >       * the way of verifying: RESTful APIs, ping, ssh, and so on
> >
> >
> > Thanks,
> > Masayuki Igawa
> >
> > _______________________________________________
> > openstack-qa mailing list
> > openstack-qa at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> >
> 
> 
> -- 
> Sean Dague
> http://dague.net
> 
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa