[openstack-qa] [Tempest] Blueprint: add-scenario-tests

Masayuki Igawa masayuki.igawa at gmail.com
Wed Mar 27 20:03:32 UTC 2013


On Thu, Mar 28, 2013 at 1:58 AM, Venkatesan, Ravikumar <
ravikumar.venkatesan at hp.com> wrote:

> Earlier, the  proposal for the same (under name: life cycle) came up for
> discussion  with blueprint  from HP
> https://blueprints.launchpad.net/tempest/+spec/nova-vmlifecycle-test
> https://review.openstack.org/#/c/20901/
>
>
Though my proposal has similar points to the above, I think there is
difference that my proposal is for testing across the components such as
Nova, Keystone, Glance and so on.
nova-vmlifecycle-test proposal is for Nova testing only.
I missed something?

Regards,

Masayuki Igawa


Regards,
>
> Ravi
>
>
> -----Original Message-----
> From: Jay Pipes [mailto:jaypipes at gmail.com]
> Sent: Wednesday, March 27, 2013 6:43 AM
> To: openstack-qa at lists.openstack.org
> Subject: Re: [openstack-qa] [Tempest] Blueprint: add-scenario-tests
>
> On 03/26/2013 09:27 PM, Yaniv Kaul wrote:
> > Without the ability to configure OpenStack,  it's not going to fly.
>  Configure means:
> > 0. Installation scenarios (installation of different components on
> > different hosts) 1. Ini files manipulation (nova.conf and friends)
> > and restarting services 2. JSON files manipulation (policy files) 3.
> > Configuration via the API (and/or nova-manage)
> >
> > The above are basic blocks to any true scenario (and end to end) testing.
> > Y.
>
> Tempest does not need to do any of the above. Tempest assumes that
> environment has been set up already, and executes its tests for expected
> behaviour regardless of the configuration of the environment.
>
> Projects like devstack-gate (which runs devstack and configures the
> environments used in continuous integration testing) also write Tempest's
> configuration file, which tells Tempest certain key details about the
> environment it will execute against, including capabilities like ability to
> resize images, perform migrations, etc.
>
> Now what *would* enable better support for scenario testing would be
> API-based discovery of an OpenStack environment's capabilities and
> configuration settings... that would enable Tempest to customize some of
> its settings based on what it can query the OpenStack endpoints for.
>
> Best,
> -jay
>
> > Masayuki Igawa <igawa at mxs.nes.nec.co.jp> wrote:
> >
> > Hi,
> >
> > Here is a blueprint I've registered.
> >   https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests
> >
> > This blueprint proposes to add a new category "scenario tests" in
> Tempest.
> > Scenario tests are for testing across the components such as Nova,
> Keystone, Glance and so on.
> > By scenario tests, developers will be able to check whether a new code
> (bug-fixes or features) will cause a side effect to the other components.
> > We will be able to check OpenStack basic functions by common use case
> scenarios.
> > I'd like to propose a minimum basic scenario test that must work at any
> OpenStack environments.
> >
> > Implementation
> > ===============
> > * add tempest/tests/scenario directory
> > * add scenario tests in the directory like following
> >
> > """
> > Scenario-1: very small cloud, basic life-cycle scenario, single-flavor,
> single-image, single-tenant...
> > - for admin user
> > 1. Create a flavor
> > 2. Create a image
> > 3. Configure network settings
> > 4. Create & configure a member role, a project, a quota, a user
> > - for regular user
> > 5. Create a key-pair
> > 6. Boot a instance. List & show the instance 7. Create a volume. List
> > & show the volume 8. Attach the volume 9. Detach the volume 10. Delete
> > the instance
> > - for admin user
> > 11. Delete the user, the role, the network, the image, the tenant, and
> > the flavor """
> >
> > We think that the scenario tests can raise the use case coverage that is
> important for user's viewpoint.
> >
> > We'd like to discuss the following points, so any comments are welcome.
> >  * Points of the implementation
> >    * How do we implement the scenario tests?
> >      * via CLI/RestClient/client library...
> >      * Where do we implement in the directory?
> >    * Points of Scenarios
> >      * scenario categories: Private cloud, Public cloud, VPC
> >      * the scale: # of tenants, users, networks, and so on
> >      * the way of verifying: RESTful APIs, ping, ssh, and so on
> >
> >
> > Thanks,
> > Masayuki Igawa
> >
> > _______________________________________________
> > openstack-qa mailing list
> > openstack-qa at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> >
> > _______________________________________________
> > openstack-qa mailing list
> > openstack-qa at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> >
>
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>



-- 
Masayuki Igawa
twitter: http://twitter.com/masayukig
Facebook: http://www.facebook.com/masayuki.igawa
blog: http://br.0r2.info/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-qa/attachments/20130328/781d0de0/attachment.html>


More information about the openstack-qa mailing list