[openstack-qa] [Openstack-qa-team] Script to prepare the stack for full tempest run.

Maru Newby marun at redhat.com
Fri Jun 14 14:26:46 UTC 2013

On Jun 14, 2013, at 3:09 AM, Attila Fazekas <afazekas at redhat.com> wrote:

> Lot of people does not see the true value of tempest like installation
> and deployment independent testing.

> - These are the tests what all OpenStack user/admin/system integrator can share on.
> - Tempest contains most of the tests what can work regardless to any topology or infra...
> - Tempest can be isolated from the inspected deployment by multiple ways.
> This bug about isolation tempest cli tests and cli utilities:
> https://bugs.launchpad.net/tempest/+bug/1150296
> Tempest even does not have to be on any machine which is part of the deployment,
> so it can be even on different machine when you issue a multi-node packstack install.
> This is the place where I prefer the isolation, instead of saving nodes.
> But, you can run multiple tempest process on single node against multiple different deployment.
> (tempest itself not a resource hunger process)


> - run packstack or devstack or whatever is not the first step in any testing and not the last.
> - packstack is not the only way to deploy RDO or RHOS. 
> If I had choose where to implement a __generic__ solution for fully prepare tempest test,
> it will not be in packstack repo or the tempest repo.

I apologize if I wasn't clear, but my focus is updating the tempest support in the stackforge/puppet-* modules.   The work will be reusable beyond packstack.  I would expect similar functionality to find its way into the openstack chef recipes.

While I agree that a deployment-agnostic mechanism would be useful, I don't think it would replace the need for puppet/chef-native mechanisms.  It would, however, be essential to testing clouds that are already deployed.  In my ideal world such a mechanism would be implemented in python for ease of testing and maintenance, and live in the tempest repo.

> What I would really recommend is extending the tempest documentation
> with a generic manual setup documentation
> which contains copy paste-able code snippets and descriptions.

+1 - Please do!

> Preparing tempest automatically is not an issue.

I'm not sure I agree.  I believe that CI is essential to ensuring the quality of a distributed system like openstack, and CI depends on being able to build and run tests without human intervention.  To this end, I'm convinced that a given deployment mechanism is not feature-complete unless it supports automatic configuration of tempest (or some other automated test suite).

> What I would recommend (I created bugs in the bz) for packstack, 
> is changing the default settings to be more tempest friendly.
> I have additional packstack enhancement recommendation, 
> about what logic could be moved to the packstack.

As always, patches welcome!


> ----- Original Message -----
> From: "Jaroslav Henner" <jhenner at redhat.com>
> To: "All Things QA." <openstack-qa at lists.openstack.org>
> Cc: "Sean Dague" <sean at dague.net>
> Sent: Thursday, June 13, 2013 5:40:17 PM
> Subject: Re: [openstack-qa] [Openstack-qa-team] Script to prepare the stack for full tempest run.
> On Čt 13. červen 2013, 17:08:52 CEST, Sean Dague wrote:
>> On 06/13/2013 10:59 AM, James Michel wrote:
>>> On 13/06/13 14:50, Jaroslav Henner wrote:
>>>> Hi,
>>>> We wish to create some tool for automated preparation of stack for full
>>>> tempest run. We need it to be able to work with deployment that has
>>>> been
>>>> prepared by Packstack [1].
>>>> Packstack can give as an output some "answer file", which is
>>>> actually an
>>>> ini file. We can parse that file, take some other values as input, for
>>>> example images locations, ... and create a tempest.conf accordingly.
>>> Actually we were looking at some smart way of configuring tempest.conf
>>> after having deployed openstack. Is this more-or-less what you had in
>>> mind?
> Yes but it is ... bit less than I had in mind. We need at least to 
> upload the images for testing (cirros), increase quotas, add testing 
> users. That's not what I would expect from the deploying tool to do. 
> Maybe if I called it with some --test parameter I would.
>>>> The questions now are:
>>>>  * Should this be a part of Tempest?
>>>>  * Should this be a separate project?
>>>>  * Should this be a part of the Packstack?
>>>>  * What about other deploying tools - should that tool have several
>>>> frond-ends?
>>>>  * Does anybody have something already?
>>>> Any comments?
>>>> [1] https://wiki.openstack.org/wiki/Packstack
>>>> PS: As Attila Fazekas pointed, most things we can find dynamically from
>>>> the keystone/nova. True, but I don't quite like the idea to ssh to nova
>>>> host to get nova SQL password and tricks like that.
>>>> Jaroslav Henner
>> So I really think that these tempest configuration generators should
>> be back in the install tools themselves. We do this in devstack today,
>> and the installer should just configure tempest.conf as it's output.
>> It's not clear that an intermediary tool is really useful, because it
>> means that when your installer enables a new feature, you have to fix
>> the installer -> question file, then fix question engine -> tempest.
>>    -Sean
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa

More information about the openstack-qa mailing list