Open Stack

----- Original Message -----
> From: "Maru Newby" <marun at redhat.com>
> To: "Attila Fazekas" <afazekas at redhat.com>
> Cc: "Jaroslav Henner" <jhenner at redhat.com>, "All Things QA." <openstack-qa at lists.openstack.org>
> Sent: Friday, June 14, 2013 4:26:46 PM
> Subject: Re: [openstack-qa] [Openstack-qa-team] Script to prepare the stack for full tempest run.
> 
> 
> On Jun 14, 2013, at 3:09 AM, Attila Fazekas <afazekas at redhat.com> wrote:
> 
> > Lot of people does not see the true value of tempest like installation
> > and deployment independent testing.
> 
> > 
> > - These are the tests what all OpenStack user/admin/system integrator can
> > share on.
> > - Tempest contains most of the tests what can work regardless to any
> > topology or infra...
> > - Tempest can be isolated from the inspected deployment by multiple ways.
> > 
> > This bug about isolation tempest cli tests and cli utilities:
> > https://bugs.launchpad.net/tempest/+bug/1150296
> > 
> > Tempest even does not have to be on any machine which is part of the
> > deployment,
> > so it can be even on different machine when you issue a multi-node
> > packstack install.
> > This is the place where I prefer the isolation, instead of saving nodes.
> > But, you can run multiple tempest process on single node against multiple
> > different deployment.
> > (tempest itself not a resource hunger process)
> 
> +1
> 
> > 
> > - run packstack or devstack or whatever is not the first step in any
> > testing and not the last.
> > - packstack is not the only way to deploy RDO or RHOS.
> > 
> > If I had choose where to implement a __generic__ solution for fully prepare
> > tempest test,
> > it will not be in packstack repo or the tempest repo.
> 
> I apologize if I wasn't clear, but my focus is updating the tempest support
> in the stackforge/puppet-* modules.   The work will be reusable beyond
> packstack.  I would expect similar functionality to find its way into the
> openstack chef recipes.
> 
> While I agree that a deployment-agnostic mechanism would be useful, I don't
> think it would replace the need for puppet/chef-native mechanisms.  It
> would, however, be essential to testing clouds that are already deployed.
> In my ideal world such a mechanism would be implemented in python for ease
> of testing and maintenance, and live in the tempest repo.
>  
> 
> > 
> > What I would really recommend is extending the tempest documentation
> > with a generic manual setup documentation
> > which contains copy paste-able code snippets and descriptions.
> 
> +1 - Please do!
> > 
> > Preparing tempest automatically is not an issue.
> 
> I'm not sure I agree.  I believe that CI is essential to ensuring the quality
> of a distributed system like openstack, and CI depends on being able to
> build and run tests without human intervention.  To this end, I'm convinced
> that a given deployment mechanism is not feature-complete unless it supports
> automatic configuration of tempest (or some other automated test suite).
> 


May be you miss understood me.

This is not a type of issue where you usually need anything to be at the tempest repo, however
having helper script is not an unusual thing in the other OpenStack projects.
And not unusual thing to calling them form devstack.

So it can be in tempest as well.

The issue what I wanted to solve with the doc to help anybody who new to tempest to try it.

The new comers usually have this kind of issue:
https://bugs.launchpad.net/tempest/+bug/1188932

If doc at the end just says call this script it is good.

Looks like even OpenStack doc style is not the same as the Fedora wiki,
so copy pasting code part's wasn't a good idea.

> > 
> > What I would recommend (I created bugs in the bz) for packstack,
> > is changing the default settings to be more tempest friendly.
> > 
> > I have additional packstack enhancement recommendation,
> > about what logic could be moved to the packstack.
> 
> As always, patches welcome!
> 
> 
> m.
> 
> 
> > 
> > 
> > 
> > ----- Original Message -----
> > From: "Jaroslav Henner" <jhenner at redhat.com>
> > To: "All Things QA." <openstack-qa at lists.openstack.org>
> > Cc: "Sean Dague" <sean at dague.net>
> > Sent: Thursday, June 13, 2013 5:40:17 PM
> > Subject: Re: [openstack-qa] [Openstack-qa-team] Script to prepare the stack
> > for full tempest run.
> > 
> > On Čt 13. červen 2013, 17:08:52 CEST, Sean Dague wrote:
> >> On 06/13/2013 10:59 AM, James Michel wrote:
> >>> On 13/06/13 14:50, Jaroslav Henner wrote:
> >>>> Hi,
> >>>> 
> >>>> We wish to create some tool for automated preparation of stack for full
> >>>> tempest run. We need it to be able to work with deployment that has
> >>>> been
> >>>> prepared by Packstack [1].
> >>>> 
> >>>> Packstack can give as an output some "answer file", which is
> >>>> actually an
> >>>> ini file. We can parse that file, take some other values as input, for
> >>>> example images locations, ... and create a tempest.conf accordingly.
> >>> 
> >>> Actually we were looking at some smart way of configuring tempest.conf
> >>> after having deployed openstack. Is this more-or-less what you had in
> >>> mind?
> > 
> > Yes but it is ... bit less than I had in mind. We need at least to
> > upload the images for testing (cirros), increase quotas, add testing
> > users. That's not what I would expect from the deploying tool to do.
> > Maybe if I called it with some --test parameter I would.
> > 
> >>> 
> >>>> 
> >>>> The questions now are:
> >>>>  * Should this be a part of Tempest?
> >>>>  * Should this be a separate project?
> >>>>  * Should this be a part of the Packstack?
> >>>>  * What about other deploying tools - should that tool have several
> >>>> frond-ends?
> >>>>  * Does anybody have something already?
> >>>> 
> >>>> Any comments?
> >>>> 
> >>>> [1] https://wiki.openstack.org/wiki/Packstack
> >>>> 
> >>>> PS: As Attila Fazekas pointed, most things we can find dynamically from
> >>>> the keystone/nova. True, but I don't quite like the idea to ssh to nova
> >>>> host to get nova SQL password and tricks like that.
> >>>> 
> >>>> Jaroslav Henner
> >> 
> >> So I really think that these tempest configuration generators should
> >> be back in the install tools themselves. We do this in devstack today,
> >> and the installer should just configure tempest.conf as it's output.
> >> 
> >> It's not clear that an intermediary tool is really useful, because it
> >> means that when your installer enables a new feature, you have to fix
> >> the installer -> question file, then fix question engine -> tempest.
> >> 
> >>    -Sean
> >> 
> > 
> > _______________________________________________
> > openstack-qa mailing list
> > openstack-qa at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> 
>