[openstack-dev] [all] Branchless Tempest QA Spec - final draft

Jaume Devesa devvesa at gmail.com
Wed Apr 16 19:22:54 UTC 2014


I thought that OpenStack just support one release backwards, if we have to
support three versions, this is not useful.

There are already ways to enable/disable modules in tempest to adapt to
each deployment needs. Just wanted to avoid more configuration options.




On 16 April 2014 21:14, David Kranz <dkranz at redhat.com> wrote:

>  On 04/16/2014 11:48 AM, Jaume Devesa wrote:
>
>  Hi Sean,
>
>  for what I understood, we will need a new feature flag for each new
> feature, and a feature flag (default to false) for each deprecated one. My
> concern is: since the goal is make tempest a confident tool to test any
> installation and not and 'tempest.conf' will not be auto-generated from any
> tool as devstack does, wouldn't be too hard to prepare a tempest.conf file
> with so many feature flags to enable and disable?
>
> If we go down this route, and I think we should, we probably need to
> accept that it will be hard for users to manually configure tempest.conf.
> Tempest configuration would have to be done by whatever installation
> technology was used, as devstack does, or by auto-discovery. That implies
> that the presence of new features should be discoverable through the api
> which is a good idea anyway. Of course some one could configure it manually
> but IMO that is not desirable even with where we are now.
>
>
>  Maybe I am simplifying too much, but wouldn't be enough with a pair of
> functions decorators like
>
>  @new
> @deprecated
>
>  Then, in tempest.conf it could be a flag to say which OpenStack
> installation are you testing:
>
>  installation = [icehouse|juno]
>
>  if you choose Juno, tests with @new decorator will be executed and tests
> with @deprecated will be skipped.
>  If you choose Icehouse, tests with @new decorator will be skipped, and
> tests with @deprecated will be executed
>
>  I am missing some obvious case that make this approach a nonsense?
>
> There are two problems with this. First, some folks are chasing master for
> their deployments and some do not deploy all the features that are set up
> by devstack. In both cases, it is not possible to identify what can be
> tested with a simple name that corresponds to a stable release. Second,
> what happens when we move on to K? The meaning of "new" would have to
> change while retaining its old meaning as well which won't work. I think
> Sean spelled out the important scenarios.
>
>  -David
>
>
>  Regards,
> jaume
>
>
> On 14 April 2014 15:21, Sean Dague <sean at dague.net> wrote:
>
>> As we're coming up on the stable/icehouse release the QA team is looking
>> pretty positive at no longer branching Tempest. The QA Spec draft for
>> this is here -
>>
>> http://docs-draft.openstack.org/77/86577/2/check/gate-qa-specs-docs/3f84796/doc/build/html/specs/branchless-tempest.html
>> and hopefully address a lot of the questions we've seen so far.
>>
>> Additional comments are welcome on the review -
>> https://review.openstack.org/#/c/86577/
>> or as responses on this ML thread.
>>
>>         -Sean
>>
>> --
>> Sean Dague
>> Samsung Research America
>> sean at dague.net / sean.dague at samsung.com
>> http://dague.net
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> _______________________________________________
> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140416/e3d1b7b5/attachment.html>


More information about the OpenStack-dev mailing list