Open Stack

On Sun, Jul 12, 2015 at 09:29:57PM -0600, Matt Fischer wrote:
> We used Tempest for a time against our production environment. It was a
> pain to clean up but ephemeral test jobs solves that for you. A few
> questions

Just curious how long ago was this? Because resource leaks have always been
treated as bugs in tempest. We've done a lot of work over the past year to get
better about our cleanup in tempest. There are also config options to limit the
dynamic creation of resources and provide a static set of user/tenants to
limit any potential dangling users. [1] Although we should always be calling
delete for created users and projects as part of test setup. (this had to be
fixed for other internal mechanisms to work)

I'm just looking to get more feedback about this, because I see statements like
this from time to time but not many bugs being filed about specific issues
people encounter.

> 
> What version of tempest will be using?
> Will we maintain a blacklist if there are known failures? (although this is
> a pain to keep updated)

Do you mean because of configuration differences you don't want to run certain
tests or because of bugs in OpenStack causing intermittent failures?

-Matt Treinish

[1] http://docs.openstack.org/developer/tempest/configuration.html#locking-test-accounts-aka-accounts-yaml-or-accounts-file

> 
> On Sun, Jul 12, 2015 at 8:44 PM, Emilien Macchi <emilien.macchi at gmail.com>
> wrote:
> 
> > Hi,
> >
> > I would like to propose to run Tempest at the end of the beaker jobs, for
> > testing purpose now.
> >
> > As a start, we would accept 0 & 1 as return code, because this is really
> > experimental.
> > Though I think it will be interesting to see how it behaves, specially
> > when we implement new configurations or plugins in our modules.
> >
> > I already did a PoC for puppet-keystone:
> > https://review.openstack.org/198561
> > (failing because it needs more work to pass keystone v3 tests but v2 tests
> > are successful).
> > As you can see the code is really light, since we use puppet-tempest.
> >
> > Any suggestion is welcome !
> >
> > --
> > Emilien Macchi
> >
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150713/53c80679/attachment.pgp>