Open Stack

On Wed, Apr 22, 2015 at 02:03:13PM -0400, Emilien Macchi wrote:
> 
> 
> On 04/22/2015 11:53 AM, Spencer Krum wrote:
> > Emillen,
> > 
> > Do you see shelling out to openstackclient in the keystone test to
> > verify that the keystone resources have been created? Do you see trying
> > to hit the api from something like aviator? Ultimately I'd like to see
> > us spin up an entire openstack in one test then hit it with tempest.
> 
> * shell testing: yes because it's the way we wrote our providers.
> * aviator: no because we gave up some months ago in favor of using
> openstackclient. I'm not in favor of using aviator which would add yet
> another dependency and complexity. Maybe I'm wrong though.
> * tempest: well... tempest aims to test API features while we only want
> to check Keystone is actually running. I think serverspec + some
> shelling could help. Having tempest is (to me) overkill and could slow
> down our CI if something's wrong in Tempest.

I kinda take issue with this comment, more likely than not when there are issues
with running tempest it's either a config or service problem. That's not to say
there aren't tempest bugs, I just don't think tempest breaking your CI should be
a  primary concern. (especially how often it's used for this exact purpose) If
it did break something it's not like fixing it is a complicated procedure. We've
been running tempest as the gating tests for some time so we've got some experience
fixing issues with fixing the test suite when things really go haywire. 

I actually think running tempest against against a deployment using the puppet
modules would probably be very useful. It'll let you test that everything comes
together and actually works. It's used for gating devstack commits to verify 
basically the same thing. I'm also interested in doing this because I think
it'll help us improve tempest by having direct feedback loop from running it
against a non-devstack environment. That's something I'd really like to have
because it's something I think we're missing right now.

If you need any help with setting something like this up, I'm definitely
available to give a hand with it.

> 
> > It may be possible to use a very narrow version of tempest to validate
> > just keystone.
> 
> Like, only a small set of tests that we would run with testr?

So this is the intent of the smoke tag in tempest, to provide a small set of
tests to do a quick sanity check that a deployment is working. Unfortunately
right now the tag doesn't do that because it got conflated as part of the
neutron testing bring up. We just need to spend some time sitting down 
and rationalizing the use of the tag to provide this. 

Also, if you want to just test service specific testing tempest already does
service tagging of tests. So you can run a subset by using the service name as
the filter. (ie identity for tests which talk to keystone or compute for nova
tests)

-Matt Treinish

> > On Wed, Apr 22, 2015 at 7:51 AM, Emilien Macchi <emilien at redhat.com
> > <mailto:emilien at redhat.com>> wrote:
> > 
> >     Hi,
> > 
> >     Some important work is being done on Keystone v3 API support in
> >     puppet-keystone.
> >     We've clearly seen there is a lack of review and I think we all worry
> >     about breaking something.
> >     Spencer & I are working on beaker tests lately and the jobs are
> >     non-voting for now.
> > 
> >     I propose:
> >     * to review (and eventually merge) the beaker-tests patches [1] [2] for
> >     Keystone & openstacklib.
> >     * to patch project-config [3] to make vote Beaker jobs in Puppet
> >     OpenStack gate for puppet-keystone & puppet-openstacklib. Why voting?
> >     Because otherwise I'm not sure people will notice the failure and some
> >     patches will be merged while beaker is red.
> > 
> >     So we can have a good set of tests that will help us to detect some
> >     issues in the future.
> >     I don't think we will catch all mistakes we can do, but this is a good
> >     start.
> > 
> >     To vote this proposal, you can use the gerrit patches and let any
> >     feedback.
> > 
> >     Thanks,
> > 
> >     [1] puppet-keystone: https://review.openstack.org/#/c/155873/
> >     [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/
> >     [3] project-config: https://review.openstack.org/176343
> >     --
> >     Emilien Macchi
> > 
> >     --
> > 
> >     To unsubscribe from this group and stop receiving emails from it,
> >     send an email to puppet-openstack+unsubscribe at puppetlabs.com
> >     <mailto:puppet-openstack%2Bunsubscribe at puppetlabs.com>.
> > 
> > 
> > 
> > 
> > -- 
> > Spencer Krum
> > (619)-980-7820
> > 
> > -- 
> > 
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to puppet-openstack+unsubscribe at puppetlabs.com
> > <mailto:puppet-openstack+unsubscribe at puppetlabs.com>.
> 
> -- 
> Emilien Macchi
> 



> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150422/756a8bba/attachment.pgp>