Open Stack

On 10/28/2015 01:12 PM, Chris Dent wrote:
> On Wed, 28 Oct 2015, Emilien Macchi wrote:
> 
>> As a user[1], I would like to functionally test OpenStack services.
> 
> Bless you. Let's all be more like that.
> 
>> Until now I was happy, until this bug [4] (TL;DR: Aodh can't be testing
>> with Tempest which is a bug I'm working on, and not really related to
>> this thread).
>> I realized Aodh [5] (and apparently some other projects like Ceilometer)
>> were using something else (gabbi [6]) for testing.
> 
> There's a fair amount of history here for which I don't know all the
> details so I'll just try to address the gabbi related questions below
> after this initial comment:
> 
> aodh is not yet in tempest, but that's simply because it is not yet
> done, not because there are not plans to do it. Tomorrow morning at
> 9am in the Kiri room we're having a design session about functional
> and integration testing in all three of ceilometer, aodh and gnocchi
> and one of the primary topics is: getting all three to have tempest
> plugins. One of the other primary topics is making new and existing
> in-tree functional test as good and useful as possible. A lot of those
> tests are not tempest because they've been extracted from pre-existing
> so-called unit tests but were determined to not be because they use a
> database. A small segment are API tests driven by gabbi as a result of
> this spec[1]
> 
> If you or anyone else have time to show up to that session, that would
> be great.

I was here and I'm happy to see the efforts [1] that are proposed for
the next cycle.
Having a Tempest plugin is really what I was asking in my initial
request, thank you for considering that work.

[1] https://etherpad.openstack.org/p/mitaka-telemetry-testing

> 
>> How come some big tent projects do not use Tempest anymore for
>> functional testing? I thought there was/is a move with tempest plugins
>> that will help projects to host their tempest tests in their repos.
> 
> Ceilometer projects started a migration to more in-tree functional
> tests in advance of tempest-lib existing. These manifest as tests that
> require a storage backend and have their own non-tempest job
> description in project-config.
> 
>> Am I missing something? Any official decision taken?
>> Is gabbi supported by OpenStack?
> 
> When I first released gabbi there was talk about it becoming either
> part of QA or oslo but after showing it around the world a bit I had
> feedback from several non-openstack people that they would be _far_
> more likely to contribute to it if it was not subject to the openstack
> development model[0]. So it lives now as a project on github to which
> people submit pull requests and travis takes care of CI. To avoid bus
> termination errors, there are two other admins in addition to me who
> have all the same rights with regard to merging and releasing and
> maintaining on python. One of them is another OpenStack contributor
> (jasonamyers).
> 
> I'm obviously biased, but I think gabbi is teh ossum and makes writing
> API tests easy and perhaps more importantly makes reading them later
> fantastic.
> 
> Within gnocchi and aodh, gabbi has been so successful at making it easy
> to write API tests that it is now being used for writing integration
> tests of aodh+gnocchi+ceilometer+heat.
> 
>> I feel like there is currently 2 paths that try to do the same thing and
>> as a user, I'm not happy.
> 
> Yeah, that's perhaps a problem. One thing I'm hoping to explore (or
> hoping someone else will explore) is making it easy to do gabbi
> tests within a tempest plugin. This ought to be possible but there
> are some humps to deal with regard to how gabbi orders and groups
> tests.
> 
> Again, I'm biased, but I think that gabbi would be a _huge_ asset for
> API tests in tempest because by its very nature it is very close to
> HTTP without a notion of a (python) client being involved. I think
> this is an excellent guard for ensuring that OpenStack APIs are
> sufficiently agnostic about their context.
> 
>> Please help me to understand,
> 
> I hope that adds a bit more info. I know it doesn't actually answer
> the real question though. I hope some other voices will come along.

Thanks for taking care of this topic, I'm glad you replied during the
Summit so we can move forward on this topic and we can make sure our
puppet-aodh module will be tested (one day) with Tempest.

> 
> [0] I'm happy to discuss this elsewhere (in person, on another thread,
> whatever) but it would be bad to let it distract this current thread.
> [1]
> http://specs.openstack.org/openstack/ceilometer-specs/specs/kilo/declarative-http-tests.html
> 
> 

-- 
Emilien Macchi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151029/a09faa42/attachment.pgp>