Open Stack

Regarding novaclient, I was debating this myself last night. Our backfire
suite currently uses it. However, I can see scenarios where nova has
advanced but novaclient has yet to, and because all three projects are
independently developed, we're stuck temporarily. I can see utility in
tests specifically for Novaclient, and ones that use a built in client (as
the suite that Soren provided currently does.)

Also, Backfire uses a module Kevin Mitchell wrote named Dtest that
actually provides a lot of that same functionality you mention, Tim. It
also gives you the ability to spawn the tests individually in greenthreads
so they execute in parallel. We got push back when we initially tried to
merge our tests into Nova, partially because it uses a 3rd party library
for executing the tests. We can try merging these things together, if
that's what everyone wants.

On 9/12/11 11:39 AM, "Tim Simpson" <tim.simpson at rackspace.com> wrote:

>I'm with the Reddwarf (Database as a Service) team at Rackspace. We
>employ a large set of integration / functional tests which run in a
>Vagrant controlled VM.
>
>Our tests use python-novaclient as well as a similar client for Reddwarf
>we've written ourselves. The motivation is to eat our own dog food- if
>the novaclient is the official rich client for Python, why shouldn't the
>test make use of it?
>
>We also use a library called Proboscis so we can order our tests without
>prefixing numbers to them, automatically skip related tests when
>something in a dependency fails, and avoid global variables even as we
>pass state between related tests. I think the openstack-integration-tests
>would definitely benefit from it.
>
>I'd love to have a conversation about getting the traits outlined above
>adopted into this standardized testing solution. :)
>
>Output of our tests running: http://pastie.org/2521835
>Proboscis documentation: http://packages.python.org/proboscis/
>________________________________________
>From: openstack-bounces+tim.simpson=rackspace.com at lists.launchpad.net
>[openstack-bounces+tim.simpson=rackspace.com at lists.launchpad.net] on
>behalf of Jay Pipes [jaypipes at gmail.com]
>Sent: Monday, September 12, 2011 8:20 AM
>To: Matt Dietz
>Cc: openstack at lists.launchpad.net
>Subject: Re: [Openstack] Integration tests
>
>Wow, another integration test framework emerges out of the woodwork :)
>
>Looking forward to getting all of these into a single place... and
>clearing up the lack of communication between teams on this subject!
>There's Kong, Stacktester, and Backfire. Any others out there we
>should know about?
>
>Cheers,
>jay
>
>On Sun, Sep 11, 2011 at 9:09 PM, Matt Dietz <matt.dietz at rackspace.com>
>wrote:
>> Suite looks great, Soren!
>> Wanted to mention that we actually developed our own suite of tests,
>>which
>> you can find at https://github.com/ohthree/backfire I'm planning to
>> reconcile the difference so we can stop the independent efforts, but
>>it's
>> going to take time. Something else I'd like to see in the suite, that we
>> currently have in backfire via a custom test module, is the ability to
>> parallelize the tests for execution.
>> From: Soren Hansen <soren at linux2go.dk>
>> Date: Sat, 10 Sep 2011 00:21:10 +0200
>> To: <openstack at lists.launchpad.net>
>> Subject: Re: [Openstack] Integration tests
>>
>> That link shouldn't have included the +bug bit. Copy/paste fail. :(
>>
>> Sent from my phone. Pardon my brevity.
>>
>> _______________________________________________ Mailing list:
>> https://launchpad.net/~openstack Post to : openstack at lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack More help :
>> https://help.launchpad.net/ListHelp This email may include confidential
>> information. If you received it in error, please delete it.
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack at lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>_______________________________________________
>Mailing list: https://launchpad.net/~openstack
>Post to     : openstack at lists.launchpad.net
>Unsubscribe : https://launchpad.net/~openstack
>More help   : https://help.launchpad.net/ListHelp

This email may include confidential information. If you received it in error, please delete it.