Open Stack

On Mon, Aug 4, 2014 at 3:29 AM, Chris Dent <chdent at redhat.com> wrote:

>
> In the "Thoughts on the patch test failure rate and moving forward"
> thread[1] there's discussion of moving some of the burden for
> functional testing to the individual projects. This seems like a good
> idea to me, but also seems like it could be a source of confusion so I
> thought I'd start another thread to focus on the details of just this
> topic, separate from the gate-oriented discussion in the other.
>
> In a couple of messages[2] Sean mentions "the vision". Is there a wiki
> page or spec or other kind of document where this nascent vision is
> starting to form? Even if we can't quite get started just yet, it
> would be good to have an opporunity to think about the constraints and
> goals that we'll be working with.
>

There is no single document on this that I know of. But two good places to
start are the functional testing systems we have for swift and neutron:

http://logs.openstack.org/70/116570/3/check/check-swift-dsvm-functional/4894055/console.html


http://logs.openstack.org/70/116570/3/check/check-neutron-dsvm-functional/bf68c61/console.html
http://git.openstack.org/cgit/openstack/neutron/tree/tox.ini#n29



>
> Not just the goal of moving tests around, but what, for example, makes
> a good functional test?
>

Here is a non-exhaustive list of some differences between what belongs in
tempest and functional tests

* functional tests should ideally not need external services (external
services makes it closer to an integration test)
* functional tests can be whitebox tests
* a functional test environment should be easier to set up then devstack



>
> For constraints: Will tempest be available as a stable library? Is using
> tempest (or other same library across all projects) a good or bad thing?
> Seems there's some disagreement on both of these.
>

Yes, there is a separate thread on spinning out a tempest-lib (not sure on
what final name will be yet) that functional tests can use. Although I
think there is a lot  to be done before  needing the tempest-lib.


>
> Personally I'm quite eager to to vastly increase the amount of testing
> I can do on my own machine(s) before letting the gate touch my code.
>

Why can't you run devstack locally? Maybe there are some changes we can
make so its easier to run devstack locally first.


>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2014-
> July/thread.html#41057
> [2]
> http://lists.openstack.org/pipermail/openstack-dev/2014-July/041188.html
> http://lists.openstack.org/pipermail/openstack-dev/2014-July/041252.html
>
> --
> Chris Dent tw:@anticdent freenode:cdent
> https://tank.peermore.com/tanks/cdent
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140825/66a444ca/attachment.html>