<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Aug 4, 2014 at 3:29 AM, Chris Dent <span dir="ltr"><<a href="mailto:chdent@redhat.com" target="_blank">chdent@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
In the "Thoughts on the patch test failure rate and moving forward"<br>
thread[1] there's discussion of moving some of the burden for<br>
functional testing to the individual projects. This seems like a good<br>
idea to me, but also seems like it could be a source of confusion so I<br>
thought I'd start another thread to focus on the details of just this<br>
topic, separate from the gate-oriented discussion in the other.<br>
<br>
In a couple of messages[2] Sean mentions "the vision". Is there a wiki<br>
page or spec or other kind of document where this nascent vision is<br>
starting to form? Even if we can't quite get started just yet, it<br>
would be good to have an opporunity to think about the constraints and<br>
goals that we'll be working with.<br></blockquote><div><br></div><div>There is no single document on this that I know of. But two good places to start are the functional testing systems we have for swift and neutron:</div>
<div><br></div><div><a href="http://logs.openstack.org/70/116570/3/check/check-swift-dsvm-functional/4894055/console.html">http://logs.openstack.org/70/116570/3/check/check-swift-dsvm-functional/4894055/console.html</a><br>
</div><div><br></div><div><br></div><div><a href="http://logs.openstack.org/70/116570/3/check/check-neutron-dsvm-functional/bf68c61/console.html">http://logs.openstack.org/70/116570/3/check/check-neutron-dsvm-functional/bf68c61/console.html</a><br>
</div><div><a href="http://git.openstack.org/cgit/openstack/neutron/tree/tox.ini#n29">http://git.openstack.org/cgit/openstack/neutron/tree/tox.ini#n29</a><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Not just the goal of moving tests around, but what, for example, makes<br>
a good functional test?<br></blockquote><div><br></div><div>Here is a non-exhaustive list of some differences between what belongs in tempest and functional tests </div><div><br></div><div>* functional tests should ideally not need external services (external services makes it closer to an integration test)</div>
<div>* functional tests can be whitebox tests</div><div>* a functional test environment should be easier to set up then devstack</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
For constraints: Will tempest be available as a stable library? Is using<br>
tempest (or other same library across all projects) a good or bad thing?<br>
Seems there's some disagreement on both of these.<br></blockquote><div><br></div><div>Yes, there is a separate thread on spinning out a tempest-lib (not sure on what final name will be yet) that functional tests can use. Although I think there is a lot to be done before needing the tempest-lib.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Personally I'm quite eager to to vastly increase the amount of testing<br>
I can do on my own machine(s) before letting the gate touch my code.<br></blockquote><div><br></div><div>Why can't you run devstack locally? Maybe there are some changes we can make so its easier to run devstack locally first.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
[1]<br>
<a href="http://lists.openstack.org/pipermail/openstack-dev/2014-July/thread.html#41057" target="_blank">http://lists.openstack.org/<u></u>pipermail/openstack-dev/2014-<u></u>July/thread.html#41057</a><br>
[2]<br>
<a href="http://lists.openstack.org/pipermail/openstack-dev/2014-July/041188.html" target="_blank">http://lists.openstack.org/<u></u>pipermail/openstack-dev/2014-<u></u>July/041188.html</a><br>
<a href="http://lists.openstack.org/pipermail/openstack-dev/2014-July/041252.html" target="_blank">http://lists.openstack.org/<u></u>pipermail/openstack-dev/2014-<u></u>July/041252.html</a><span class=""><font color="#888888"><br>
<br>
-- <br>
Chris Dent tw:@anticdent freenode:cdent<br>
<a href="https://tank.peermore.com/tanks/cdent" target="_blank">https://tank.peermore.com/<u></u>tanks/cdent</a><br>
<br>
______________________________<u></u>_________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.<u></u>org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
</font></span></blockquote></div><br></div></div>