Add me as a +1 to ensuring that unittests are actually unittests instead of a mix'n'match of different styles of test. This would make writing and updating tests much more straightforward and actually catch problems in the correct layer of tests, for example a change in network should not under any circumstance cause an api unittest to fail. I think we all understand the nightmare of trying to fix tests in unrelated areas of the code that have failed for no immediately apparent reason. How can I help make this happen?<br>
<br>-tr3buchet<br><br><div class="gmail_quote">On Tue, Nov 22, 2011 at 8:45 AM, Sandy Walsh <span dir="ltr"><<a href="mailto:sandy.walsh@rackspace.com">sandy.walsh@rackspace.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Excellent!<br>
<br>
I wrote a few blog posts recently, mostly based on my experience with openstack automated tests:<br>
<br>
<a href="http://www.sandywalsh.com/2011/06/effective-units-tests-and-integration.html" target="_blank">http://www.sandywalsh.com/2011/06/effective-units-tests-and-integration.html</a><br>
<a href="http://www.sandywalsh.com/2011/08/pain-of-unit-tests-and-dynamically.html" target="_blank">http://www.sandywalsh.com/2011/08/pain-of-unit-tests-and-dynamically.html</a><br>
<br>
Would love to see some of those changes make it in.<br>
<br>
-Sandy<br>
<br>
________________________________________<br>
From: openstack-bounces+sandy.walsh=<a href="mailto:rackspace.com@lists.launchpad.net">rackspace.com@lists.launchpad.net</a> [openstack-bounces+sandy.walsh=<a href="mailto:rackspace.com@lists.launchpad.net">rackspace.com@lists.launchpad.net</a>] on behalf of Soren Hansen [<a href="mailto:soren@linux2go.dk">soren@linux2go.dk</a>]<br>
Sent: Monday, November 21, 2011 8:24 AM<br>
To: <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Subject: [Openstack] [nova-testing] Efforts for Essex<br>
<div class="HOEnZb"><div class="h5"><br>
Hi, guys.<br>
<br>
We're scattered across enough different timezones to make real-time<br>
communication really awkward, so let's see if we can get by using e-mail<br>
instead.<br>
<br>
A good test suite will let you keep up the pace of development. It will<br>
offer confidence that your change didn't break any expectations, and<br>
will help you understand how things are supposed to work, etc. A bad<br>
test suite, on the other hand, will actually do the opposite, so the<br>
quality of the unit test suite is incredibly important to the overall<br>
health of the project. Integration tests are important as well, but unit<br>
tests all we can expect people to run on a regular basis.<br>
<br>
I'd like to start a bit of discussion around efforts around unit testing<br>
for Essex. Think of it as brainstorming. Input can be anything from<br>
small, actionable items, to broad ideas, to measurable goals, random<br>
thoughts etc. Anything goes. At some point, we can distill this input to<br>
a set of common themes, set goals, define action items, etc.<br>
<br>
A few things from the back of my mind to get the discussion going:<br>
<br>
= Speed up the test suite =<br>
<br>
A slow test suite gets run much less frequently than a fast one.<br>
Currently, when wrapped in eatmydata, a complete test run takes more<br>
than 6 minutes.<br>
<br>
Goal: We should get that down to less than one minute.<br>
<br>
= Review of existing tests =<br>
Our current tests have a lot of problems:<br>
<br>
* They overlap (multiple tests effectively testing the same code),<br>
* They're hard to understand. Not only are their intent not always<br>
clear, but it's often hard to tell how they're doing it.<br>
* They're slooooow.<br>
* They're interdependent. The failure of one test often cascades and<br>
makes others fail, too.<br>
* They're riddled with duplicated code.<br>
<br>
I think it would be great if we could come up with some guidelines for<br>
good tests and then go through the existing tests and highlight where<br>
they violate these guidelines.<br>
<br>
= Test coverage =<br>
We should increase test coverage.<br>
<br>
Adding tests to legacy code is hard. Generally, it's much easier to<br>
write tests for new production code as you go along. The two primary<br>
reasons are:<br>
<br>
* If you're writing tests and production code at the same time (or<br>
perhaps even writing the tests first), the code will almost<br>
automatically designed to be easily tested.<br>
* You (hopefully1) know how the code is supposed to work. This is not<br>
always obvious for existing code (often written by someone else).<br>
<br>
Therefore, the most approachable strategy for increasing test coverage<br>
is simply to ensure that any new code added is accompanied by tests, but<br>
of course new tests for currently untested code is fantastically<br>
welcome.<br>
<br>
--<br>
Soren Hansen | <a href="http://linux2go.dk/" target="_blank">http://linux2go.dk/</a><br>
Ubuntu Developer | <a href="http://www.ubuntu.com/" target="_blank">http://www.ubuntu.com/</a><br>
OpenStack Developer | <a href="http://www.openstack.org/" target="_blank">http://www.openstack.org/</a><br>
<br>
_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br>
_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
</div></div></blockquote></div><br>