Open Stack

A few examples just done in 5 minutes, not extensive...

https://github.com/openstack/nova/blob/master/nova/tests/test_objectstore.py
- that starts up an s3 server, huh?

https://github.com/openstack/nova/blob/master/nova/tests/test_cinder.py
- nova is making sure that the cinder client works, huh?

It seems to be getting better in grizzly now that we have all these client libraries to act as the intermediary, so that's progress, just it don't know why it took until grizzly to do this....

Sent from my really tiny device...

On Feb 10, 2013, at 10:47 AM, "Joshua Harlow" <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>> wrote:

100% agree, there is a mix-mash of unit tests and things called unit tests
(more likely integration tests) in the code bases. In anvil I often actually
have to remove certain tests from running since said test depends on some
other component "strongly" (swift tests in nova if I remember
correctly).

Sort of sad that it requires a whole complex system of setup
this before that, just to run unit tests...

Sent from my really tiny
device...

On Feb 9, 2013, at 11:25 AM, "John Griffith"
<john.griffith at solidfire.com<mailto:john.griffith at solidfire.com>> wrote:

Hey Everyone,

I just wanted to
get some thoughts going around our unit testing process.  The good thing is
we seem to do a pretty good job of requiring unit tests with patches, the
bad thing is I'm not sure I think our strategy on how we implement those
unit tests is the best.  There are a couple of things I've been noticing
lately that is somewhat annoying for me.

First, I guess I have my own
view of what a Unit test is and maybe that's not in line with the rest of
the community.  My opinion is that a unit test should be the smallest white
box test possible on a piece of code, and the boundary of a unit test should
be the class the code that's being exercised belongs/lives in.  This means
that for things like db calls, calls out to other modules etc that fakes, or
mocks should be used to provide the inputs and responses needed by the code
under test.  This of course requires a bit more discipline around
consistency in our public interfaces which IMO we desperately need as
well.

We seem to be going down a path of requiring more and more
infrastructure and reliance from modules inside of the respective projects
code base, and lately (the most annoying thing to me) is actually requiring
the installation of the project and it's services in order for our unit
tests to run.  This seems wrong to me.  I also think that we're getting some
great benefit from a lot of hard work that's been going on around our unit
test framework and infrastructure, but it seems that we keep making it
increasingly complex and difficult to run and most of all use our unit tests
for debugging purposes.  It seems like we've really moved more towards
integration tests rather than unit tests IMO.  Yes; I can use venv (prefer
not to), Yes; I can use devstack (and I do for integration and functional
tests), but I don't think that unit tests should be so heavy in terms of
services, and inter-project dependencies.

I think that unit tests
should be easy to run, with only as much complexity as is minimally required
in order to test your segment of code.  I think that unit tests should be
*unit* tests, not integration tests, not inter-module dependency tests,
installer tests etc.  Those sorts of things should be handled in
devstack/gate tests and in Tempest.

Anyway, I don't want to go on a
rant or anything here.  I just wanted to throw some opinions out there and
see if I'm in the minority or if any others have similar thoughts around
this.

Thanks,
John

_______________________________________________
OpenStack-dev mailing
list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130210/16015b9b/attachment.html>