Open Stack

Hey Everyone,

I just wanted to get some thoughts going around our unit testing process.
The good thing is we seem to do a pretty good job of requiring unit tests
with patches, the bad thing is I'm not sure I think our strategy on how we
implement those unit tests is the best.  There are a couple of things I've
been noticing lately that is somewhat annoying for me.

First, I guess I have my own view of what a Unit test is and maybe that's
not in line with the rest of the community.  My opinion is that a unit test
should be the smallest white box test possible on a piece of code, and the
boundary of a unit test should be the class the code that's being exercised
belongs/lives in.  This means that for things like db calls, calls out to
other modules etc that fakes, or mocks should be used to provide the inputs
and responses needed by the code under test.  This of course requires a bit
more discipline around consistency in our public interfaces which IMO we
desperately need as well.

We seem to be going down a path of requiring more and more infrastructure
and reliance from modules inside of the respective projects code base, and
lately (the most annoying thing to me) is actually requiring the
installation of the project and it's services in order for our unit tests
to run.  This seems wrong to me.  I also think that we're getting some
great benefit from a lot of hard work that's been going on around our unit
test framework and infrastructure, but it seems that we keep making it
increasingly complex and difficult to run and most of all use our unit
tests for debugging purposes.  It seems like we've really moved more
towards integration tests rather than unit tests IMO.  Yes; I can use venv
(prefer not to), Yes; I can use devstack (and I do for integration and
functional tests), but I don't think that unit tests should be so heavy in
terms of services, and inter-project dependencies.

I think that unit tests should be easy to run, with only as much complexity
as is minimally required in order to test your segment of code.  I think
that unit tests should be *unit* tests, not integration tests, not
inter-module dependency tests, installer tests etc.  Those sorts of things
should be handled in devstack/gate tests and in Tempest.

Anyway, I don't want to go on a rant or anything here.  I just wanted to
throw some opinions out there and see if I'm in the minority or if any
others have similar thoughts around this.

Thanks,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130209/ffe4bd4d/attachment.html>