Open Stack

100% agree, there is a mix-mash of unit tests and things called unit tests (more likely integration tests) in the code bases. In anvil I often actually have to remove certain tests from running since said test depends on some other component "strongly" (swift tests in nova if I remember correctly).

Sort of sad that it requires a whole complex system of setup this before that, just to run unit tests...

Sent from my really tiny device...

On Feb 9, 2013, at 11:25 AM, "John Griffith" <john.griffith at solidfire.com> wrote:

> Hey Everyone,
> 
> I just wanted to get some thoughts going around our unit testing process.  The good thing is we seem to do a pretty good job of requiring unit tests with patches, the bad thing is I'm not sure I think our strategy on how we implement those unit tests is the best.  There are a couple of things I've been noticing lately that is somewhat annoying for me.
> 
> First, I guess I have my own view of what a Unit test is and maybe that's not in line with the rest of the community.  My opinion is that a unit test should be the smallest white box test possible on a piece of code, and the boundary of a unit test should be the class the code that's being exercised belongs/lives in.  This means that for things like db calls, calls out to other modules etc that fakes, or mocks should be used to provide the inputs and responses needed by the code under test.  This of course requires a bit more discipline around consistency in our public interfaces which IMO we desperately need as well.
> 
> We seem to be going down a path of requiring more and more infrastructure and reliance from modules inside of the respective projects code base, and lately (the most annoying thing to me) is actually requiring the installation of the project and it's services in order for our unit tests to run.  This seems wrong to me.  I also think that we're getting some great benefit from a lot of hard work that's been going on around our unit test framework and infrastructure, but it seems that we keep making it increasingly complex and difficult to run and most of all use our unit tests for debugging purposes.  It seems like we've really moved more towards integration tests rather than unit tests IMO.  Yes; I can use venv (prefer not to), Yes; I can use devstack (and I do for integration and functional tests), but I don't think that unit tests should be so heavy in terms of services, and inter-project dependencies.
> 
> I think that unit tests should be easy to run, with only as much complexity as is minimally required in order to test your segment of code.  I think that unit tests should be *unit* tests, not integration tests, not inter-module dependency tests, installer tests etc.  Those sorts of things should be handled in devstack/gate tests and in Tempest. 
> 
> Anyway, I don't want to go on a rant or anything here.  I just wanted to throw some opinions out there and see if I'm in the minority or if any others have similar thoughts around this.
> 
> Thanks,
> John
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev