Open Stack

Hi Salvatore,

I'm working on a patch to swap pep8 for flake8, which will increase the automation of mechanical review.  In the process of doing the cleanup required for the patch, I have seen some ways that quantum's test suite could be improved:

 -  An api method and its serialization format (xml, json) are orthogonal (or should be) and could be tested as such.  For plugins that test both serialization formats for each api method, test execution time is effectively doubled with no benefit to coverage.

 - Many plugins contain duplicate code and tests (e.g. get_port_from_device and its associated tests appear in each of the nec, openvswitch, ryu, and linuxbridge plugins).  Common code could be factored out and tested only once.

- Many quantum tests evaluate low-level functionality through a web api rather than isolating the functionality under test.  Writing more focused unit tests (and fewer functional and integration tests ala http://www.martinfowler.com/bliki/TestPyramid.html) could allow for better coverage at a lower cost in both maintenance and execution.

The technical aspects of these potential improvements are straightforward, but will require a degree of developer and reviewer education to ensure that they are properly implemented and maintained going forward.  Do you think the summit session you've proposed would be an appropriate venue, or should it be done separately?

Cheers,


Maru