[Openstack] [OpenStack][Nova] Minimum required code coverage per file

Lorin Hochstein lorin at nimbisservices.com
Wed Apr 25 03:08:51 UTC 2012


On Apr 24, 2012, at 4:11 PM, Joe Gordon wrote:

> Hi All,
> 
> I would like to propose a minimum required code coverage level per file in Nova.  Say 80%.  This would mean that any new feature/file should only be accepted if it has over 80% code coverage.  Exceptions to this rule would be allowed for code that is covered by skipped tests (as long as 80% is reached when the tests are not skipped).
> 

I like the idea of looking at code coverage numbers. For any particular merge proposal, I'd also like to know whether it increases or decreases the overall code coverage of the project. I don't think we should gate on this, but it would be helpful for a reviewer to see that, especially for larger proposals.


> With 193 python files in nova/tests, Nova unit tests produce 85% overall code coverage (calculated with ./run_test.sh -c [1]).  But 23% of files (125 files) have lower then 80% code coverage (30 tests skipped on my machine).  Getting all files to hit the 80% code coverage mark should be one of the goals for Folsom.
> 

I would really like to see a visualization of the code coverage distribution, in order to help spot the outliers. 


Along these lines, there's been a lot of work in the software engineering research community about predicting which parts of the code are most likely to contain bugs ("fault prone" is a good keyword to find this stuff, e.g.: http://scholar.google.com/scholar?q=fault+prone, big names include Nachi Nagappan at MS Research and Elaine Weyuker, formerly of AT&T Research). I would *love* to see some academic researchers try to apply those techniques to OpenStack to help guide QA activities by identifying which parts of the code should get more rigorous  testing and review. 

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120424/3a6257cd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4897 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120424/3a6257cd/attachment.bin>


More information about the Openstack mailing list