Open Stack

On Fri, 2012-08-10 at 08:01 -0500, Christopher B Ferris wrote:
> The next sentence in HACKING.rst goes a step further and requires that
> bug fixes for features that lack unit tests add them. Given that the 
> XML API code has been around for a while, it begs the question: why is
> the coverage so poor? Collectively, we need to fix both the symptom
> and the underlying cause. 

It's important to note that the unit test coverage is not poor; for
almost everything for which there's an XML template, there is also unit
tests covering that template serialization.  (The weasel word is only
because I haven't done a thorough review to verify my assertion for 100%
of cases.)  For that matter, when I contributed the XML template code, I
went through all existing extensions and created templates, even for
those that did not have XML support at the time, and when I do reviews
for new extensions, I try to ensure that there is a template for those
extensions.

The problem comes from the fact that there are several extensions that
have been added which I personally did not review, and for which there
exists no XML template.  The unit tests are perfectly adequate to test
those new extensions, but there is obviously no new serializer, and thus
no serializer unit tests.  It isn't that our test coverage is poor, it
is that our code is actually deficient, and we reviewers are directly to
blame for not enforcing that all API extensions must have working and
tested XML handling.

Note also that the unit tests are orthogonal to the tests being
discussed in this thread, which are _integration_ tests; those test
suites are apparently, to date, not testing the XML side of the API.
Remember also that these test suites are *outside* the nova code base,
and would also have to be updated to cover any newly-added extension;
I'm not currently aware of any process in place which ensures that they
get updated in that situation.
-- 
Kevin L. Mitchell <kevin.mitchell at rackspace.com>