Open Stack

Coverage for tempest is different than coverage for unit tests. It 
doesn't start at process start, it is turned on once nova is running.

Which means the eval lines will have already executed prior to coverage 
counters being enabled. And as far as I can tell, in most cases, python 
coverage only counts the def line when the function is first loaded (not 
executed). As such it's not surprising that the decorators look weird.

I'm honestly surprised any of the imports are marked as executed, wonder 
if Matt's got any other thoughts.

	-Sean

On 02/04/2013 06:21 AM, Jaroslav Henner wrote:
> Hi
>
> I fail to understand the coverage report. For example in
> http://logs.openstack.org/periodic/periodic-tempest-devstack-coverage-vm-full/15/logs/coverage-report/nova_compute_api.html
>
>   1) the import section. How is it possible some lines are reported to
> be not-executed, while the others are?
>   2) 91-95 The same question as 1)
>   3) How can it be that the condition (109) was not exectued, but the
> 110 -- the statement was?
>
>
>
> On 30.1.2013 08:21, Yaniv Kaul wrote:
>> On 30/01/13 00:54, Matthew Treinish wrote:
>>> Hi Everyone,
>>>
>>> I've started working on an analysis of the coverage reports generated
>>> by the
>>> periodic Jenkins jobs:
>>>
>>> https://jenkins.openstack.org/job/periodic-tempest-devstack-coverage-vm-full/
>>>
>>>
>>>
>>> I wanted to see how good a job tempest is doing for covering the Nova
>>> API code.
>>> Right now we've got some gaps in what is being tested in a tempest
>>> full gate
>>> run. I've started putting up a list of what functionality is missing
>>> from the
>>> tempest runs here:
>>>
>>> https://etherpad.openstack.org/coverage-analysis
>>>
>>> The periodic jobs uses the same configuration as our current gate
>>> jobs, so
>>> there are couple cases where we having testing but they depend on
>>> things not
>>> set in tempest.conf (Resize action for example)
>>>
>>> Then based on the results of the coverage report I put together a
>>> list of
>>> proposed additional tests here:
>>>
>>> https://etherpad.openstack.org/MissingTempestTests
>>
>> Will you be converting some of the bigger and/or contained tasks to
>> blueprints for Tempest?
>> Y.
>>
>>>
>>> Keep in mind that both of these etherpads are still WIP (and quite
>>> incomplete)
>>> and I'll be continually updating them over the next few days until I
>>> go through
>>> the rest of the coverage reports. I just wanted to share what I'm
>>> working on
>>> and some of the initial findings. Feel free to ping me on IRC for
>>> specifics or
>>> questions about any of this.
>>>
>>> Thanks,
>>>
>>> Matt Treinish
>>>
>>>
>>> _______________________________________________
>>> openstack-qa mailing list
>>> openstack-qa at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>
>>
>> _______________________________________________
>> openstack-qa mailing list
>> openstack-qa at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>
>
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>


-- 
Sean Dague
IBM Linux Technology Center
email: sdague at linux.vnet.ibm.com
alt-email: sldague at us.ibm.com