Open Stack

On Po 4. únor 2013, 13:09:54 CET, Sean Dague wrote:
> Coverage for tempest is different than coverage for unit tests. It
> doesn't start at process start, it is turned on once nova is running.
I am aware of that.

>
> Which means the eval lines will have already executed prior to
> coverage counters being enabled. And as far as I can tell, in most
> cases, python coverage only counts the def line when the function is
> first loaded (not executed). As such it's not surprising that the
> decorators look weird.
>
> I'm honestly surprised any of the imports are marked as executed,
> wonder if Matt's got any other thoughts.

Sure. The def lines in module are executed when module is being loaded. 
The content of the def is executed when the function is called.

I think the problem here is that gevent, greenlet and similar are doing 
some magic with the stack. See [1] and [2].



In any case, the coverage reports like this are quite inaccurate, if 
not invalid. Maybe we should rather use the "fixed" coverage tool [3]

[1] 
https://bitbucket.org/ned/coveragepy/issue/149/coverage-gevent-looks-broken
[2] http://nedbatchelder.com/code/coverage/trouble.html
[3] https://github.com/newbrough/coverage
>
>     -Sean
>
> On 02/04/2013 06:21 AM, Jaroslav Henner wrote:
>> Hi
>>
>> I fail to understand the coverage report. For example in
>> http://logs.openstack.org/periodic/periodic-tempest-devstack-coverage-vm-full/15/logs/coverage-report/nova_compute_api.html
>>
>>
>>   1) the import section. How is it possible some lines are reported to
>> be not-executed, while the others are?
>>   2) 91-95 The same question as 1)
>>   3) How can it be that the condition (109) was not exectued, but the
>> 110 -- the statement was?
>>
>>
>>
>> On 30.1.2013 08:21, Yaniv Kaul wrote:
>>> On 30/01/13 00:54, Matthew Treinish wrote:
>>>> Hi Everyone,
>>>>
>>>> I've started working on an analysis of the coverage reports generated
>>>> by the
>>>> periodic Jenkins jobs:
>>>>
>>>> https://jenkins.openstack.org/job/periodic-tempest-devstack-coverage-vm-full/
>>>>
>>>>
>>>>
>>>>
>>>> I wanted to see how good a job tempest is doing for covering the Nova
>>>> API code.
>>>> Right now we've got some gaps in what is being tested in a tempest
>>>> full gate
>>>> run. I've started putting up a list of what functionality is missing
>>>> from the
>>>> tempest runs here:
>>>>
>>>> https://etherpad.openstack.org/coverage-analysis
>>>>
>>>> The periodic jobs uses the same configuration as our current gate
>>>> jobs, so
>>>> there are couple cases where we having testing but they depend on
>>>> things not
>>>> set in tempest.conf (Resize action for example)
>>>>
>>>> Then based on the results of the coverage report I put together a
>>>> list of
>>>> proposed additional tests here:
>>>>
>>>> https://etherpad.openstack.org/MissingTempestTests
>>>
>>> Will you be converting some of the bigger and/or contained tasks to
>>> blueprints for Tempest?
>>> Y.
>>>
>>>>
>>>> Keep in mind that both of these etherpads are still WIP (and quite
>>>> incomplete)
>>>> and I'll be continually updating them over the next few days until I
>>>> go through
>>>> the rest of the coverage reports. I just wanted to share what I'm
>>>> working on
>>>> and some of the initial findings. Feel free to ping me on IRC for
>>>> specifics or
>>>> questions about any of this.
>>>>
>>>> Thanks,
>>>>
>>>> Matt Treinish
>>>>
>>>>
>>>> _______________________________________________
>>>> openstack-qa mailing list
>>>> openstack-qa at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>
>>>
>>> _______________________________________________
>>> openstack-qa mailing list
>>> openstack-qa at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>
>>
>> _______________________________________________
>> openstack-qa mailing list
>> openstack-qa at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>
>
>