[openstack-qa] Tempest Coverage Analysis and Gaps in Compute Tests

Jaroslav Henner jhenner at redhat.com
Mon Feb 4 11:21:32 UTC 2013


Hi

I fail to understand the coverage report. For example in
http://logs.openstack.org/periodic/periodic-tempest-devstack-coverage-vm-full/15/logs/coverage-report/nova_compute_api.html
  1) the import section. How is it possible some lines are reported to 
be not-executed, while the others are?
  2) 91-95 The same question as 1)
  3) How can it be that the condition (109) was not exectued, but the 
110 -- the statement was?



On 30.1.2013 08:21, Yaniv Kaul wrote:
> On 30/01/13 00:54, Matthew Treinish wrote:
>> Hi Everyone,
>>
>> I've started working on an analysis of the coverage reports generated
>> by the
>> periodic Jenkins jobs:
>>
>> https://jenkins.openstack.org/job/periodic-tempest-devstack-coverage-vm-full/
>>
>>
>> I wanted to see how good a job tempest is doing for covering the Nova
>> API code.
>> Right now we've got some gaps in what is being tested in a tempest
>> full gate
>> run. I've started putting up a list of what functionality is missing
>> from the
>> tempest runs here:
>>
>> https://etherpad.openstack.org/coverage-analysis
>>
>> The periodic jobs uses the same configuration as our current gate
>> jobs, so
>> there are couple cases where we having testing but they depend on
>> things not
>> set in tempest.conf (Resize action for example)
>>
>> Then based on the results of the coverage report I put together a list of
>> proposed additional tests here:
>>
>> https://etherpad.openstack.org/MissingTempestTests
>
> Will you be converting some of the bigger and/or contained tasks to
> blueprints for Tempest?
> Y.
>
>>
>> Keep in mind that both of these etherpads are still WIP (and quite
>> incomplete)
>> and I'll be continually updating them over the next few days until I
>> go through
>> the rest of the coverage reports. I just wanted to share what I'm
>> working on
>> and some of the initial findings. Feel free to ping me on IRC for
>> specifics or
>> questions about any of this.
>>
>> Thanks,
>>
>> Matt Treinish
>>
>>
>> _______________________________________________
>> openstack-qa mailing list
>> openstack-qa at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>
>
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa




More information about the openstack-qa mailing list