Open Stack

On Tue, Sep 27, 2016 at 12:18 PM, Timur Nurlygayanov <
tnurlygayanov at mirantis.com> wrote:

> Hi milan,
>
> we have measured the test coverage for OpenStack components with
> coverage.py tool [1]. It is very easy tool and it allows measure the
> coverage by lines of code and etc. (several metrics are available).
>
> [1] https://coverage.readthedocs.io/en/coverage-4.2/
>

coverage also supports aggregating results from multiple runs, so you can
measure results from combinations such as:

1) Unit tests
2) Functional tests
3) Integration tests
4) 1 + 2
5) 1 + 2 + 3

To my eyes 3 and 4 make the most sense. Unit and functional tests are
supposed to give you low level coverage, keeping in mind that 'functional
tests' is an overloaded term and actually means something else in every
community. Integration tests aren't about code coverage, they're about user
facing flows, so it'd be interesting to measure coverage
from integration tests, then comparing coverage coming from integration
tests, and getting the set difference between the two: That's the area that
needs more unit and functional tests.


>
> On Tue, Sep 27, 2016 at 1:06 PM, Jordan Pittier <
> jordan.pittier at scality.com> wrote:
>
>> Hi,
>>
>> On Tue, Sep 27, 2016 at 11:43 AM, milanisko k <vetrisko at gmail.com> wrote:
>>
>>> Dear Stackers,
>>> I'd like to gather some overview on the $Sub: is there some
>>> infrastructure in place to gather such stats? Are there any groups
>>> interested in it? Any plans to establish such infrastructure?
>>>
>> I am working on such a tool with mixed results so far. Here's my approach
>> taking let's say Nova as an example:
>>
>> 1) Print all the routes known to nova (available as a python-routes
>> object:  nova.api.openstack.compute.APIRouterV21())
>> 2) "Normalize" the Nova routes
>> 3) Take the logs produced by Tempest during a tempest run (in
>> logs/tempest.txt.gz). Grep for what looks like a Nova URL (based on port
>> 8774)
>> 4) "Normalize" the tested-by-tempest Nova routes.
>> 5) Compare the two sets of routes
>> 6) ????
>> 7) Profit !!
>>
>> So the hard part is obviously the normalizing of the URLs. I am currently
>> using a tons of regex.... :) That's not fun.
>>
>> I'll let you guys know if I have something to show.
>>
>> I think there's real interest on the topic (it comes up every year or
>> so), but no definitive answer/tool.
>>
>> Cheers,
>> Jordan
>>
>>
>>
>>
>> <https://www.scality.com/backup/?utm_source=signatures&utm_medium=email&utm_campaign=backup2016>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
>
> Timur,
> Senior QA Manager
> OpenStack Projects
> Mirantis Inc
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160927/d0e35a8a/attachment.html>