[openstack-dev] [Sahara] [QA] [tests coverage] Can we add CI job to control the unit tests coverage?

Anastasia Kuznetsova akuznetsova at mirantis.com
Fri Jul 3 10:29:58 UTC 2015


Boris,

thanks for an explanation! I will take a closer look at the cover.sh script.

On Fri, Jul 3, 2015 at 12:07 AM, Boris Pavlovic <bpavlovic at mirantis.com>
wrote:

> Anastasia,
>
> because new patch may not be just a new code, committer may delete
>> something or fix typos in docsting, etc.
>
>
> This job compares amount of non covered lines (before and after patch).
> If you just remove code there will be less lines that should be covered so
> amount of non covered lines will be less or the same (if everything was
> covered before)
>
> Fixing typos in docstrings won't introduce new lines.
>
> Btw job allows you to introduce  N (few) new lines that are not covered by
> unit tests that are uncovered in some cases.
>
>
> Best regards,
> Boris Pavlovic
>
> On Thu, Jul 2, 2015 at 10:46 AM, Anastasia Kuznetsova <
> akuznetsova at mirantis.com> wrote:
>
>> Hi Timur,
>>
>> Generally I think that it is a good idea to have a gate that will check
>> whether new code is covered by unit tests or not. But I am not sure that
>> this gate should be voting (if I understand you correct),
>> because new patch may not be just a new code, committer may delete
>> something or fix typos in docsting, etc.
>>
>> On Thu, Jul 2, 2015 at 8:15 PM, Timur Nurlygayanov <
>> tnurlygayanov at mirantis.com> wrote:
>>
>>> Hi all,
>>>
>>> I suggest to add CI job which will check the unit tests coverage for
>>> Sahara repository and will set -1 for commits with new code and without
>>> unit tests (if we have some degradation of tests coverage).
>>> This job successfully works for Rally project and it helps to organize
>>> the right code development process when developers write new unit tests for
>>> new functionality.
>>>
>>> we can just copy this job from Rally and start to use it for Sahara:
>>> Coverage control script:
>>> https://github.com/openstack/rally/blob/master/tests/ci/cover.sh
>>> Configuration file for coverage plugin (to exclude code which shouldn't
>>> be affected): https://github.com/openstack/rally/blob/master/.coveragerc
>>> Example of job in infra repository:
>>> https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L4088
>>>
>>> I expect that it will help to increase the tests coverage by unit tests.
>>>
>>> Do we have any objections?
>>>
>>> --
>>>
>>> Timur,
>>> Senior QA Engineer
>>> OpenStack Projects
>>> Mirantis Inc
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Anastasia Kuznetsova
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best regards,
Anastasia Kuznetsova
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150703/68d3caf8/attachment.html>


More information about the OpenStack-dev mailing list