[openstack-dev] [Sahara] [QA] [tests coverage] Can we add CI job to control the unit tests coverage?

Timur Nurlygayanov tnurlygayanov at mirantis.com
Mon Jul 13 11:00:58 UTC 2015


Hi,

here is the commit on review:
https://review.openstack.org/#/c/200433

We already added non-voting job in infra repository, need to merge this
script.

On Fri, Jul 3, 2015 at 1:29 PM, Anastasia Kuznetsova <
akuznetsova at mirantis.com> wrote:

> Boris,
>
> thanks for an explanation! I will take a closer look at the cover.sh
> script.
>
> On Fri, Jul 3, 2015 at 12:07 AM, Boris Pavlovic <bpavlovic at mirantis.com>
> wrote:
>
>> Anastasia,
>>
>> because new patch may not be just a new code, committer may delete
>>> something or fix typos in docsting, etc.
>>
>>
>> This job compares amount of non covered lines (before and after patch).
>> If you just remove code there will be less lines that should be covered
>> so amount of non covered lines will be less or the same (if everything was
>> covered before)
>>
>> Fixing typos in docstrings won't introduce new lines.
>>
>> Btw job allows you to introduce  N (few) new lines that are not covered
>> by unit tests that are uncovered in some cases.
>>
>>
>> Best regards,
>> Boris Pavlovic
>>
>> On Thu, Jul 2, 2015 at 10:46 AM, Anastasia Kuznetsova <
>> akuznetsova at mirantis.com> wrote:
>>
>>> Hi Timur,
>>>
>>> Generally I think that it is a good idea to have a gate that will check
>>> whether new code is covered by unit tests or not. But I am not sure that
>>> this gate should be voting (if I understand you correct),
>>> because new patch may not be just a new code, committer may delete
>>> something or fix typos in docsting, etc.
>>>
>>> On Thu, Jul 2, 2015 at 8:15 PM, Timur Nurlygayanov <
>>> tnurlygayanov at mirantis.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I suggest to add CI job which will check the unit tests coverage for
>>>> Sahara repository and will set -1 for commits with new code and without
>>>> unit tests (if we have some degradation of tests coverage).
>>>> This job successfully works for Rally project and it helps to organize
>>>> the right code development process when developers write new unit tests for
>>>> new functionality.
>>>>
>>>> we can just copy this job from Rally and start to use it for Sahara:
>>>> Coverage control script:
>>>> https://github.com/openstack/rally/blob/master/tests/ci/cover.sh
>>>> Configuration file for coverage plugin (to exclude code which shouldn't
>>>> be affected):
>>>> https://github.com/openstack/rally/blob/master/.coveragerc
>>>> Example of job in infra repository:
>>>> https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L4088
>>>>
>>>> I expect that it will help to increase the tests coverage by unit tests.
>>>>
>>>> Do we have any objections?
>>>>
>>>> --
>>>>
>>>> Timur,
>>>> Senior QA Engineer
>>>> OpenStack Projects
>>>> Mirantis Inc
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Anastasia Kuznetsova
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Best regards,
> Anastasia Kuznetsova
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

Timur,
Senior QA Engineer
OpenStack Projects
Mirantis Inc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150713/b5419bec/attachment.html>


More information about the OpenStack-dev mailing list