Open Stack

Hi Timur,

I absolutely disagree with this approach. IMO such checks just helps to
organize the unhealthy dev process when contributors will write tests only
to pass this check. We have a non-voting coverage job to track the unit
tests coverage in addition to the code review itself. It's the
responsibility of the core team to ask for additional unit tests if they
are missed. For Sahara project specifically there are tons of places where
unit tests will be just mocks testing and such tests are mostly useless.
For the places not covered by unit tests we have tons of integration tests.
Currently, we trying to force contributors to cover the new code with unit
tests if it's really applicable.

Let my add some numbers. We have ~ 55% total code coverage by unit tests
and ~70% by integration tests (significantly shifted in terms of code
blocks coverage comparing to the unit tests). If we'll ignore the plugins
dir that contains mostly deployment code fully tested by integration tests
we'll have ~70% unit tests coverage.

Thanks.

On Thu, Jul 2, 2015 at 8:55 PM, Timur Nurlygayanov <
tnurlygayanov at mirantis.com> wrote:

> Hi all,
>
> I suggest to add CI job which will check the unit tests coverage for
> Sahara repository and will set -1 for commits with new code and without
> unit tests (if we have some degradation of tests coverage).
> This job successfully works for Rally project and it helps to organize the
> right code development process when developers write new unit tests for new
> functionality.
>
> we can just copy this job from Rally and start to use it for Sahara:
> Coverage control script:
> https://github.com/openstack/rally/blob/master/tests/ci/cover.sh
> Configuration file for coverage plugin (to exclude code which shouldn't be
> affected): https://github.com/openstack/rally/blob/master/.coveragerc
> Example of job in infra repository:
> https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L4088
>
> I expect that it will help to increase the tests coverage by unit tests.
>
> Do we have any objections?
>
> --
>
> Timur,
> Senior QA Engineer
> OpenStack Projects
> Mirantis Inc
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Sincerely yours,
Sergey Lukjanov
Sahara Technical Lead
(OpenStack Data Processing)
Principal Software Engineer
Mirantis Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150702/11e3bcd2/attachment.html>