Open Stack

Hi All,

In last week's Cross Project IRC meeting [1] I sought to determine the
merits and feasibility of having a central web location for code
coverage of various/all OpenStack projects.

Presently a few projects add code coverage as a non-voting gate and
that provides a per-commit viewing option, for example [2],[3].
I would first propose after a merge is committed, that coverage that
was generated for the commit is copied to a set and fixed URL so this
can be easily referenced in documentation, etc. (e.g.
coverage.openstack.org/openstack-infra/cover and
coverage.openstack.org/openstack/designate/cover respectively)

The purpose of recording code coverage is not to down vote any code
contributions due to lack of code coverage. The purpose includes:

1. Provide a central location to see current code coverage across
projects, and provide known public URLs.
2. Enable an entry point for discussion of improving code coverage,
especially as a means for newer developers to contribute to projects.
3. To provide analysis of project code coverage by recording coverage
reports over time.

fungi provided https://review.openstack.org/#/c/192253/ which is a
draft spec for a much wider ci-watch effort however I see this a more
specific example. I am seeking feedback from various projects on
interest, concerns and recommendations before creating a spec.

I see the initial steps to implement this as.

1. Agree on future domain url location for reports (e.g.
coverage.openstack.org)
2. Add to the merge committing process a step similar to the existing
coverage-log publisher (project-config/jenkins/jobs/macros.yaml) to
agreed location.
3. Provide initially a top level index page at url for published reports

These steps enable a single current occurrence of code coverage per
project. It is not the objective to keep the historical HTML output.

Following this, the next steps are to record coverage for historical
analysis with the following.

1. Alter tox.ini [testenv:cover]  to produce a text based version (aka
coverage report -m).  This is much easier to parse and compare
different versions. I would propose we keep a weekly/daily dated file
of this output, which would be automatically published as per previous
publisher.
2. Use these dated files to produce basic per project analysis of
lines/percentage coverage over time, publishing this to a set
directory structure (e.g. /analysis/) and linked from the top level
index page.  Over time this can be improved to provide simple
graphical output.



[1]
http://eavesdrop.openstack.org/meetings/crossproject/2015/crossproject.2015-08-25-21.00.html
[2]
http://logs.openstack.org/82/165682/8/check/nodepool-coverage/08c6340/cover/
[3]
http://logs.openstack.org/38/212738/12/check/designate-coverage/bc2e329/cover/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150831/42b71057/attachment.html>