Open Stack

Hello,

In a prior thread we discussed moving coverage jobs from "post" jobs
to the check queue [1].

Firstly, if you have no idea what I'm talking about, the coverage jobs
run the unit tests under the coverage tool [2] and produce some output
like [3] which identifies which parts of the code have been executed.

It seems the history of these jobs in "post" was that they took a
little too long in the check queue with added measurement overhead.
Looking at, for example, nova jobs, the coverage job is currently in
the 10 minute range.  I'm not sure if the coverage tool has got better
or nova reduced the job time; both are likely.  This has led to much
inconsistency as to where we run the coverage jobs -- it mostly looks
like where they run depends on which other project you used as a
template.  Some projects run in both check and post which seems
unnecessary.

The coverage job results in post are quite hard to find.  You need to
firstly know the job even runs, then find the correct commit sha
(which is probably the merge commit, not the one in gerrit) and then
know how to manually navigate the logs.openstack.org file-hierarchy.
It's probably no surprise that according to apache logs, nobody has
accessed a post coverage-job output at all within about the last
month.  Also, as per the prior email, if the job is actually failing
you get no notification at all.

A recent change has made "-nv" (non-voting) coverage jobs available
[4] which simplifies this somewhat, as there is no need to put special
regexes in to stop voting.

I have proposed [5] which moves all coverage post jobs to non-voting
check queue jobs.  It also renames them with our standard "gate-" for
consistency.  I believe this will improve usage of the tool and also
clean-up our layout.yaml a bit.

Feel free to raise concerns in [5]

Thanks,

-i

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-July/099491.html
[2] https://coverage.readthedocs.io/en/coverage-4.3.4/
[3] http://logs.openstack.org/c3/c3671ee7da154e251c2915d4aced2b1a2bd8dfa9/post/nova-coverage-db-ubuntu-xenial/bc21639/cover/
[4] https://review.openstack.org/431783
[5] https://review.openstack.org/432836