On 2016-07-13 13:25:44 +0200 (+0200), Markus Zoeller wrote: > For some reason the gate docs job didn't find the issue (wrong json > format) which got fixed with change [1]. It doesn't even emit a warning. > Locally, the execution of "tox -e docs" does find the issue. Can we make > the gate docs job aware of such json format issues? [...] It looks like your "docs" tox env is doing some additional checks[1] with the json.tool module. The gate-{name}-docs CI jobs don't call `tox -e docs` but `tox -evenv -- python setup.py build_sphinx` directly[2]. Changing that has been discussed[3][4] (and rejected/withdrawn) by the TC in the past. I agree with Andreas that if you want additional checking of this you probably need it in a different job. The same goes for tests of your config/policy generation and API guide/reference builds. The goal of gate-{name}-docs is to make sure that the setup.py build_sphinx entrypoint works across all covered projects (per the CTI[5]) without requiring any extra nonstandard magic. [1] http://git.openstack.org/cgit/openstack/nova/tree/tox.ini?id=df0aa8a#n97 [2] http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/run-docs.sh [3] http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-09-30-20.01.log.html#l-27 [4] https://review.openstack.org/119875 [5] http://governance.openstack.org/reference/cti/python_cti.html#documentation -- Jeremy Stanley