Open Stack

The tests in os-api-ref recently broke:

  http://logs.openstack.org/62/606362/1/check/openstack-tox-py35/8b709de/testr_results.html.gz

Specifically, we're seeing errors likes this:

  ft1.1: os_api_ref.tests.test_basic_example.TestBasicExample.test_rest_method_StringException: Traceback (most recent call last):
    File "/home/zuul/src/git.openstack.org/openstack/os-api-ref/.tox/py35/lib/python3.5/site-packages/sphinx_testing/util.py", line 143, in decorator
       func(*(args + (app, status, warning)), **kwargs)
     File "/home/zuul/src/git.openstack.org/openstack/os-api-ref/os_api_ref/tests/test_basic_example.py", line 41, in setUp
       self.html = (app.outdir / 'index.html').read_text(encoding='utf-8')
   TypeError: unsupported operand type(s) for /: 'str' and 'str'

Which is wrong because 'app.outdir' is not supposed to be an instance
of 'unicode' but rather 'sphinx_testing.path.path' [1] (which overrides
the '/' operator to act as concatenation because operator overloading
is always a good idea 😒) [2].

Anyway, we can go figure out what's changed here and handle it but this
is, at best, going to be a band aid. The fact is 'sphinx_testing' is
unmaintained and has been for some time now. The new hotness is
'sphinx.testing' [3], which is provided (with zero documentation) as
part of Sphinx. Unfortunately, this uses pytest fixtures [4] which I'm
pretty sure Monty (and a few others?) are vehemently against using in
OpenStack. That leaves us with three options:

 * Take over 'sphinx_testing' and bring it up-to-date. Maintain
   forever.
 * Start using 'sphinx.testing' and everything it comes with
 * Delete any tests that use 'sphinx_testing' and deal with the lack of
   coverage

For what it's worth, I use 'sphinx.testing' in 'sphinxcontrib-apidoc'
[5] without *too* many issues, but that lives outside the 'openstack'
namespace and isn't bound by upper-constraints and the likes.

Stephen

[1] https://github.com/sphinx-doc/sphinx-testing/blob/0.7.2/src/sphinx_testing/path.py#L217
[2] https://github.com/sphinx-doc/sphinx-testing/blob/0.7.2/src/sphinx_testing/util.py#L45-L63
[3] https://github.com/sphinx-doc/sphinx/tree/v1.8.0/sphinx/testing
[4] https://docs.pytest.org/en/latest/fixture.html
[5] https://github.com/sphinx-contrib/apidoc/blob/0.3.0/tests/test_ext.py