Open Stack

tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
remove the feature at this point. However, this is going to necessitate some
mechanical changes for most projects with docs and this mail serves as a heads
up and request for input before we proceed.

--

Since pretty much the beginning, pbr has provided a custom 'build_sphinx'
setuptools command [1], which can be run like so:

    python setup.py build_sphinx

This is described in the pbr docs [2] and is is based on Sphinx's own
'build_sphinx' command [3], which is described in the Sphinx docs [4].

Historically, this custom version has provided a couple of features that the
upstream Sphinx extension didn't. These are outlined below. However, this
derivative has resulted in an inordinately large number of bugs for what should
be a simple feature (#1702872, #1691129, #1681983, #1674795, #1379998,
#1496882, ...). This is due to the lack of extension points in the upstream
code and the way that we've been forced to extend the command (we break almost
every time Sphinx change their code).

The latest of these bugs, #1702872, breaks a couple of things when using Sphinx
> 1.6:

- We no longer provide confoverrides when using Sphinx 1.6+, which means you
  may see "node class 'xxx' is already registered' errors, and the project
  name and version used in the docs may be incorrect [5]
- The '[build_sphinx] builders' in 'setup.cfg' option is no longer read, so
  only 'html' docs will be built (no man pages)
- The apidoc tool won't be automatically run

Fixing these issues is possible in the short-term and won't make the thing any
more maintainable going forward. Given the amount of issues this feature has
caused and the fact that Sphinx has since gained many of the features we
provided itself, we'd like to just drop the whole thing now. Some of the extra
this feature provide are no longer necessary, but the migration paths for the
ones that are are not great and will require mechanical changes to projects.

Here are the features that the plugin provides, along with the current
migration plans:

- Automatically runs sphinx-apidoc [6] before building docs

  It seems that adding the 'sphinx.ext.apidoc' extension to the 'extensions'
  list in 'doc/source/conf.py' will ensure this continues to happen
  automatically. However, I need to check this and will get back when I have.

- Automatically runs 'autodoc', which seems to do something similar to 'apidoc'
  but in a different way (I don't know what, tbh)

  Drop this feature entirely as 'apidoc' appears to do something very similar.

- Automatically sets project name and version using pbr's machinery

  Try to set this from 'openstackdocstheme'. If this isn't possible, folks will
  need to add some some lines to 'conf.py' like keystoneauth does [7]

- Supports multiple builders via the '[build_sphinx] builders' option in
  'setup.cfg'

  This is natively supported since Sphinx 1.6. You should replace
  '[build_sphinx] builders' with '[build_sphinx] builder' (singular) now

- Historically, supported turning build warning into build falures via the
  '[build_sphinx] warnerrors' option in 'setup.cfg'

  This is no longer supported by pbr, as Sphinx now provides a '[build_sphinx]
  warning-is-error' option. I wrote a load of patches to fix this recently
  using the topic 'sphinx15', but if I missed your project you need to remove
  '[pbr] warnerrors' from 'setup.cfg' and add '[build_sphinx]
  warning-is-error'. You should do this now.

Does anyone see any issues with this strategy going forward? If not, we're
going to start making changes to projects using the 'sphinx16' topic and would
appreciate reviews on these as they come in. We're not going to get to every
project, so if you can make the necessary changes to your own project then
please do.

Cheers,
Stephen
irc:stephenfin (formerly sfinucan)

- 

[1] https://github.com/openstack-dev/pbr/blob/master/pbr/builddoc.py
[2] https://docs.openstack.org/pbr/latest/user/using.html#build-sphinx
[3] https://github.com/sphinx-doc/sphinx/blob/master/sphinx/setup_command.py
[4] http://www.sphinx-doc.org/en/stable/setuptools.html
[5] http://paste.openstack.org/show/614730/
[5] www.sphinx-doc.org/en/stable/man/sphinx-apidoc.html
[7] https://github.com/openstack/keystoneauth/blob/master/doc/source/conf.py#L6
5-L72