Open Stack

First of all, thanks a lot for nice summary - I would like to deeply
read and put comments later.

And @mtreinish, please see my reply inline:

Matthew Treinish wrote on 9/14/2018 5:09 AM:
> On Thu, Sep 13, 2018 at 07:23:53AM -0600, Doug Hellmann wrote:
>> Excerpts from Michel Peterson's message of 2018-09-13 10:04:27 +0300:
>>> On Thu, Sep 13, 2018 at 1:09 AM, Doug Hellmann <doug at doughellmann.com>
>>> wrote:
>>>
>>>> The longer version is that we want to continue to use the existing
>>>> tox environment in each project as the basis for the job, since
>>>> that allows teams to control the version of python used, the
>>>> dependencies installed, and add custom steps to their build (such
>>>> as for pre-processing the documentation). So, the new or updated
>>>> job will start by running "tox -e docs" as it does today. Then it
>>>> will run Sphinx again with the instructions to build PDF output,
>>>> and copy the results into the directory that the publish job will
>>>> use to sync to the web server. And then it will run the scripts to
>>>> build translated versions of the documentation as HTML, and copy
>>>> the results into place for publishing.
>>>>
>>> Just a question out of curiosity. You mention that we still want to use the
>>> docs environment because it allows fine grained control over how the
>>> documentation is created. However, as I understand, the PDF output will
>>> happen in a more standardized way and outside of that fine grained control,
>>> right? That couldn't lead to differences in both documentations? Do we have
>>> to even worry about that?
>> Good question.  The idea is to run "tox -e docs" to get the regular
>> HTML, then something like
>>
>>    .tox/docs/bin/sphinx-build -b latex doc/build doc/build/latex
>>    cd doc/build/latex
>>    make
>>    cp doc/build/latex/*.pdf doc/build/html
> To be fair, I've looked at this several times in the past, and sphinx's latex
> generation is good enough for the simple case, but on more complex documents
> it doesn't really work too well. For example, on nova I added this a while ago:
>
> https://github.com/openstack/nova/blob/master/tools/build_latex_pdf.sh

After seeing what the script is doing, I wanna divide into several parts
and would like to tell with some generic approach:

- svg -> png
 : PDF builds ideally convert all svg files into PDF with no problems,
but there are some realistic problems
   such as problems on determining bounding sbox size on vector svg
files, and big memory problems with lots of tags in svg files.
 : Maybe it would be solved if we check all svg files with correct
formatting,
   or if all svg files are converted to png files with temporal changes
on rst file (.svg -> .png), wouldn't it?

- non-latin code problems:
 : By default, Sphinx uses latex builder, which doesn't support
non-latin codes and customized fonts [1].
   Documentation team tried to make use of xelatex instead of latex in
Sphinx configuration and now it is overridden
   on openstackdocstheme >=1.20. So non-latin code would not generate
problems if you use openstackdocstheme >=1.20.

- other things
 : I could not capture the background on other changes such as
additional packages.
   If u provide more background on other things, I would like to
investigate on how to approach by changing a rst file
   to make compatible with pdf builds or how to support all pdf builds
on many project repos as much as possible.

When I test PDF builds on current nova repo with master branch, it seems
that the rst document is too big
(876 pages with error) and more dealing with overcoming memory problems
was needed.
I would like to think how to overcome this, but it would be also nice if
someone shares advices or comments on this.


With many thanks,

/Ian

[1] https://tug.org/pipermail/xetex/2011-September/021324.html
[2] https://review.openstack.org/#/c/552070/5/openstackdocstheme/ext.py@227

> To work around some issues with this workflow. It was enough to get the
> generated latex to actually compile back then. But, that script has bitrotted
> and needs to be updated, because the latex from sphinx for nova's docs no
> longer compiles. (also I submitted a patch to sphinx in the meantime to
> fix the check mark latex output) I'm afraid that it'll be a constant game
> of cat and mouse trying to get everything to build.
>
> I think that we'll find that on most projects' documentation we will need
> to massage the latex output from sphinx to build pdfs.
>
> -Matt Treinish
>
>> We would run the HTML translation builds in a similar way by invoking
>> sphinx-build from the virtualenv repeatedly with different locale
>> settings based on what translations exist.
>>
>> In my earlier comment, I was thinking of the case where a team runs
>> a script to generate rst content files before invoking sphinx to
>> build the HTML. That script would have been run before the PDF
>> generation happens, so the content should be the same. That also
>> applies for anyone using sphinx add-ons, which will be available
>> to the latex builder because we'll be using the version of sphinx
>> installed in the virtualenv managed by tox.
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev