Open Stack

Akihiro Motoki wrote on 9/3/2019 11:12 PM:
> On Tue, Sep 3, 2019 at 10:18 PM Doug Hellmann <doug at doughellmann.com> wrote:
>>
>>
>>> On Sep 3, 2019, at 9:04 AM, Stephen Finucane <sfinucan at redhat.com> wrote:
>>>
>>> On Tue, 2019-09-03 at 08:42 -0400, Doug Hellmann wrote:
>>>>> On Sep 3, 2019, at 5:54 AM, Stephen Finucane <sfinucan at redhat.com> wrote:
>>>>>
>>>>> On Mon, 2019-09-02 at 15:31 -0400, Doug Hellmann wrote:
>>>>>>> On Sep 2, 2019, at 3:07 AM, Akihiro Motoki <amotoki at gmail.com> wrote:
>>>>> [snip]
>>>>>
>>>>>>> When the goal is defined the docs team thought the doc gate job can
>>>>>>> handle the PDF build
>>>>>>> without extra tox env and zuul job configuration. However, during
>>>>>>> implementing the zuul job support
>>>>>>> it turns out at least a new tox env or an extra zuul job configuration
>>>>>>> is required in each project
>>>>>>> to make the docs job fail when PDF build failure is detected. As a
>>>>>>> result, we changes the approach
>>>>>>> and the new tox target is now required in each project repo.
>>>>>> The whole point of structuring the goal the way we did was that we do
>>>>>> not want to update every single repo this cycle so we could roll out
>>>>>> PDF building transparently. We said we would allow the job to pass
>>>>>> even if the PDF build failed, because this was phase 1 of making all
>>>>>> of this work.
>>>>>>
>>>>>> The plan was to 1. extend the current job to make PDF building
>>>>>> optional; 2. examine the results to see how many repos need
>>>>>> significant work; 3. add a feature flag via a setting somewhere in
>>>>>> the repo to control whether the job fails if PDFs cannot be built.
>>>>>> That avoids a second doc job running in parallel, and still allows us
>>>>>> to roll out the PDF build requirement over time when  we have enough
>>>>>> information to do so.
>>>>> Unfortunately when we tried to implement this we found that virtually
>>>>> every project we looked at required _some_ amount of tweaks just to
>>>>> build, let alone look sensible. This was certainly true of the big
>>>>> service projects (nova, neutron, cinder, ...) which all ran afoul of a
>>>>> bug [1] in the Sphinx LaTeX builder. Given the issues with previous
>>>>> approach, such as the inability to easily reproduce locally and the
>>>>> general "hackiness" of the thing, along with the fact that we now had
>>>>> to submit changes against projects anyway, a collective decision was
>>>>> made [2] to drop that plan and persue the 'pdfdocs' tox target
>>>>> approach.
>>>> We wanted to avoid making a bunch of the same changes to projects just to
>>>> add the PDF building instructions. If the *content* of a project’s documentation
>>>> needs work, that’s different. We should make those changes.
>>> I thought the only reason to hack the docs venv in a Zuul job was to
>>> avoid having to mass patch projects to add tox configuration? As such,
>>> if we're already having to mass patch projects because they don't build
>>> otherwise, why wouldn't we add the tox configuration? Was there another
>>> reason to pursue the zuul-only approach that I've forgotten about/never
>>> knew?
>> I expected to need to fix formatting (even up to the point of commenting things
>> out, like we found with the giant config sample files). Those are content changes,
>> and would be mostly unique across projects.
>>
>> I wanted to avoid a large number of roughly identical changes to add tox environments,
>> zuul jobs, etc. because having a lot of patches like that across all the repos makes
>> extra work for small gain, especially when we can get the same results with a small
>> number of changes in one repository.
>>
>> The approach we discussed was to update the docs job to run some extra steps using
>> scripts that lived in the openstackdocstheme repository. That shouldn’t require
>> adding any extra software or otherwise modifying the tox environments. Did that approach
>> not work out?
> We explored ways only to update the docs job to run extra commands to
> build PDF docs,
> but there is one problem that the job cannot know whether PDF build is
> ready or not.
> If we ignore an error from PDF build, it works for repositories which
> are not ready for PDF build,
> but we cannot prevent PDF build failure again for repositories ready
> for PDF build
> As my project team hat of neutron team, we don't want to have PDF
> build failure again
> once the PDF build starts to work.
> To avoid this, stephenfin, asettle, AJaeger and I agree that some flag
> to determine if the PDF build
> is ready or not is needed. As of now, "pdf-docs" tox env is used as the flag.
> Another way we considered is a variable in openstack-tox-docs job, but
> we cannot pass a variable
> to zuul project template, so we didn't use this way.
> If there is a more efficient way, I am happy to use it.
>
> Thanks,
> Akihiro
>
Hello,


Sorry for joining in this thread late, but to I first would like to try 
to figure out the current status regarding the current discussion on the 
thread:

- openstackdocstheme has docstheme-build-pdf script [1]

- build-pdf-docs Zuul job in openstack-zuul-jobs pre-installs all 
required packages [2]

- Current guidance for project repos is that 1) is to just add to 
latex_documents settings [3] and add pdf-docs environment for trigger [4]

- Project repos additionally need to change more for successful PDF 
builds like adding more options on conf.py [5] and changing more on rst 
files to explictly options like [6] .


Now my questions from comments are:

a) How about checking an option in somewhere else like .zuul.yaml or 
using grep in docs env part, not doing grep to check the existance of 
"pdf-docs" tox env [3]?

b) Can we call docstheme-build-pdf in openstackdocstheme [1] instead of 
direct Sphinx & make commands in "pdf-docs" environment [4]?

c) Ultimately, would executing docstheme-build-pdf command in 
build-pdf-docs Zuul job with another kind of trigger like bullet a) be 
feasible and/or be implemented by the end of this cycle?



With many thanks,


/Ian


[1] https://review.opendev.org/#/c/665163/

[2] 
https://review.opendev.org/#/c/664555/25/roles/prepare-build-pdf-docs/tasks/main.yaml@3

[3] https://review.opendev.org/#/c/678393/4/doc/source/conf.py

[4] https://review.opendev.org/#/c/678393/4/tox.ini

[5] https://review.opendev.org/#/c/678747/1/doc/source/conf.py@270

[6] https://review.opendev.org/#/c/678747/1/doc/source/index.rst@13