On Mon, 2019-09-02 at 16:07 +0900, Akihiro Motoki wrote:

On Mon, Sep 2, 2019 at 1:12 PM Thomas Bechtold <tbechtold@suse.com> wrote:

...

Hi,

On 8/27/19 7:58 AM, Akihiro Motoki wrote:

[...]

...

How to get started ------------------

- "How to get started" section in the PDF goal etherpad [1] explains the minimum steps. You can find useful examples there too.

This is a bit confusing because the goal[1] mentions that there should be no extra tox target declared for the gate job. But the etherpad explains that there should be a new tox target[2].

So do we need a new tox target in the project repo? Or is that optional and just for local testing?

The new tox target in the project repo is required now. The PDF doc will be generated only when the "pdf-docs" tox target does exists.

When the goal is defined the docs team thought the doc gate job can handle the PDF build without extra tox env and zuul job configuration. However, during implementing the zuul job support it turns out at least a new tox env or an extra zuul job configuration is required in each project to make the docs job fail when PDF build failure is detected. As a result, we changes the approach and the new tox target is now required in each project repo.

Perhaps we need to update the description of the goal definition document.

This is something I can propose. I will update here when I have updated. Thanks, Alex

Thanks, Akihiro

...

Cheers,

Tom

[1] https://governance.openstack.org/tc/goals/selected/train/pdf-doc-ge neration.html#completion-criteria [2] https://etherpad.openstack.org/p/train-pdf-support-goal

...

- To build PDF docs locally, you need to install LaTex related packages. See "To test locally" in the etherpad [1]. - If you hit problems during PDF build, check the common problems etherpad [2]. We are collecting knowledges there. - If you have questions, feel free to ask #openstack-doc IRC channel.

Also Please sign up your name to "Project volunteers" in [1].

Useful links ------------

[1] https://etherpad.openstack.org/p/train-pdf-support-goal [2] https://etherpad.openstack.org/p/pdf-goal-train-common-proble ms [3] Ongoing reviews: https://review.opendev.org/#/q/topic:build-pdf-docs+(status:open+ OR+status:merged)

Thanks, Akihiro Motoki (amotoki)

-- Alexandra Settle <a.settle@outlook.com> IRC: asettle

On Mon, 2019-09-02 at 15:31 -0400, Doug Hellmann wrote:

...

On Sep 2, 2019, at 3:07 AM, Akihiro Motoki <amotoki@gmail.com> wrote:

[snip]

...

When the goal is defined the docs team thought the doc gate job can handle the PDF build without extra tox env and zuul job configuration. However, during implementing the zuul job support it turns out at least a new tox env or an extra zuul job configuration is required in each project to make the docs job fail when PDF build failure is detected. As a result, we changes the approach and the new tox target is now required in each project repo.

The whole point of structuring the goal the way we did was that we do not want to update every single repo this cycle so we could roll out PDF building transparently. We said we would allow the job to pass even if the PDF build failed, because this was phase 1 of making all of this work.

The plan was to 1. extend the current job to make PDF building optional; 2. examine the results to see how many repos need significant work; 3. add a feature flag via a setting somewhere in the repo to control whether the job fails if PDFs cannot be built. That avoids a second doc job running in parallel, and still allows us to roll out the PDF build requirement over time when we have enough information to do so.

Unfortunately when we tried to implement this we found that virtually every project we looked at required _some_ amount of tweaks just to build, let alone look sensible. This was certainly true of the big service projects (nova, neutron, cinder, ...) which all ran afoul of a bug [1] in the Sphinx LaTeX builder. Given the issues with previous approach, such as the inability to easily reproduce locally and the general "hackiness" of the thing, along with the fact that we now had to submit changes against projects anyway, a collective decision was made [2] to drop that plan and persue the 'pdfdocs' tox target approach. If we're concerned about the difficulty of closing this out this cycle, I'd be in favour of just limiting our scope. IMO, the service projects are the ones that would benefit most from PDF documentation. These are the things people actually use and they tend to have the most complete documentation. Libraries can be self-documenting (yes, I know), in so far as once can use introspection, existing code examples, and the 'help' built-in to piece together what information they need. We should aim to close that gap long-term, but for now requiring modifications to ensure we have _some_ PDFs sounds a lot better than requiring no modifications and having no PDFs. Cheers, Stephen [1] https://github.com/sphinx-doc/sphinx/issues/3099 [2] http://eavesdrop.openstack.org/irclogs/%23openstack-doc/%23openstack-doc.201...

...

Perhaps we need to update the description of the goal definition document.

I don’t think it’s a good idea to change the scope of the goal at this point in the release cycle.

On Sep 3, 2019, at 5:54 AM, Stephen Finucane <sfinucan@redhat.com> wrote:

On Mon, 2019-09-02 at 15:31 -0400, Doug Hellmann wrote:

...

...

On Sep 2, 2019, at 3:07 AM, Akihiro Motoki <amotoki@gmail.com> wrote:

[snip]

...

...

When the goal is defined the docs team thought the doc gate job can handle the PDF build without extra tox env and zuul job configuration. However, during implementing the zuul job support it turns out at least a new tox env or an extra zuul job configuration is required in each project to make the docs job fail when PDF build failure is detected. As a result, we changes the approach and the new tox target is now required in each project repo.

The whole point of structuring the goal the way we did was that we do not want to update every single repo this cycle so we could roll out PDF building transparently. We said we would allow the job to pass even if the PDF build failed, because this was phase 1 of making all of this work.

The plan was to 1. extend the current job to make PDF building optional; 2. examine the results to see how many repos need significant work; 3. add a feature flag via a setting somewhere in the repo to control whether the job fails if PDFs cannot be built. That avoids a second doc job running in parallel, and still allows us to roll out the PDF build requirement over time when we have enough information to do so.

Unfortunately when we tried to implement this we found that virtually every project we looked at required _some_ amount of tweaks just to build, let alone look sensible. This was certainly true of the big service projects (nova, neutron, cinder, ...) which all ran afoul of a bug [1] in the Sphinx LaTeX builder. Given the issues with previous approach, such as the inability to easily reproduce locally and the general "hackiness" of the thing, along with the fact that we now had to submit changes against projects anyway, a collective decision was made [2] to drop that plan and persue the 'pdfdocs' tox target approach.

We wanted to avoid making a bunch of the same changes to projects just to add the PDF building instructions. If the *content* of a project’s documentation needs work, that’s different. We should make those changes.

If we're concerned about the difficulty of closing this out this cycle, I'd be in favour of just limiting our scope. IMO, the service projects are the ones that would benefit most from PDF documentation. These are the things people actually use and they tend to have the most complete documentation. Libraries can be self-documenting (yes, I know), in so far as once can use introspection, existing code examples, and the 'help' built-in to piece together what information they need. We should aim to close that gap long-term, but for now requiring modifications to ensure we have _some_ PDFs sounds a lot better than requiring no modifications and having no PDFs.

Cheers, Stephen

[1] https://github.com/sphinx-doc/sphinx/issues/3099 [2] http://eavesdrop.openstack.org/irclogs/%23openstack-doc/%23openstack-doc.201...

...

...

Perhaps we need to update the description of the goal definition document.

I don’t think it’s a good idea to change the scope of the goal at this point in the release cycle.

On Sep 3, 2019, at 9:04 AM, Stephen Finucane <sfinucan@redhat.com> wrote:

On Tue, 2019-09-03 at 08:42 -0400, Doug Hellmann wrote:

...

...

On Sep 3, 2019, at 5:54 AM, Stephen Finucane <sfinucan@redhat.com> wrote:

On Mon, 2019-09-02 at 15:31 -0400, Doug Hellmann wrote:

...

...

On Sep 2, 2019, at 3:07 AM, Akihiro Motoki <amotoki@gmail.com> wrote:

[snip]

...

...

When the goal is defined the docs team thought the doc gate job can handle the PDF build without extra tox env and zuul job configuration. However, during implementing the zuul job support it turns out at least a new tox env or an extra zuul job configuration is required in each project to make the docs job fail when PDF build failure is detected. As a result, we changes the approach and the new tox target is now required in each project repo.

The whole point of structuring the goal the way we did was that we do not want to update every single repo this cycle so we could roll out PDF building transparently. We said we would allow the job to pass even if the PDF build failed, because this was phase 1 of making all of this work.

The plan was to 1. extend the current job to make PDF building optional; 2. examine the results to see how many repos need significant work; 3. add a feature flag via a setting somewhere in the repo to control whether the job fails if PDFs cannot be built. That avoids a second doc job running in parallel, and still allows us to roll out the PDF build requirement over time when we have enough information to do so.

Unfortunately when we tried to implement this we found that virtually every project we looked at required _some_ amount of tweaks just to build, let alone look sensible. This was certainly true of the big service projects (nova, neutron, cinder, ...) which all ran afoul of a bug [1] in the Sphinx LaTeX builder. Given the issues with previous approach, such as the inability to easily reproduce locally and the general "hackiness" of the thing, along with the fact that we now had to submit changes against projects anyway, a collective decision was made [2] to drop that plan and persue the 'pdfdocs' tox target approach.

We wanted to avoid making a bunch of the same changes to projects just to add the PDF building instructions. If the *content* of a project’s documentation needs work, that’s different. We should make those changes.

I thought the only reason to hack the docs venv in a Zuul job was to avoid having to mass patch projects to add tox configuration? As such, if we're already having to mass patch projects because they don't build otherwise, why wouldn't we add the tox configuration? Was there another reason to pursue the zuul-only approach that I've forgotten about/never knew?

I expected to need to fix formatting (even up to the point of commenting things out, like we found with the giant config sample files). Those are content changes, and would be mostly unique across projects. I wanted to avoid a large number of roughly identical changes to add tox environments, zuul jobs, etc. because having a lot of patches like that across all the repos makes extra work for small gain, especially when we can get the same results with a small number of changes in one repository. The approach we discussed was to update the docs job to run some extra steps using scripts that lived in the openstackdocstheme repository. That shouldn’t require adding any extra software or otherwise modifying the tox environments. Did that approach not work out? Doug

Akihiro Motoki wrote on 9/3/2019 11:12 PM:

On Tue, Sep 3, 2019 at 10:18 PM Doug Hellmann <doug@doughellmann.com> wrote:

...

...

On Sep 3, 2019, at 9:04 AM, Stephen Finucane <sfinucan@redhat.com> wrote:

On Tue, 2019-09-03 at 08:42 -0400, Doug Hellmann wrote:

...

...

On Sep 3, 2019, at 5:54 AM, Stephen Finucane <sfinucan@redhat.com> wrote:

On Mon, 2019-09-02 at 15:31 -0400, Doug Hellmann wrote:

...

> On Sep 2, 2019, at 3:07 AM, Akihiro Motoki <amotoki@gmail.com> wrote: [snip]

...

> When the goal is defined the docs team thought the doc gate job can > handle the PDF build > without extra tox env and zuul job configuration. However, during > implementing the zuul job support > it turns out at least a new tox env or an extra zuul job configuration > is required in each project > to make the docs job fail when PDF build failure is detected. As a > result, we changes the approach > and the new tox target is now required in each project repo. The whole point of structuring the goal the way we did was that we do not want to update every single repo this cycle so we could roll out PDF building transparently. We said we would allow the job to pass even if the PDF build failed, because this was phase 1 of making all of this work.

The plan was to 1. extend the current job to make PDF building optional; 2. examine the results to see how many repos need significant work; 3. add a feature flag via a setting somewhere in the repo to control whether the job fails if PDFs cannot be built. That avoids a second doc job running in parallel, and still allows us to roll out the PDF build requirement over time when we have enough information to do so. Unfortunately when we tried to implement this we found that virtually every project we looked at required _some_ amount of tweaks just to build, let alone look sensible. This was certainly true of the big service projects (nova, neutron, cinder, ...) which all ran afoul of a bug [1] in the Sphinx LaTeX builder. Given the issues with previous approach, such as the inability to easily reproduce locally and the general "hackiness" of the thing, along with the fact that we now had to submit changes against projects anyway, a collective decision was made [2] to drop that plan and persue the 'pdfdocs' tox target approach. We wanted to avoid making a bunch of the same changes to projects just to add the PDF building instructions. If the *content* of a project’s documentation needs work, that’s different. We should make those changes. I thought the only reason to hack the docs venv in a Zuul job was to avoid having to mass patch projects to add tox configuration? As such, if we're already having to mass patch projects because they don't build otherwise, why wouldn't we add the tox configuration? Was there another reason to pursue the zuul-only approach that I've forgotten about/never knew? I expected to need to fix formatting (even up to the point of commenting things out, like we found with the giant config sample files). Those are content changes, and would be mostly unique across projects.

I wanted to avoid a large number of roughly identical changes to add tox environments, zuul jobs, etc. because having a lot of patches like that across all the repos makes extra work for small gain, especially when we can get the same results with a small number of changes in one repository.

The approach we discussed was to update the docs job to run some extra steps using scripts that lived in the openstackdocstheme repository. That shouldn’t require adding any extra software or otherwise modifying the tox environments. Did that approach not work out?

We explored ways only to update the docs job to run extra commands to build PDF docs, but there is one problem that the job cannot know whether PDF build is ready or not. If we ignore an error from PDF build, it works for repositories which are not ready for PDF build, but we cannot prevent PDF build failure again for repositories ready for PDF build As my project team hat of neutron team, we don't want to have PDF build failure again once the PDF build starts to work. To avoid this, stephenfin, asettle, AJaeger and I agree that some flag to determine if the PDF build is ready or not is needed. As of now, "pdf-docs" tox env is used as the flag. Another way we considered is a variable in openstack-tox-docs job, but we cannot pass a variable to zuul project template, so we didn't use this way. If there is a more efficient way, I am happy to use it.

Thanks, Akihiro

Hello, Sorry for joining in this thread late, but to I first would like to try to figure out the current status regarding the current discussion on the thread: - openstackdocstheme has docstheme-build-pdf script [1] - build-pdf-docs Zuul job in openstack-zuul-jobs pre-installs all required packages [2] - Current guidance for project repos is that 1) is to just add to latex_documents settings [3] and add pdf-docs environment for trigger [4] - Project repos additionally need to change more for successful PDF builds like adding more options on conf.py [5] and changing more on rst files to explictly options like [6] . Now my questions from comments are: a) How about checking an option in somewhere else like .zuul.yaml or using grep in docs env part, not doing grep to check the existance of "pdf-docs" tox env [3]? b) Can we call docstheme-build-pdf in openstackdocstheme [1] instead of direct Sphinx & make commands in "pdf-docs" environment [4]? c) Ultimately, would executing docstheme-build-pdf command in build-pdf-docs Zuul job with another kind of trigger like bullet a) be feasible and/or be implemented by the end of this cycle? With many thanks, /Ian [1] https://review.opendev.org/#/c/665163/ [2] https://review.opendev.org/#/c/664555/25/roles/prepare-build-pdf-docs/tasks/... [3] https://review.opendev.org/#/c/678393/4/doc/source/conf.py [4] https://review.opendev.org/#/c/678393/4/tox.ini [5] https://review.opendev.org/#/c/678747/1/doc/source/conf.py@270 [6] https://review.opendev.org/#/c/678747/1/doc/source/index.rst@13

Akihiro Motoki wrote on 9/4/2019 11:06 PM:

On Wed, Sep 4, 2019 at 12:43 AM Ian Y. Choi <ianyrchoi@gmail.com> wrote:

...

Akihiro Motoki wrote on 9/3/2019 11:12 PM:

...

On Tue, Sep 3, 2019 at 10:18 PM Doug Hellmann <doug@doughellmann.com> wrote:

...

...

On Sep 3, 2019, at 9:04 AM, Stephen Finucane <sfinucan@redhat.com> wrote:

On Tue, 2019-09-03 at 08:42 -0400, Doug Hellmann wrote:

...

> On Sep 3, 2019, at 5:54 AM, Stephen Finucane <sfinucan@redhat.com> wrote: > > On Mon, 2019-09-02 at 15:31 -0400, Doug Hellmann wrote: >>> On Sep 2, 2019, at 3:07 AM, Akihiro Motoki <amotoki@gmail.com> wrote: > [snip] > >>> When the goal is defined the docs team thought the doc gate job can >>> handle the PDF build >>> without extra tox env and zuul job configuration. However, during >>> implementing the zuul job support >>> it turns out at least a new tox env or an extra zuul job configuration >>> is required in each project >>> to make the docs job fail when PDF build failure is detected. As a >>> result, we changes the approach >>> and the new tox target is now required in each project repo. >> The whole point of structuring the goal the way we did was that we do >> not want to update every single repo this cycle so we could roll out >> PDF building transparently. We said we would allow the job to pass >> even if the PDF build failed, because this was phase 1 of making all >> of this work. >> >> The plan was to 1. extend the current job to make PDF building >> optional; 2. examine the results to see how many repos need >> significant work; 3. add a feature flag via a setting somewhere in >> the repo to control whether the job fails if PDFs cannot be built. >> That avoids a second doc job running in parallel, and still allows us >> to roll out the PDF build requirement over time when we have enough >> information to do so. > Unfortunately when we tried to implement this we found that virtually > every project we looked at required _some_ amount of tweaks just to > build, let alone look sensible. This was certainly true of the big > service projects (nova, neutron, cinder, ...) which all ran afoul of a > bug [1] in the Sphinx LaTeX builder. Given the issues with previous > approach, such as the inability to easily reproduce locally and the > general "hackiness" of the thing, along with the fact that we now had > to submit changes against projects anyway, a collective decision was > made [2] to drop that plan and persue the 'pdfdocs' tox target > approach. We wanted to avoid making a bunch of the same changes to projects just to add the PDF building instructions. If the *content* of a project’s documentation needs work, that’s different. We should make those changes. I thought the only reason to hack the docs venv in a Zuul job was to avoid having to mass patch projects to add tox configuration? As such, if we're already having to mass patch projects because they don't build otherwise, why wouldn't we add the tox configuration? Was there another reason to pursue the zuul-only approach that I've forgotten about/never knew? I expected to need to fix formatting (even up to the point of commenting things out, like we found with the giant config sample files). Those are content changes, and would be mostly unique across projects.

I wanted to avoid a large number of roughly identical changes to add tox environments, zuul jobs, etc. because having a lot of patches like that across all the repos makes extra work for small gain, especially when we can get the same results with a small number of changes in one repository.

The approach we discussed was to update the docs job to run some extra steps using scripts that lived in the openstackdocstheme repository. That shouldn’t require adding any extra software or otherwise modifying the tox environments. Did that approach not work out?

We explored ways only to update the docs job to run extra commands to build PDF docs, but there is one problem that the job cannot know whether PDF build is ready or not. If we ignore an error from PDF build, it works for repositories which are not ready for PDF build, but we cannot prevent PDF build failure again for repositories ready for PDF build As my project team hat of neutron team, we don't want to have PDF build failure again once the PDF build starts to work. To avoid this, stephenfin, asettle, AJaeger and I agree that some flag to determine if the PDF build is ready or not is needed. As of now, "pdf-docs" tox env is used as the flag. Another way we considered is a variable in openstack-tox-docs job, but we cannot pass a variable to zuul project template, so we didn't use this way. If there is a more efficient way, I am happy to use it.

Thanks, Akihiro

Hello,

Sorry for joining in this thread late, but to I first would like to try to figure out the current status regarding the current discussion on the thread:

- openstackdocstheme has docstheme-build-pdf script [1]

- build-pdf-docs Zuul job in openstack-zuul-jobs pre-installs all required packages [2]

- Current guidance for project repos is that 1) is to just add to latex_documents settings [3] and add pdf-docs environment for trigger [4]

- Project repos additionally need to change more for successful PDF builds like adding more options on conf.py [5] and changing more on rst files to explictly options like [6] . Thanks Ian.

Your understanding on the current situations is correct. Good summary, thanks.

...

Now my questions from comments are:

a) How about checking an option in somewhere else like .zuul.yaml or using grep in docs env part, not doing grep to check the existance of "pdf-docs" tox env [3]?

I am not sure how your suggestion works more efficiently than the current pdf-docs tox env approach. We explored an option to introduce a flag variable to the openstack-tox-docs job but we use a zuul project-template which wraps openstack-tox-docs job and another job. The current zuul project-template does not accept a variable and projects who want to specify a flag explicitly needs to copy the content of the project-template. Considering this we gave up this route. Regarding "using grep in docs env part", I haven't understood what you think, but it looks similar to the current approach.

...

b) Can we call docstheme-build-pdf in openstackdocstheme [1] instead of direct Sphinx & make commands in "pdf-docs" environment [4]? It can, but I am not sure whether we need to update the current proposed patches. The only advantage of using docstheme-build-pdf is that we don't need to change project repositories when we update the command lines in future, but it sounds a matter of taste.

...

c) Ultimately, would executing docstheme-build-pdf command in build-pdf-docs Zuul job with another kind of trigger like bullet a) be feasible and/or be implemented by the end of this cycle? We can, but again it is a matter of taste to me and most important thing is how we handle a flag to enable PDF build.

Thanks, Akihiro

Thank you for sharing your opinion, and I agree that it can be the matter of taste. I wanted to emphasize that the changes to project repositories are rather so small, and have tried to explore which ways can more minimize the changes to project repositories (e.g., without any change on tox.ini in project repositories). By the way, is it possible to centralize such flags into a common repository such as a repo related with build-pdf-docs Zuul job like [1] and [2] (I took examples from I18n team)? I am asking since I also agree that it would be the best if the same changes to all repos' tox.ini and other files could be minimized. If it isn't possible, than I think there would be no alternatives. Note that my asking assumes that current PDF community goal well reflects what I previously discussed with Doug [3]. With many thanks, /Ian [1] https://review.opendev.org/#/c/525028/1/zuul.d/projects.yaml [2] https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/roles/pr... [3] http://lists.openstack.org/pipermail/openstack-dev/2018-September/134609.htm...

...

With many thanks,

/Ian

[1] https://review.opendev.org/#/c/665163/

[2] https://review.opendev.org/#/c/664555/25/roles/prepare-build-pdf-docs/tasks/...

[3] https://review.opendev.org/#/c/678393/4/doc/source/conf.py

[4] https://review.opendev.org/#/c/678393/4/tox.ini

[5] https://review.opendev.org/#/c/678747/1/doc/source/conf.py@270

[6] https://review.opendev.org/#/c/678747/1/doc/source/index.rst@13