[all] Please remove all external resources from docs
Hi,
I've wrote about this earlier I guess, but I believe I need to do it once more. Often I see in the docs things like this:
.. image:: https://governance.openstack.org/tc/badges/<FOO>.svg :target: https://governance.openstack.org/tc/reference/tags/index.html
[note: this is only one simple example, there's many of other types linking to external images, I've even seen stuff linking to CDNs...]
I'd like to see these go away. Indeed, distributions like mine (ie: Debian) are packaging the doc. And it's *very* annoying when one browses the doc to have such an external link, and the browser going on the internet to fetch an external resource.
So, in Debian, I'm patching these away. Often, the most ugly way where I don't care about the resulting document (as long as the doc is there). This is both annoying, and a loss of my time.
The solution is simple: have the resource being *LOCAL* (ie: stored in the project's doc), not stored on an external site.
Thanks to anyone who will follow this advice.
Cheers,
Thomas Goirand (zigo)
On 2020-04-06 17:12:30 +0200 (+0200), Thomas Goirand wrote:
I've wrote about this earlier I guess, but I believe I need to do it once more. Often I see in the docs things like this:
.. image:: https://governance.openstack.org/tc/badges/<FOO>.svg :target: https://governance.openstack.org/tc/reference/tags/index.html
[...]
The solution is simple: have the resource being *LOCAL* (ie: stored in the project's doc), not stored on an external site.
[...]
I'm not a fan of this "feature" myself, but feel compelled to point out that it's intentionally dynamic content dependent on metadata from another repository which is intended to change at different times than the document itself changes, so including a copy of that file would make little sense. It's entirely there so that projects can have the same feel-good "badges" which they see displayed in the README files of other projects on GitHub. It exists purely so that real-time RST-to-HTML renderers will display the current state of a number of bits of governance metadata, so stripping those out of packaged documentation builds is the right thing to do. Maybe we could make it easier to have our documentation builds strip that content automatically so that distro package maintainers don't need to?
On 06/04/2020 22:18, Jeremy Stanley wrote:
On 2020-04-06 17:12:30 +0200 (+0200), Thomas Goirand wrote:
I've wrote about this earlier I guess, but I believe I need to do it once more. Often I see in the docs things like this:
.. image:: https://governance.openstack.org/tc/badges/<FOO>.svg :target: https://governance.openstack.org/tc/reference/tags/index.html
[...]
The solution is simple: have the resource being *LOCAL* (ie: stored in the project's doc), not stored on an external site.
[...]
I'm not a fan of this "feature" myself, but feel compelled to point out that it's intentionally dynamic content dependent on metadata from another repository which is intended to change at different times than the document itself changes, so including a copy of that file would make little sense.
It *might* make sense - the info it shows is time dependent, so if a project was asserting follows-stable-policy in pike, showing that is not necessarily _wrong_ ... Same as if a project was moved out of OpenStack in Havana, the docs could show the openstack project badge until then.[1]
1 - I am aware the badges didn't exist in havana
It's entirely there so that projects
can have the same feel-good "badges" which they see displayed in the README files of other projects on GitHub. It exists purely so that real-time RST-to-HTML renderers will display the current state of a number of bits of governance metadata, so stripping those out of packaged documentation builds is the right thing to do. Maybe we could make it easier to have our documentation builds strip that content automatically so that distro package maintainers don't need to?
On 2020-04-07 10:47:41 +0100 (+0100), Graham Hayes wrote: [...]
It *might* make sense - the info it shows is time dependent, so if a project was asserting follows-stable-policy in pike, showing that is not necessarily _wrong_ ... Same as if a project was moved out of OpenStack in Havana, the docs could show the openstack project badge until then.
[...]
Fair, if folks prefer that, then the docs build jobs could require the governance repo to obtain the requisite metadata and incorporate the Sphinx extension which generates these SVG images. That still might be unworkably complicated for downstream packaging in distros however, since we don't version and release the contents of the governance repo, so would likely need to start doing that (at least for the projects.yaml and Sphinx extension) and declare it as a min-versioned docs build dependency. Seems like a lot of work compared to just stripping them out. My point was that these badges are mostly only relevant to folks browsing README files in a source code hosting platform, but when the README is embedded into a compiled Sphinx document they seem really out of place and are probably best omitted.
On Tue, 2020-04-07 at 13:52 +0000, Jeremy Stanley wrote:
On 2020-04-07 10:47:41 +0100 (+0100), Graham Hayes wrote: [...]
It *might* make sense - the info it shows is time dependent, so if a project was asserting follows-stable-policy in pike, showing that is not necessarily _wrong_ ... Same as if a project was moved out of OpenStack in Havana, the docs could show the openstack project badge until then.
[...]
Fair, if folks prefer that, then the docs build jobs could require the governance repo to obtain the requisite metadata and incorporate the Sphinx extension which generates these SVG images.
for what its worth im not sure why this thread was suddenly revied after 6 months or maybe i was ignoring it but we discussed this as part of the pdf docs item back in denver and for the pdf docs we did want to convert the badges into svgs and enbed them but i dont know if that was ever done because it too a lot of time to get the initally docs builing correctly so i think the task to be able to embed rather then link to the badges may have fallen by the way side.
That still might be unworkably complicated for downstream packaging in distros however, since we don't version and release the contents of the governance repo, so would likely need to start doing that (at least for the projects.yaml and Sphinx extension) and declare it as a min-versioned docs build dependency. Seems like a lot of work compared to just stripping them out. My point was that these badges are mostly only relevant to folks browsing README files in a source code hosting platform, but when the README is embedded into a compiled Sphinx document they seem really out of place and are probably best omitted.
a simple anser would also be to not ship the readme in the docs or do its as plain text. i presonally prefer to read the rst before they are renderd anyway so if i was looking for the docs in a a disto package i would be hoping to find the rst files not the html docs.
On 2020-04-07 16:00:37 +0100 (+0100), Sean Mooney wrote: [...]
for what its worth im not sure why this thread was suddenly revied after 6 months or maybe i was ignoring it but we discussed this as part of the pdf docs item back in denver and for the pdf docs we did want to convert the badges into svgs and enbed them but i dont know if that was ever done because it too a lot of time to get the initally docs builing correctly so i think the task to be able to embed rather then link to the badges may have fallen by the way side.
[...]
And I guess that's why it's coming back up again, but also with a desire to do it for the HTML doc builds not just the PDFs.
a simple anser would also be to not ship the readme in the docs or do its as plain text. i presonally prefer to read the rst before they are renderd anyway so if i was looking for the docs in a a disto package i would be hoping to find the rst files not the html docs.
Some projects put a thorough introduction in their README.rst and don't want to have to duplicate it all in the Sphinx doc source tree so just stick a ".. include:: ../../README.rst" line into doc/source/index.rst or a similar document. It's typical for Debian packages (though I haven't looked at Thomas's) to put the raw readme somewhere like /usr/share/doc/foo/README.rst but also stick HTML builds of the full documentation in a place like /usr/share/doc/foo/html/ for ease of local indexing with tools like:
https://manpages.debian.org/dhelp https://manpages.debian.org/dwww
The entire point of shipping these HTML builds in the compiled packages is so that users have a local copy of documentation even if they're somewhere with no working Internet access, and can even build them from local copies of the source packages without working Internet, therefore they need to not depend on external assets out on the Internet and have to instead be buildable with only what's packaged in the distro.
Hi Guys!
Thanks for participating in this thread. I feel less alone now! :)
On 4/6/20 11:18 PM, Jeremy Stanley wrote:
Maybe we could make it easier to have our documentation builds strip that content automatically so that distro package maintainers don't need to?
That'd be really awesome!
On 4/7/20 3:52 PM, Jeremy Stanley wrote:
My point was that these badges are mostly only relevant to folks browsing README files in a source code hosting platform, but when the README is embedded into a compiled Sphinx document they seem really out of place and are probably best omitted.
Yes. Not only that. In Debian we consider this a "privacy breach" (see below).
On 4/7/20 5:00 PM, Sean Mooney wrote:
for what its worth im not sure why this thread was suddenly revied after 6 months
Because, as for each new release of OpenStack over the last 9 years, I'm being moronic about re-occurring issues in the Debian packages I maintain, and I didn't want to skip this tradition! :)
On 4/7/20 5:58 PM, Jeremy Stanley wrote:
Some projects put a thorough introduction in their README.rst and don't want to have to duplicate it all in the Sphinx doc source tree so just stick a ".. include:: ../../README.rst" line into doc/source/index.rst or a similar document. It's typical for Debian packages (though I haven't looked at Thomas's) to put the raw readme somewhere like /usr/share/doc/foo/README.rst but also stick HTML builds of the full documentation in a place like /usr/share/doc/foo/html/
I typically only ship the HTML documentation, as I feel like it looks much nicer than just the rst format. If one wants the raw format, well we do have source packages... :)
The entire point of shipping these HTML builds in the compiled packages is so that users have a local copy of documentation even if they're somewhere with no working Internet access, and can even build them from local copies of the source packages without working Internet, therefore they need to not depend on external assets out on the Internet and have to instead be buildable with only what's packaged in the distro.
Yes. But in this case, the problem is that the resulting HTML documentation ends up with an image looking like this:
<img src="http://<something-not-on-my-laptop>/ ...
which means:
1/ the image wont display without internet 2/ it will make a query to a website, revealing to a 3rd party that I'm browsing the documentation. That's a privacy breach...
Of course, the OpenStack Foundation has good intentions, and I don't think it will make any use of these stats. Still, this is annoying.
I therefore keep on patching these away from the README.rst, which is exhausting, because I constantly need to rebase the patches, and it's really all over the place on the 400+ source packages I maintain.
Last note: downloading the badges at build time is *not* an option (as all distros requires to build without internet connectivity).
Cheers,
Thomas Goirand (zigo)
On Wed, Apr 8, 2020 at 6:50 PM Thomas Goirand zigo@debian.org wrote:
Hi Guys!
Thanks for participating in this thread. I feel less alone now! :)
On 4/6/20 11:18 PM, Jeremy Stanley wrote:
Maybe we could make it easier to have our documentation builds strip that content automatically so that distro package maintainers don't need to?
That'd be really awesome!
On 4/7/20 3:52 PM, Jeremy Stanley wrote:
My point was that these badges are mostly only relevant to folks browsing README files in a source code hosting platform, but when the README is embedded into a compiled Sphinx document they seem really out of place and are probably best omitted.
Yes. Not only that. In Debian we consider this a "privacy breach" (see below).
On 4/7/20 5:00 PM, Sean Mooney wrote:
for what its worth im not sure why this thread was suddenly revied after 6 months
Because, as for each new release of OpenStack over the last 9 years, I'm being moronic about re-occurring issues in the Debian packages I maintain, and I didn't want to skip this tradition! :)
On 4/7/20 5:58 PM, Jeremy Stanley wrote:
Some projects put a thorough introduction in their README.rst and don't want to have to duplicate it all in the Sphinx doc source tree so just stick a ".. include:: ../../README.rst" line into doc/source/index.rst or a similar document. It's typical for Debian packages (though I haven't looked at Thomas's) to put the raw readme somewhere like /usr/share/doc/foo/README.rst but also stick HTML builds of the full documentation in a place like /usr/share/doc/foo/html/
I typically only ship the HTML documentation, as I feel like it looks much nicer than just the rst format. If one wants the raw format, well we do have source packages... :)
The entire point of shipping these HTML builds in the compiled packages is so that users have a local copy of documentation even if they're somewhere with no working Internet access, and can even build them from local copies of the source packages without working Internet, therefore they need to not depend on external assets out on the Internet and have to instead be buildable with only what's packaged in the distro.
Yes. But in this case, the problem is that the resulting HTML documentation ends up with an image looking like this:
<img src="http://<something-not-on-my-laptop>/ ...
which means:
1/ the image wont display without internet 2/ it will make a query to a website, revealing to a 3rd party that I'm browsing the documentation. That's a privacy breach...
Of course, the OpenStack Foundation has good intentions, and I don't think it will make any use of these stats. Still, this is annoying.
I therefore keep on patching these away from the README.rst, which is exhausting, because I constantly need to rebase the patches, and it's really all over the place on the 400+ source packages I maintain.
I know it's not a solution, but I wonder if it would be easier to add
sed "/.. image/,+1d" README.rst
to your packaging instead of rebasing patches.
Last note: downloading the badges at build time is *not* an option (as all distros requires to build without internet connectivity).
Cheers,
Thomas Goirand (zigo)
Sphinx supports exiting with an error for any external references like images. Only a few projects are suppressing that warning [1], but perhaps not every project is treating sphinx warnings as errors that break the build.
[1] http://codesearch.openstack.org/?q=nonlocal_uri&i=nope&files=&re...
On Apr 8, 2020, at 12:54 PM, Dmitry Tantsur dtantsur@redhat.com wrote:
On Wed, Apr 8, 2020 at 6:50 PM Thomas Goirand <zigo@debian.org mailto:zigo@debian.org> wrote: Hi Guys!
Thanks for participating in this thread. I feel less alone now! :)
On 4/6/20 11:18 PM, Jeremy Stanley wrote:
Maybe we could make it easier to have our documentation builds strip that content automatically so that distro package maintainers don't need to?
That'd be really awesome!
On 4/7/20 3:52 PM, Jeremy Stanley wrote:
My point was that these badges are mostly only relevant to folks browsing README files in a source code hosting platform, but when the README is embedded into a compiled Sphinx document they seem really out of place and are probably best omitted.
Yes. Not only that. In Debian we consider this a "privacy breach" (see below).
On 4/7/20 5:00 PM, Sean Mooney wrote:
for what its worth im not sure why this thread was suddenly revied after 6 months
Because, as for each new release of OpenStack over the last 9 years, I'm being moronic about re-occurring issues in the Debian packages I maintain, and I didn't want to skip this tradition! :)
On 4/7/20 5:58 PM, Jeremy Stanley wrote:
Some projects put a thorough introduction in their README.rst and don't want to have to duplicate it all in the Sphinx doc source tree so just stick a ".. include:: ../../README.rst" line into doc/source/index.rst or a similar document. It's typical for Debian packages (though I haven't looked at Thomas's) to put the raw readme somewhere like /usr/share/doc/foo/README.rst but also stick HTML builds of the full documentation in a place like /usr/share/doc/foo/html/
I typically only ship the HTML documentation, as I feel like it looks much nicer than just the rst format. If one wants the raw format, well we do have source packages... :)
The entire point of shipping these HTML builds in the compiled packages is so that users have a local copy of documentation even if they're somewhere with no working Internet access, and can even build them from local copies of the source packages without working Internet, therefore they need to not depend on external assets out on the Internet and have to instead be buildable with only what's packaged in the distro.
Yes. But in this case, the problem is that the resulting HTML documentation ends up with an image looking like this:
<img src="http://<something-not-on-my-laptop>/ ...
which means:
1/ the image wont display without internet 2/ it will make a query to a website, revealing to a 3rd party that I'm browsing the documentation. That's a privacy breach...
Of course, the OpenStack Foundation has good intentions, and I don't think it will make any use of these stats. Still, this is annoying.
I therefore keep on patching these away from the README.rst, which is exhausting, because I constantly need to rebase the patches, and it's really all over the place on the 400+ source packages I maintain.
I know it's not a solution, but I wonder if it would be easier to add
sed "/.. image/,+1d" README.rst
to your packaging instead of rebasing patches.
Last note: downloading the badges at build time is *not* an option (as all distros requires to build without internet connectivity).
Cheers,
Thomas Goirand (zigo)
Thomas Goirand wrote:
Hi Guys!
Thanks for participating in this thread. I feel less alone now! :)
On 4/6/20 11:18 PM, Jeremy Stanley wrote:
Maybe we could make it easier to have our documentation builds strip that content automatically so that distro package maintainers don't need to?
That'd be really awesome!
On 4/7/20 3:52 PM, Jeremy Stanley wrote:
My point was that these badges are mostly only relevant to folks browsing README files in a source code hosting platform, but when the README is embedded into a compiled Sphinx document they seem really out of place and are probably best omitted.
Yes. Not only that. In Debian we consider this a "privacy breach" (see below).
If it was just us, I'd say that at this point it's just easier to get rid of the tag-badge: it brings limited value and also does not "behave" like individual badges when clicked on.
However I am not sure that would not really solve your problem, as teams routinely add badges to their README files like the PyPI version badge in:
https://opendev.org/openstack/python-novaclient/src/branch/master/README.rst
So I think the simplest is to automate the removal of external images so that the result is compliant with Debian packaging. I bet it's a pretty common packaging problem with all those projects on GitHub using badges those days?
participants (7)
-
Dmitry Tantsur
-
Doug Hellmann
-
Graham Hayes
-
Jeremy Stanley
-
Sean Mooney
-
Thierry Carrez
-
Thomas Goirand