[openstack-dev] [docs] sitemap automation suggestions

Jimmy McArthur jimmy at openstack.org
Sat Oct 7 18:59:02 UTC 2017


Megan put that one in at my urging.  It's basically an extension of 
Google Analytics. It gives reports on 404s and tips on how to improve 
your sitemap. There is nothing to install on the server, we just have to 
put a little html or code snippet include into the site so that Google 
can verify the site.

All that said, you're right it's been renamed to Google Search Console 
and, for the moment, we don't seem to have access to it, even as an o.o 
Google Analytics admin. As usual, Google does the worst job possible 
about telling you how to switch to their new tool.  Le sigh.


> Jeremy Stanley <mailto:fungi at yuggoth.org>
> October 6, 2017 at 9:58 AM
> On 2017-10-05 22:51:31 -0400 (-0400), megan at openstack.org wrote:
> [...]
> [...]
>
> I'm having a hard time figuring out from that page what exactly
> "Google Webmaster tools" is, and under what license it's
> distributed. Looks like maybe it's been renamed "Google Search
> Console" according to Wikipedia, but it's still unclear to me where
> to download the source and how to install it on the server.
>
> The search console link just goes to a login page and wants me to
> have an account with Google before it will (presumably) give me any
> more detail, so that was the point where I stopped.
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> megan at openstack.org <mailto:megan at openstack.org>
> October 5, 2017 at 9:51 PM
> Hello all!
>
> As you may be aware, sitemaps generation for docs.openstack.org is 
> currently done via a manually triggered scrapy process. It currently 
> also scrapes the entirety of docs.openstack.org, making processing 
> slow. In order to improve the efficiency of this process, I would like 
> to propose the following updates to the sitemap generation toolkit:
> * keep track (in logs) of 301s, 302s, and 404s,
> * automatic pull of supported releases,
> * cron-managed automatic updates, and
> * setup of Google Webmaster tools (https://www.google.com/webmasters/)
> * a few style cleanups
>
> Beyond this, implementing more targeted crawling would improve the 
> processing speed and scope massively. This is, however, a bit of a 
> complicated matter, as it requires us to decide what, exactly, defines 
> scope relevence, in order to limit the crawl domain.
>
> These are, of course, only our precursory findings. and we would love 
> to hear some feedback about alternate methods and possible tricky 
> aspects of the suggested changes. What do you think? Let us know!
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171007/2f7af0fd/attachment.html>


More information about the OpenStack-dev mailing list