<div class="gmail_extra">Hi Daisy, <br><br>Thanks so much for this detailed proposal. I'd like you to put it on the OpenStack wiki, at <a href="http://wiki.openstack.org/Translations">http://wiki.openstack.org/Translations</a>. <br>
<br>My first read-through and discussion with the CI team brings up a few comments:<br>- Whatever we do for docs, we should also do for code strings. So unfortunately the scope for the "Goal" probably cannot be so narrow. We know Launchpad to be broken with code strings now, that data point should be reflected in this point-in-time analysis.<br>
<br>- Dashboard uses Transifex now (while the other projects unsuccessfully use Launchpad). Tres Henry, can you comment on the number of translators of Dashboard strings you have on the Transifex side already? <br><br>- Not that I want analysis paralysis, but, we may need to add a third column of a crowd-sourced translation option like Pootle that is familiar to open-source translators. Also, the lack of a translation memory/dictionary (and having to hold such a valuable asset in a wiki page) is troubling, can we also analyze an option that offers a translation dictionary? So much re-use would be available to all the projects.<br>
<br>- There seems to be assumptions that Jenkins and Gerrit will "just work" with out much description of the role those two crucial tools play. Can you further describe the workflow for those in the Slicing, Uploading, Downloading, Converging, and Generating steps? <br>
<br>I appreciate all the hard work I _know_ went into this proposal. Let's get it on the wiki, discuss more, and keep adding details. I'd like to make this a blueprint, could be for openstack-manuals, could be for horizon, I don't know yet. Thanks for stepping up and embracing our international community's needs!<br>
<br>Thanks,<br>Anne<br><br><br><br><br><br><div class="gmail_quote">On Fri, Apr 27, 2012 at 3:45 AM, Ying Chun Guo <span dir="ltr"><<a href="mailto:guoyingc@cn.ibm.com" target="_blank">guoyingc@cn.ibm.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>
<p><font face="sans-serif">Hi, all</font><br>
<br>
<font face="serif">During the </font><font face="serif">"I18N in OpenStack" discussion in design summit, it is mentioned that documents need to I18N. I also noticed some requests for a Chinese version manuals from China users. But unlike Gettext strings in the codes, there is no process for DocBook translation yet. Translators, who want to help translation, </font><font face="serif">have to take a DocBook into a tool and perform a translation on a copy which will be saved as a new file.</font><font face="serif"> This traditional translation model is not good </font><font face="serif">for collaboration. </font><font face="serif">Usually, the open source translation depends on volunteers. It's better to use the crowd translation model, which </font><font face="serif">enables a mass of translators to work on the same job, just like the Launchpad Web UI for Gettext strings translation, any people can jump in at any time and contribute to any part of the translatable contents.</font><br>
<br>
<font face="serif">In order to facilitate the manuals translation, I investigated several translation websites and several open source projects. I composed this proposal. Now it's open for suggestions and comments.</font><br>
<br>
<font face="serif" size="4"><b>Goal</b></font><br>
<font face="serif" size="4"><b>------------</b></font><br>
<font face="serif">A process for manuals translation</font><br>
<br>
<font face="serif" size="4"><b>Background</b></font><br>
<font face="serif" size="4"><b>--------------</b></font><br>
<font face="serif">OpenStack Manuals are in DocBook format. The source is on GitHub: </font><font face="serif"><a href="http://github.com/openstack/openstack-manuals" target="_blank">http://github.com/openstack/openstack-manuals</a></font><br>
<font face="serif">Launchpad and Transifex are free web based tools used for crowd translation. </font><font face="serif">Both of them provide a simple web interface in which non-technical people can help translation. They don't support DocBook format, but support the popular GNU Gettext file formats (PO Template or PO).</font><br>
<br>
<font face="serif" size="4"><b>Translation Process</b></font><br>
<font face="serif" size="4"><b>-------------------</b></font><br>
<font face="serif">In order to translate OpenStack Manuals to multiple languages, which are in DocBook format, we can slice the documents into short statements, then use a web based translation management tool to manage the translation process, and finally converge the translated content into a new copy of DocBook. </font><br>
<br>
<font face="serif">Here are the five steps of the translation process:</font><br>
<font face="serif">Step #1 Slicing - </font><font face="serif">extract translatable content from DocBooks and generate Gettext compatible POT files (PO Template or PO);</font><br>
<font face="serif">Step #2 Uploading - upload the POT (or PO) files to a web based translation management tool;</font><br>
<font face="serif">Step #3 Downloading - download PO (or MO) files from the web tool after translation and review;</font><br>
<font face="serif">Step #4 Converging - converge the translated contents into new copies of DocBook, create DocBooks in multiple languages</font><br>
<font face="serif">Step #5 Generating - generate HTML/PDF in multiple languages from DocBooks in multiple languages</font><br>
<br>
<font face="serif">The picture in the attachment describes these steps. </font><br>
<i>(See attached file: DocBook translation process.png)</i><br>
<br>
<font face="serif" size="4"><b>Compare of Launchpad and Transifex</b></font><br>
<font face="serif" size="4"><b>-------------------</b></font><br>
<font face="serif">Launchpad (</font><font face="serif"><a href="https://launchpad.net/" target="_blank">https://launchpad.net/</a></font><font face="serif">) and Transifex (</font><font face="serif"><a href="https://www.transifex.net/" target="_blank">https://www.transifex.net/</a></font><font face="serif">) are similar web based tools used for crowd translation. The goal of the compare is to find the most </font><font face="serif">appropriate tool for this scenario. The compare are made </font><font face="serif">between Launchpad and Transifex free version for open sources. (</font><font face="serif">Refer to </font><font face="serif"><a href="https://www.transifex.net/plans/" target="_blank">https://www.transifex.net/plans/</a></font><font face="serif"> to get details of “Transifex free version for open sources”</font><font face="serif">)</font><br>
<br>
<font face="serif">After considering the requirements for manuals translation, below perspectives are taking into consideration:</font><br>
<font face="serif">*Supported format</font><br>
<font face="serif">*DocBook slicing support</font><br>
<font face="serif">*Converging support</font><br>
<font face="serif">*Source uploading method</font><br>
<font face="serif">*Output downloading method</font><br>
<font face="serif">*Translation Memory support</font><br>
<font face="serif">*Translation history support</font><br>
<font face="serif">*Change management</font><br>
<font face="serif">*Translation Dictionary</font><br>
<font face="serif">Refer to Table 1 for detail information of the compare.</font><br>
<br>
<font face="serif">Another important measurement to compare is the workload. Having the five steps in the process execute automatically as much as possible will decrease the workload of translation coordinators. </font><br>
<font face="serif">Refer to Table 2 for the detail of workload compare when using Launchpad or Transifex for DocBook translation.</font><br>
<br>
<font face="serif">Here are the conclusions after the compare, </font><br>
<font face="serif">(1) the workload using Transifex is similar with using Launchpad. </font><br>
<font face="serif">(2) The advantages of Launchpad are:</font><br>
<font face="serif">* Leverage the same user id and user group of developers, users, translators of Gettext strings.</font><br>
<font face="serif">* Leverage the same </font><font face="serif">contribution calculating method </font><font face="serif">"Karma", with fixing bugs, answering questions and Gettext strings translation.</font><br>
<font face="serif">(3) The advantage of Transifex is better translation memory support. </font><br>
<font face="serif">The disadvantage of Transifex is having different user registration and user interfaces. Both the translators and the coordinators need to register in a new website and get familiar with a new user interfaces before translation.</font><br>
<br>
<font face="serif">Based on these analysis, I think, using Launchpad to do the manuals translation is a good choice.</font><br>
<br>
<font face="serif" size="4"><b>Other considerations</b></font><br>
<font face="serif" size="4"><b>-------------------</b></font><br>
<font face="serif">*Translation Dictionary</font><br>
<font face="serif">Translation Dictionary here means </font><font face="serif">terminology translation. It</font><font face="serif"> is very helpful to ensure the translation quality. Unfortunately, both Launchpad and Transifex don't support Translation Dictionary. I suggest to use wiki pages to document the terminology translation for translators reference. </font><br>
<font face="serif">Here is a sample wiki page for Eclipse globalization: </font><font face="serif"><a href="http://wiki.eclipse.org/French_Glossary" target="_blank">http://wiki.eclipse.org/French_Glossary</a></font><font face="serif">. </font><br>
<br>
<font face="serif">*Change Management</font><br>
<font face="serif">Launchpad and Transifex support the synchronize of old PO files and new PO files in their own ways</font><font face="serif">. </font><font face="serif">They will compare the new po and the existing po and handle the changes automatically. </font><font face="serif">But new PO files won't be generated automatically after DocBooks are changed. Translation coordinators need to generate new PO files by running a Python program manually. </font><br>
<font face="serif">I will suggest to develop a program in future, to monitor the update of manuals GitHub repository. When a DocBook is updated, a new PO file will be generated and synchronized with the old one in the Launchpad server.</font><br>
<br>
<font face="serif">*Machine translation</font><br>
<font face="serif">Is it necessary to include machine translation? Machine translation can be executed before human beings review. Then translators won't need to translate from scratch. Translators can review the result of machine translation and correct them.</font><br>
<font face="serif">But after investigation, I found the quality of free machine translations, which have API exported, are not so good. I doubt whether a poor quality machine translation is helpful.</font><br>
<font face="serif">Anyway, if most of the community members want to include machine translation, it is possible to improve the slicing program, to generate a PO file with the results of machine translation.</font><br>
<br>
<font face="serif" size="4"><b>Reference </b></font><br>
<font face="serif" size="4"><b>-------------------</b></font><br>
<font face="serif">Table 1 - Compare of Launchpad and Transifex</font>
</p><table border="1">
<tbody><tr valign="top"><td width="155"><img src="cid:2__=C7BBF37FDFBA49698f9e8a93df938@cn.ibm.com" alt="" border="0" height="1" width="1"></td><td width="159"><div align="center"><font face="serif">Launchpad</font></div>
</td><td width="325"><div align="center"><font face="serif">Transifex</font></div></td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Supported format</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">pot file (.pot), </font><br>
<font face="serif">po file (.po)</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">android string resources (.xml), </font><br>
<font face="serif">po file (.po), </font><br>
<font face="serif">html (.html), </font><br>
<font face="serif">WIKI file (.wiki), etc.</font><br>
<font face="serif">(</font><font face="serif">Note</font><font face="serif">, DocBook is not a supported file format; OpenStack Wiki format is not a supported wiki format.)</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">DocBook Slicing support</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">No</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">No </font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Converging support</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">No</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">No</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Source uploading method</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">Two methods:</font><br>
<font face="serif">a> Automatic template imports from Bazaar branch</font><br>
<font face="serif">b> Manually upload template (or an archive) through Launchpad's web interface.</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">Two methods:</font><br>
<font face="serif">a> Use a command tool “Transifex Client” to synchronize the server with local repository (local folder) by typing several commands.</font><br>
<font face="serif">b> Manually upload a source translation file from web interface;</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Output downloading method</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">Two methods:</font><br>
<font face="serif">a> Automatic save output files to Bazaar branch;</font><br>
<font face="serif">b> Manually download output files through web interface.</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">Two methods:</font><br>
<font face="serif">a> Use “Transifex Client” to download the latest translations from the server by typing one command.</font><br>
<font face="serif">b> Manually download through web interface.</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Translation Memory support</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">The exact same translation items in other projects can be listed as a reference.</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">The similar translation items will be listed as a reference. Translation memory can be shared within two and more projects. </font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Translation history support</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">Yes</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">Yes</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Change management</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">Launchpad will automatically update its data every time you push a new revision to the Bazaar branch.</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">When you push some local updates to server, Transifex will overwrite the existing source strings and translations with the updated version. </font><br>
<font face="serif">(</font><font face="serif">Note:</font><font face="serif"> This may lead to loss of translations. So users need to make sure the local repository contains the latest translation results in the server.)</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left:2pt"><font face="serif">Translation Dictionary</font></ul>
</td><td width="159">
<ul style="padding-left:2pt"><font face="serif">No</font></ul>
</td><td width="325">
<ul style="padding-left:2pt"><font face="serif">No</font></ul>
</td></tr>
</tbody></table>
<ul style="padding-left:2pt"></ul>
<br>
<font face="serif">Table 2 - Workload compare when using Launchpad or </font><font face="serif">Transifex</font><font face="serif"> for DocBook translation</font>
<table border="1">
<tbody><tr valign="top"><td width="211"><img src="cid:2__=C7BBF37FDFBA49698f9e8a93df938@cn.ibm.com" alt="" border="0" height="1" width="1"></td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Using Launchpad</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Using Transifex</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left:2pt"><font face="serif">Step 1: Slicing</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Python program [1] can be used to slice all the DocBook together in one command</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Same with Launchpad</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left:2pt"><font face="serif">Step 2: Uploading</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">If the source code is synchronized with Bazaar, the uploading can be automatically handled by Launchpad.</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Use “Transifex Client” to upload resources to Transifex server from local repository (local folder) by typing several commands.</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left:2pt"><font face="serif">Step 3: Downloading</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Launchpad can commit daily snapshots of the translations to a Bazaar branch in a specific folder.</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Use “Transifex Client” to download the latest translations from the server by typing one command.</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left:2pt"><font face="serif">Step 4: Converging</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Python program [2] can be used to coverge all the po files back to DocBooks</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Same with Launchpad</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left:2pt"><font face="serif">Step 5: Generating</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Maven command can be used to generate HTML/PDF from DocBooks</font></ul>
</td><td width="214">
<ul style="padding-left:2pt"><font face="serif">Same with Launchpad</font></ul>
</td></tr>
</tbody></table>
<ul style="padding-left:2pt"></ul>
<font face="serif">[1] </font><font face="serif">The Python program can be written based on “xml2po” to slice all DocBooks of the manuals project to translatable strings in batch. “xml2po” is an existing Python program in GNOME gnome-doc-utils package which can extracts translatable content from free-form XML documents and outputs gettext compatible POT files.</font><br>
<font face="serif">[2] The Python program </font><font face="serif">can be written based on “xml2po”, to converge the translated strings back to copies of DocBooks in batch.</font><br>
<br>
<br>
<font face="sans-serif">Regards<br>
Daisy Guo<br>
</font><p></p></div><br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div>