<html><body>
<p><font size="2" face="sans-serif">Hi, all</font><br>
<br>
<font size="2" face="serif">During the </font><font size="2" face="serif">"I18N in OpenStack" discussion in design summit, it is mentioned that documents need to I18N. I also noticed some requests for a Chinese version manuals from China users. But unlike Gettext strings in the codes, there is no process for DocBook translation yet. Translators, who want to help translation, </font><font size="2" face="serif">have to take a DocBook into a tool and perform a translation on a copy which will be saved as a new file.</font><font size="2" face="serif"> This traditional translation model is not good </font><font size="2" face="serif">for collaboration. </font><font size="2" face="serif">Usually, the open source translation depends on volunteers. It's better to use the crowd translation model, which </font><font size="2" face="serif">enables a mass of translators to work on the same job, just like the Launchpad Web UI for Gettext strings translation, any people can jump in at any time and contribute to any part of the translatable contents.</font><br>
<br>
<font size="2" face="serif">In order to facilitate the manuals translation, I investigated several translation websites and several open source projects. I composed this proposal. Now it's open for suggestions and comments.</font><br>
<br>
<font size="4" face="serif"><b>Goal</b></font><br>
<font size="4" face="serif"><b>------------</b></font><br>
<font size="2" face="serif">A process for manuals translation</font><br>
<br>
<font size="4" face="serif"><b>Background</b></font><br>
<font size="4" face="serif"><b>--------------</b></font><br>
<font size="2" face="serif">OpenStack Manuals are in DocBook format. The source is on GitHub: </font><font size="2" face="serif"><a href="http://github.com/openstack/openstack-manuals">http://github.com/openstack/openstack-manuals</a></font><br>
<font size="2" face="serif">Launchpad and Transifex are free web based tools used for crowd translation. </font><font size="2" face="serif">Both of them provide a simple web interface in which non-technical people can help translation. They don't support DocBook format, but support the popular GNU Gettext file formats (PO Template or PO).</font><br>
<br>
<font size="4" face="serif"><b>Translation Process</b></font><br>
<font size="4" face="serif"><b>-------------------</b></font><br>
<font size="2" face="serif">In order to translate OpenStack Manuals to multiple languages, which are in DocBook format, we can slice the documents into short statements, then use a web based translation management tool to manage the translation process, and finally converge the translated content into a new copy of DocBook. </font><br>
<br>
<font size="2" face="serif">Here are the five steps of the translation process:</font><br>
<font size="2" face="serif">Step #1 Slicing - </font><font size="2" face="serif">extract translatable content from DocBooks and generate Gettext compatible POT files (PO Template or PO);</font><br>
<font size="2" face="serif">Step #2 Uploading - upload the POT (or PO) files to a web based translation management tool;</font><br>
<font size="2" face="serif">Step #3 Downloading - download PO (or MO) files from the web tool after translation and review;</font><br>
<font size="2" face="serif">Step #4 Converging - converge the translated contents into new copies of DocBook, create DocBooks in multiple languages</font><br>
<font size="2" face="serif">Step #5 Generating - generate HTML/PDF in multiple languages from DocBooks in multiple languages</font><br>
<br>
<font size="2" face="serif">The picture in the attachment describes these steps. </font><br>
<i>(See attached file: DocBook translation process.png)</i><br>
<br>
<font size="4" face="serif"><b>Compare of Launchpad and Transifex</b></font><br>
<font size="4" face="serif"><b>-------------------</b></font><br>
<font size="2" face="serif">Launchpad (</font><font size="2" face="serif"><a href="https://launchpad.net/">https://launchpad.net/</a></font><font size="2" face="serif">) and Transifex (</font><font size="2" face="serif"><a href="https://www.transifex.net/">https://www.transifex.net/</a></font><font size="2" face="serif">) are similar web based tools used for crowd translation. The goal of the compare is to find the most </font><font size="2" face="serif">appropriate tool for this scenario. The compare are made </font><font size="2" face="serif">between Launchpad and Transifex free version for open sources. (</font><font size="2" face="serif">Refer to </font><font size="2" face="serif"><a href="https://www.transifex.net/plans/">https://www.transifex.net/plans/</a></font><font size="2" face="serif"> to get details of Ħ°Transifex free version for open sourcesĦħ</font><font size="2" face="serif">)</font><br>
<br>
<font size="2" face="serif">After considering the requirements for manuals translation, below perspectives are taking into consideration:</font><br>
<font size="2" face="serif">*Supported format</font><br>
<font size="2" face="serif">*DocBook slicing support</font><br>
<font size="2" face="serif">*Converging support</font><br>
<font size="2" face="serif">*Source uploading method</font><br>
<font size="2" face="serif">*Output downloading method</font><br>
<font size="2" face="serif">*Translation Memory support</font><br>
<font size="2" face="serif">*Translation history support</font><br>
<font size="2" face="serif">*Change management</font><br>
<font size="2" face="serif">*Translation Dictionary</font><br>
<font size="2" face="serif">Refer to Table 1 for detail information of the compare.</font><br>
<br>
<font size="2" face="serif">Another important measurement to compare is the workload. Having the five steps in the process execute automatically as much as possible will decrease the workload of translation coordinators. </font><br>
<font size="2" face="serif">Refer to Table 2 for the detail of workload compare when using Launchpad or Transifex for DocBook translation.</font><br>
<br>
<font size="2" face="serif">Here are the conclusions after the compare, </font><br>
<font size="2" face="serif">(1) the workload using Transifex is similar with using Launchpad. </font><br>
<font size="2" face="serif">(2) The advantages of Launchpad are:</font><br>
<font size="2" face="serif">* Leverage the same user id and user group of developers, users, translators of Gettext strings.</font><br>
<font size="2" face="serif">* Leverage the same </font><font size="2" face="serif">contribution calculating method </font><font size="2" face="serif">"Karma", with fixing bugs, answering questions and Gettext strings translation.</font><br>
<font size="2" face="serif">(3) The advantage of Transifex is better translation memory support. </font><br>
<font size="2" face="serif">The disadvantage of Transifex is having different user registration and user interfaces. Both the translators and the coordinators need to register in a new website and get familiar with a new user interfaces before translation.</font><br>
<br>
<font size="2" face="serif">Based on these analysis, I think, using Launchpad to do the manuals translation is a good choice.</font><br>
<br>
<font size="4" face="serif"><b>Other considerations</b></font><br>
<font size="4" face="serif"><b>-------------------</b></font><br>
<font size="2" face="serif">*Translation Dictionary</font><br>
<font size="2" face="serif">Translation Dictionary here means </font><font size="2" face="serif">terminology translation. It</font><font size="2" face="serif"> is very helpful to ensure the translation quality. Unfortunately, both Launchpad and Transifex don't support Translation Dictionary. I suggest to use wiki pages to document the terminology translation for translators reference. </font><br>
<font size="2" face="serif">Here is a sample wiki page for Eclipse globalization: </font><font size="2" face="serif"><a href="http://wiki.eclipse.org/French_Glossary">http://wiki.eclipse.org/French_Glossary</a></font><font size="2" face="serif">. </font><br>
<br>
<font size="2" face="serif">*Change Management</font><br>
<font size="2" face="serif">Launchpad and Transifex support the synchronize of old PO files and new PO files in their own ways</font><font size="2" face="serif">. </font><font size="2" face="serif">They will compare the new po and the existing po and handle the changes automatically. </font><font size="2" face="serif">But new PO files won't be generated automatically after DocBooks are changed. Translation coordinators need to generate new PO files by running a Python program manually. </font><br>
<font size="2" face="serif">I will suggest to develop a program in future, to monitor the update of manuals GitHub repository. When a DocBook is updated, a new PO file will be generated and synchronized with the old one in the Launchpad server.</font><br>
<br>
<font size="2" face="serif">*Machine translation</font><br>
<font size="2" face="serif">Is it necessary to include machine translation? Machine translation can be executed before human beings review. Then translators won't need to translate from scratch. Translators can review the result of machine translation and correct them.</font><br>
<font size="2" face="serif">But after investigation, I found the quality of free machine translations, which have API exported, are not so good. I doubt whether a poor quality machine translation is helpful.</font><br>
<font size="2" face="serif">Anyway, if most of the community members want to include machine translation, it is possible to improve the slicing program, to generate a PO file with the results of machine translation.</font><br>
<br>
<font size="4" face="serif"><b>Reference </b></font><br>
<font size="4" face="serif"><b>-------------------</b></font><br>
<font size="2" face="serif">Table 1 - Compare of Launchpad and Transifex</font>
<table border="1">
<tr valign="top"><td width="155"><img width="1" height="1" src="cid:2__=C7BBF37FDFBA49698f9e8a93df938@cn.ibm.com" border="0" alt=""></td><td width="159"><div align="center"><font size="2" face="serif">Launchpad</font></div></td><td width="325"><div align="center"><font size="2" face="serif">Transifex</font></div></td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Supported format</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">pot file (.pot), </font><br>
<font size="2" face="serif">po file (.po)</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">android string resources (.xml), </font><br>
<font size="2" face="serif">po file (.po), </font><br>
<font size="2" face="serif">html (.html), </font><br>
<font size="2" face="serif">WIKI file (.wiki), etc.</font><br>
<font size="2" face="serif">(</font><font size="2" face="serif">Note</font><font size="2" face="serif">, DocBook is not a supported file format; OpenStack Wiki format is not a supported wiki format.)</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">DocBook Slicing support</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">No</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">No </font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Converging support</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">No</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">No</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Source uploading method</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">Two methods:</font><br>
<font size="2" face="serif">a> Automatic template imports from Bazaar branch</font><br>
<font size="2" face="serif">b> Manually upload template (or an archive) through Launchpad's web interface.</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">Two methods:</font><br>
<font size="2" face="serif">a> Use a command tool Ħ°Transifex ClientĦħ to synchronize the server with local repository (local folder) by typing several commands.</font><br>
<font size="2" face="serif">b> Manually upload a source translation file from web interface;</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Output downloading method</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">Two methods:</font><br>
<font size="2" face="serif">a> Automatic save output files to Bazaar branch;</font><br>
<font size="2" face="serif">b> Manually download output files through web interface.</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">Two methods:</font><br>
<font size="2" face="serif">a> Use Ħ°Transifex ClientĦħ to download the latest translations from the server by typing one command.</font><br>
<font size="2" face="serif">b> Manually download through web interface.</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Translation Memory support</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">The exact same translation items in other projects can be listed as a reference.</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">The similar translation items will be listed as a reference. Translation memory can be shared within two and more projects. </font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Translation history support</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">Yes</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">Yes</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Change management</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">Launchpad will automatically update its data every time you push a new revision to the Bazaar branch.</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">When you push some local updates to server, Transifex will overwrite the existing source strings and translations with the updated version. </font><br>
<font size="2" face="serif">(</font><font size="2" face="serif">Note:</font><font size="2" face="serif"> This may lead to loss of translations. So users need to make sure the local repository contains the latest translation results in the server.)</font></ul>
</td></tr>
<tr valign="top"><td width="155">
<ul style="padding-left: 2pt"><font size="2" face="serif">Translation Dictionary</font></ul>
</td><td width="159">
<ul style="padding-left: 2pt"><font size="2" face="serif">No</font></ul>
</td><td width="325">
<ul style="padding-left: 2pt"><font size="2" face="serif">No</font></ul>
</td></tr>
</table>
<ul style="padding-left: 2pt"></ul>
<br>
<font size="2" face="serif">Table 2 - Workload compare when using Launchpad or </font><font size="2" face="serif">Transifex</font><font size="2" face="serif"> for DocBook translation</font>
<table border="1">
<tr valign="top"><td width="211"><img width="1" height="1" src="cid:2__=C7BBF37FDFBA49698f9e8a93df938@cn.ibm.com" border="0" alt=""></td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Using Launchpad</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Using Transifex</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left: 2pt"><font size="2" face="serif">Step 1: Slicing</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Python program [1] can be used to slice all the DocBook together in one command</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Same with Launchpad</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left: 2pt"><font size="2" face="serif">Step 2: Uploading</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">If the source code is synchronized with Bazaar, the uploading can be automatically handled by Launchpad.</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Use Ħ°Transifex ClientĦħ to upload resources to Transifex server from local repository (local folder) by typing several commands.</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left: 2pt"><font size="2" face="serif">Step 3: Downloading</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Launchpad can commit daily snapshots of the translations to a Bazaar branch in a specific folder.</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Use Ħ°Transifex ClientĦħ to download the latest translations from the server by typing one command.</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left: 2pt"><font size="2" face="serif">Step 4: Converging</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Python program [2] can be used to coverge all the po files back to DocBooks</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Same with Launchpad</font></ul>
</td></tr>
<tr valign="top"><td width="211">
<ul style="padding-left: 2pt"><font size="2" face="serif">Step 5: Generating</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Maven command can be used to generate HTML/PDF from DocBooks</font></ul>
</td><td width="214">
<ul style="padding-left: 2pt"><font size="2" face="serif">Same with Launchpad</font></ul>
</td></tr>
</table>
<ul style="padding-left: 2pt"></ul>
<font size="2" face="serif">[1] </font><font size="2" face="serif">The Python program can be written based on Ħ°xml2poĦħ to slice all DocBooks of the manuals project to translatable strings in batch. Ħ°xml2poĦħ is an existing Python program in GNOME gnome-doc-utils package which can extracts translatable content from free-form XML documents and outputs gettext compatible POT files.</font><br>
<font size="2" face="serif">[2] The Python program </font><font size="2" face="serif">can be written based on Ħ°xml2poĦħ, to converge the translated strings back to copies of DocBooks in batch.</font><br>
<br>
<br>
<font size="2" face="sans-serif">Regards<br>
Daisy Guo<br>
</font></body></html>