Horizon translations - and update of strings
Looking at my patch to import back the Horizon translations (https://review.openstack.org/#/c/82176) and the comments by Akihiro Motoki, I notice that horizon has no command to update the message catalogs. For all other projects, we basically do the following as part of the propose job: * Download all translation PO files from transifex * Update POT file * Update PO files based on pot file ("python setup.py update_catalogs") * Upload POT and PO files to transifex Horizon does not implement update_catalogs at all. Is this step really needed or is something that transifex does for us and thus we could remove the step? Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
On 03/23/2014 09:20 PM, Andreas Jaeger wrote:
Looking at my patch to import back the Horizon translations (https://review.openstack.org/#/c/82176) and the comments by Akihiro Motoki, I notice that horizon has no command to update the message catalogs.
For all other projects, we basically do the following as part of the propose job: * Download all translation PO files from transifex * Update POT file * Update PO files based on pot file ("python setup.py update_catalogs") * Upload POT and PO files to transifex
Horizon does not implement update_catalogs at all. Is this step really needed or is something that transifex does for us and thus we could remove the step?
Also, do we really need to push the POT file to transifex? It should be done as part of the uptream_translation post job, isn't it? Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
23 mar 2014 21:25 "Andreas Jaeger" <aj@suse.com> napisał(a):
On 03/23/2014 09:20 PM, Andreas Jaeger wrote:
Looking at my patch to import back the Horizon translations (https://review.openstack.org/#/c/82176) and the comments by Akihiro Motoki, I notice that horizon has no command to update the message
catalogs.
For all other projects, we basically do the following as part of the propose job: * Download all translation PO files from transifex * Update POT file * Update PO files based on pot file ("python setup.py update_catalogs") * Upload POT and PO files to transifex
Horizon does not implement update_catalogs at all. Is this step really needed or is something that transifex does for us and thus we could remove the step?
Also, do we really need to push the POT file to transifex? It should be done as part of the uptream_translation post job, isn't it?
Yes, the POT file should only be updated/uploaded in the post gate if it changes. -- DeeJay1
On Mon, Mar 24, 2014 at 5:46 AM, Łukasz Jernaś <deejay1@srem.org> wrote:
23 mar 2014 21:25 "Andreas Jaeger" <aj@suse.com> napisał(a):
On 03/23/2014 09:20 PM, Andreas Jaeger wrote:
Looking at my patch to import back the Horizon translations (https://review.openstack.org/#/c/82176) and the comments by Akihiro Motoki, I notice that horizon has no command to update the message catalogs.
For all other projects, we basically do the following as part of the propose job: * Download all translation PO files from transifex * Update POT file * Update PO files based on pot file ("python setup.py update_catalogs") * Upload POT and PO files to transifex
Horizon does not implement update_catalogs at all. Is this step really needed or is something that transifex does for us and thus we could remove the step?
Yes. As long as there is no update in POT file, there is no need to update PO files based on PO file. When PO file is uploaded to Transifex, Transifex merges translation data with upload PO file. In my understanding, new POT files are uploaded by upstream_translation post job and they are kept up-to-date, so there is no need to update PO files in propose_translation job. Precisely speaking, ./run_tests.sh --makemessages for specific language does the same job for update_catalogs for other projects. Horizon uses Django framework and does not use babel to manage PO/POT files. This is the reason Horizon's POT files don't have .pot suffix. I think update_catalogs in other projects propose_translation job tries to keep POT and PO consistent, but it brings no additional values because if there is new strings in POT there is no translation for new strings even if update_catalog is done and gettext mechanism works well even if there is a inconsistent points between PO and POT.
Also, do we really need to push the POT file to transifex? It should be done as part of the uptream_translation post job, isn't it?
Yes, the POT file should only be updated/uploaded in the post gate if it changes.
Yes, as Lukasz commented. Thanks, Akihiro
On 03/24/2014 12:53 AM, Akihiro Motoki wrote:
On Mon, Mar 24, 2014 at 5:46 AM, Łukasz Jernaś <deejay1@srem.org> wrote:
23 mar 2014 21:25 "Andreas Jaeger" <aj@suse.com> napisał(a):
On 03/23/2014 09:20 PM, Andreas Jaeger wrote:
Looking at my patch to import back the Horizon translations (https://review.openstack.org/#/c/82176) and the comments by Akihiro Motoki, I notice that horizon has no command to update the message catalogs.
For all other projects, we basically do the following as part of the propose job: * Download all translation PO files from transifex * Update POT file * Update PO files based on pot file ("python setup.py update_catalogs") * Upload POT and PO files to transifex
Horizon does not implement update_catalogs at all. Is this step really needed or is something that transifex does for us and thus we could remove the step?
Yes. As long as there is no update in POT file, there is no need to update PO files based on PO file. When PO file is uploaded to Transifex, Transifex merges translation data with upload PO file. In my understanding, new POT files are uploaded by upstream_translation post job and they are kept up-to-date, so there is no need to update PO files in propose_translation job.
Precisely speaking, ./run_tests.sh --makemessages for specific language does the same job for update_catalogs for other projects. Horizon uses Django framework and does not use babel to manage PO/POT files. This is the reason Horizon's POT files don't have .pot suffix.
I think update_catalogs in other projects propose_translation job tries to keep POT and PO consistent, but it brings no additional values because if there is new strings in POT there is no translation for new strings even if update_catalog is done and gettext mechanism works well even if there is a inconsistent points between PO and POT.
I have one use case here: Some translators - like you and Sascha - do the translations without using the transifex web client locally. Now, you cannot use the local Japanese or German file since it contains outdated strings, you need to run msgmerge locally to update the file. Once the translator updates the file, msgmerge is called on the server and the next download will get a new version with the updated strings. Right now a "tx pull -a" will not download merged language pot files - see for example https://review.openstack.org/#/c/82434/ where doc/install-guide/locale/install-guide.pot is updated but not any of the po files. 1) Is the above correct? 2) Is that a use case we care about? Akihiro, Sascha, how is your workflow? Could you document it in the wiki, I don't see that covered...
Also, do we really need to push the POT file to transifex? It should be done as part of the uptream_translation post job, isn't it?
Yes, the POT file should only be updated/uploaded in the post gate if it changes.
Yes, as Lukasz commented.
I don't know why Clark did the upload when writing the initial script. Clark, do you remember? But we need to *update* it locally so that it can get committed to git (the update script does not handle updating of git) - just the upload looks unnecessary. Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
Hi Andreas, On Mon, Mar 24, 2014 at 4:00 PM, Andreas Jaeger <aj@suse.com> wrote:
On 03/24/2014 12:53 AM, Akihiro Motoki wrote:
On Mon, Mar 24, 2014 at 5:46 AM, Łukasz Jernaś <deejay1@srem.org> wrote:
23 mar 2014 21:25 "Andreas Jaeger" <aj@suse.com> napisał(a):
On 03/23/2014 09:20 PM, Andreas Jaeger wrote:
Looking at my patch to import back the Horizon translations (https://review.openstack.org/#/c/82176) and the comments by Akihiro Motoki, I notice that horizon has no command to update the message catalogs.
For all other projects, we basically do the following as part of the propose job: * Download all translation PO files from transifex * Update POT file * Update PO files based on pot file ("python setup.py update_catalogs") * Upload POT and PO files to transifex
Horizon does not implement update_catalogs at all. Is this step really needed or is something that transifex does for us and thus we could remove the step?
Yes. As long as there is no update in POT file, there is no need to update PO files based on PO file. When PO file is uploaded to Transifex, Transifex merges translation data with upload PO file. In my understanding, new POT files are uploaded by upstream_translation post job and they are kept up-to-date, so there is no need to update PO files in propose_translation job.
Precisely speaking, ./run_tests.sh --makemessages for specific language does the same job for update_catalogs for other projects. Horizon uses Django framework and does not use babel to manage PO/POT files. This is the reason Horizon's POT files don't have .pot suffix.
I think update_catalogs in other projects propose_translation job tries to keep POT and PO consistent, but it brings no additional values because if there is new strings in POT there is no translation for new strings even if update_catalog is done and gettext mechanism works well even if there is a inconsistent points between PO and POT.
I have one use case here: Some translators - like you and Sascha - do the translations without using the transifex web client locally.
I don't usually use local files for translations when Transifex is used, but it is a good assumption.
Now, you cannot use the local Japanese or German file since it contains outdated strings, you need to run msgmerge locally to update the file. Once the translator updates the file, msgmerge is called on the server and the next download will get a new version with the updated strings.
I am not sure about the relation between the first sentence and the second. Is POT file updated between the first and the second sentences? If there is no translation update on the server side, it works as expected. Note that Transifex does not merge translations on Transifex and uploaded translations.
Right now a "tx pull -a" will not download merged language pot files - see for example https://review.openstack.org/#/c/82434/ where doc/install-guide/locale/install-guide.pot is updated but not any of the po files.
Downloaded translations are already synced with POT file when POT file is uploaded to the server. In case of propose_translations_manuals, 1. POT is updated (generatepot) 2. POT is pushed to Transifex if there are any update. (At this point, translations on Transifex is "msgmege"d with the uploaded POT and translaitons becomes up-to-date.) 3. "tx pull -a" retrieves all up-to-date translations. 4 Finally "git review" if required No translations will be lost in this flow and it looks good. The flow in the original propose_translation_horizon in your review is different. tx pull -> generatepot -> msgmerge -> tx push translations This is the reason I wrote it may lose translations. On the other hand, there is no need in propose_translation_XXX.sh. If there are new or changed strings in POT file in step 2 above, we don't have translations for such strings in step 3. Even if POT and PO files are not synced, we will also get the same results (no translations for new or changed strings). This is the reason I wrote step 2 above is necessarily required in propose_translation script. Does it make sense?
1) Is the above correct? 2) Is that a use case we care about? Akihiro, Sascha, how is your workflow? Could you document it in the wiki, I don't see that covered...
My use case to use "tx pull" locally is just to check translations with some conversion (for manuals) or programs like horizon. I don't think I am doing special things. I don't usually use local files for translations if Transifex is used. I am not sure I answer what you ask.
Also, do we really need to push the POT file to transifex? It should be done as part of the uptream_translation post job, isn't it?
Yes, the POT file should only be updated/uploaded in the post gate if it changes.
Yes, as Lukasz commented.
I don't know why Clark did the upload when writing the initial script. Clark, do you remember?
But we need to *update* it locally so that it can get committed to git (the update script does not handle updating of git) - just the upload looks unnecessary.
As I wrote above, small out-of-sync between POT and PO files and synced PO file with untranslated strings have no difference as a result. I hope I answered your question. Thanks, Akihiro
Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
On 03/24/2014 09:47 AM, Akihiro Motoki wrote:
Hi Andreas,
On Mon, Mar 24, 2014 at 4:00 PM, Andreas Jaeger <aj@suse.com> wrote:
On 03/24/2014 12:53 AM, Akihiro Motoki wrote:
On Mon, Mar 24, 2014 at 5:46 AM, Łukasz Jernaś <deejay1@srem.org> wrote:
23 mar 2014 21:25 "Andreas Jaeger" <aj@suse.com> napisał(a):
On 03/23/2014 09:20 PM, Andreas Jaeger wrote:
Looking at my patch to import back the Horizon translations (https://review.openstack.org/#/c/82176) and the comments by Akihiro Motoki, I notice that horizon has no command to update the message catalogs.
For all other projects, we basically do the following as part of the propose job: * Download all translation PO files from transifex * Update POT file * Update PO files based on pot file ("python setup.py update_catalogs") * Upload POT and PO files to transifex
Horizon does not implement update_catalogs at all. Is this step really needed or is something that transifex does for us and thus we could remove the step?
Yes. As long as there is no update in POT file, there is no need to update PO files based on PO file. When PO file is uploaded to Transifex, Transifex merges translation data with upload PO file. In my understanding, new POT files are uploaded by upstream_translation post job and they are kept up-to-date, so there is no need to update PO files in propose_translation job.
Precisely speaking, ./run_tests.sh --makemessages for specific language does the same job for update_catalogs for other projects. Horizon uses Django framework and does not use babel to manage PO/POT files. This is the reason Horizon's POT files don't have .pot suffix.
I think update_catalogs in other projects propose_translation job tries to keep POT and PO consistent, but it brings no additional values because if there is new strings in POT there is no translation for new strings even if update_catalog is done and gettext mechanism works well even if there is a inconsistent points between PO and POT.
I have one use case here: Some translators - like you and Sascha - do the translations without using the transifex web client locally.
I don't usually use local files for translations when Transifex is used, but it is a good assumption.
Now, you cannot use the local Japanese or German file since it contains outdated strings, you need to run msgmerge locally to update the file. Once the translator updates the file, msgmerge is called on the server and the next download will get a new version with the updated strings.
I am not sure about the relation between the first sentence and the second.
It was written before my first coffee - sorry ;( Let's rewrite the second one as: On the other hand, once a translator updates a file, msgmerge is called on the server and the next download will get a new version with the updated content.
Is POT file updated between the first and the second sentences?
No.
If there is no translation update on the server side, it works as expected. Note that Transifex does not merge translations on Transifex and uploaded translations.
Right now a "tx pull -a" will not download merged language pot files - see for example https://review.openstack.org/#/c/82434/ where doc/install-guide/locale/install-guide.pot is updated but not any of the po files.
Downloaded translations are already synced with POT file when POT file is uploaded to the server.
In case of propose_translations_manuals, 1. POT is updated (generatepot) 2. POT is pushed to Transifex if there are any update. (At this point, translations on Transifex is "msgmege"d with the uploaded POT and translaitons becomes up-to-date.) 3. "tx pull -a" retrieves all up-to-date translations. 4 Finally "git review" if required
No translations will be lost in this flow and it looks good. The flow in the original propose_translation_horizon in your review is different. tx pull -> generatepot -> msgmerge -> tx push translations This is the reason I wrote it may lose translations.
On the other hand, there is no need in propose_translation_XXX.sh. If there are new or changed strings in POT file in step 2 above, we don't have translations for such strings in step 3. Even if POT and PO files are not synced, we will also get the same results (no translations for new or changed strings). This is the reason I wrote step 2 above is necessarily required in propose_translation script. Does it make sense?
Yes, makes sense - and in that case all our scripts are wrong. Let me double check this later and document this.
1) Is the above correct? 2) Is that a use case we care about? Akihiro, Sascha, how is your workflow? Could you document it in the wiki, I don't see that covered...
My use case to use "tx pull" locally is just to check translations with some conversion (for manuals) or programs like horizon. I don't think I am doing special things. I don't usually use local files for translations if Transifex is used.
I am not sure I answer what you ask.
It helps, thanks!
Also, do we really need to push the POT file to transifex? It should be done as part of the uptream_translation post job, isn't it?
Yes, the POT file should only be updated/uploaded in the post gate if it changes.
Yes, as Lukasz commented.
I don't know why Clark did the upload when writing the initial script. Clark, do you remember?
But we need to *update* it locally so that it can get committed to git (the update script does not handle updating of git) - just the upload looks unnecessary.
As I wrote above, small out-of-sync between POT and PO files and synced PO file with untranslated strings have no difference as a result.
I hope I answered your question.
Thanks, Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
participants (3)
-
Akihiro Motoki
-
Andreas Jaeger
-
Łukasz Jernaś