On Mon, Jan 16, 2017, at 09:37 AM, Ian Y. Choi wrote:
Hello,
I already talked with Alex through IRC: Testing copy version with a smaller scale project was completely fine.
As ianw shared: http://lists.openstack.org/pipermail/openstack-infra/2017-January/005058.htm... and http://lists.openstack.org/pipermail/openstack-i18n/2017-January/002670.html , gc and database status might be suspected which contribute to too slow Zanata.
Some additional thoughts from me are:
- Different mysql driver from Trusty: https://review.openstack.org/#/c/406398/1/manifests/init.pp might contribute, after seeing Alex's comment on "mysql driver". - Current master version of openstack-manuals in translate.o.o (not translate-dev.o.o) has 17,423,952 words, which is larger than the number of words for versions: test-version-creation2, test-version-creation, and stable-newton. If it is really the bug from Zanata, then future execution of creating a new version: "stable-ocata" after Ocata documentation release will contribute to production server downtime.
@clarkb, then might it be better to also test with larger memory size? Or testing openstackid-dev for a while would be a good idea now?
One correction: I thought http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/s... enumerates the root admins in translate.openstack.org and also translate-dev.openstack.org, but it was my complete misinterpretation on the file. After deeper investigation, I have found that the list is for Zanata instance admins : https://review.openstack.org/#/c/172528/ .
I'd like to make sure we get back on the same page with everything going on as I feel like we have all gone in about 5 different directions the last couple days. Clarifications: We have not upgraded to 3.9.6 yet. We are merely running the same version of Zanata as in production (3.7.3) on top of Xenial and Java 8 instead of Trusty and Java 7. This is because it is easier to separate the OS upgrade from the application upgrade and will allow us to upgrade the OS for translate.openstack.org first, then upgrade Zanata. The MySQL instance is running in a Trove instance and is not collocated with Zanata on the same instance. The Zanata wildfly instance can use 4096 Megabytes of memory (according to the Xmx value) on an 8GB memory VM. TODO: There have been a list of things that have been suggested that we try and I would like use to decide whcih are most important and start there. Increase memory on translate-dev.openstack.org host. I think the next biggest VM size is 16GB of ram. That doubles system memory and we can double Xmx to 8192MB as well. Increase size of Trove database instance. It is currently 2GB RAM instance, we can double that to 4GB. Upgrade Zanata to 3.9.6 on translate-dev.openstack.org if you are happy with calling this issue and Zanata bug and go from there. Then once at least some of this is done and we are happy with it we can upgrade the production instance. What are the things that people feel are important here? Personally I think that if we only have performance issues when hitting what is probably a Zanata bug then I wouldn't bother increasing the resources available to the dev instance and instead continue focusing on testing of the upgrade. My suggestion would be to upgrade to 3.9.6 and if it looks good do the production instance as well. But feedback is much appreciated as I know that others have been putting a bit of effort into this and may have better ideas than me :) Clark