Open Stack

On Mon, Jan 16, 2017, at 09:37 AM, Ian Y. Choi wrote:
> Hello,
> 
> I already talked with Alex through IRC: Testing copy version with a 
> smaller scale project was completely fine.
> 
> As ianw shared: 
> http://lists.openstack.org/pipermail/openstack-infra/2017-January/005058.html 
> and
> http://lists.openstack.org/pipermail/openstack-i18n/2017-January/002670.html 
> ,
> gc and database status might be suspected which contribute to too slow 
> Zanata.
> 
> Some additional thoughts from me are:
> 
> - Different mysql driver from Trusty: 
> https://review.openstack.org/#/c/406398/1/manifests/init.pp
>    might contribute, after seeing Alex's comment on "mysql driver".
> - Current master version of openstack-manuals in translate.o.o (not 
> translate-dev.o.o) has 17,423,952 words, which is larger
>    than the number of words for versions: test-version-creation2, 
> test-version-creation, and stable-newton.
>    If it is really the bug from Zanata, then future execution of 
> creating a new version: "stable-ocata" after Ocata documentation release
>    will contribute to production server downtime.
> 
> @clarkb, then might it be better to also test with larger memory size?
>      Or testing openstackid-dev for a while would be a good idea now?
> 
> 
> One correction: I thought 
> http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n1392
> enumerates the root admins in translate.openstack.org and also 
> translate-dev.openstack.org, but it was my complete
> misinterpretation on the file. After deeper investigation, I have found 
> that the list is for Zanata instance admins
> : https://review.openstack.org/#/c/172528/ .

I'd like to make sure we get back on the same page with everything going
on as I feel like we have all gone in about 5 different directions the
last couple days.

Clarifications:

We have not upgraded to 3.9.6 yet. We are merely running the same
version of Zanata as in production (3.7.3) on top of Xenial and Java 8
instead of Trusty and Java 7. This is because it is easier to separate
the OS upgrade from the application upgrade and will allow us to upgrade
the OS for translate.openstack.org first,  then upgrade Zanata.

The MySQL instance is running in a Trove instance and is not collocated
with Zanata on the same instance.

The Zanata wildfly instance can use 4096 Megabytes of memory (according
to the Xmx value) on an 8GB memory VM.

TODO:

There have been a list of things that have been suggested that we try
and I would like use to decide whcih are most important and start there.

Increase memory on translate-dev.openstack.org host. I think the next
biggest VM size is 16GB of ram. That doubles system memory and we can
double Xmx to 8192MB as well.

Increase size of Trove database instance. It is currently 2GB RAM
instance, we can double that to 4GB.

Upgrade Zanata to 3.9.6 on translate-dev.openstack.org if you are happy
with calling this issue and Zanata bug and go from there.

Then once at least some of this is done and we are happy with it we can
upgrade the production instance.

What are the things that people feel are important here? Personally I
think that if we only have performance issues when hitting what is
probably a Zanata bug then I wouldn't bother increasing the resources
available to the dev instance and instead continue focusing on testing
of the upgrade. My suggestion would be to upgrade to 3.9.6 and if it
looks good do the production instance as well.

But feedback is much appreciated as I know that others have been putting
a bit of effort into this and may have better ideas than me :)

Clark