[Openstack-operators] Galera setup testing

Joe Topjian joe at topjian.net
Fri Dec 11 16:06:41 UTC 2015

We do something similar: Instead of McRouter, we use the repcached patches
to replicate data between two memcached nodes. We then use HAProxy as a
single entry point for memcached requests.

We've been doing this for 6+ months and it's been working great. It's
effectively solved the issue I described in this thread last year:


On Fri, Dec 11, 2015 at 6:06 AM, Bajin, Joseph <jbajin at verisign.com> wrote:

> At this point, we use Keystone and UUID’s for our setup, but we don’t
> store the UUID tokens in the Database.  We use Memcache to do that.
> Actually we use McRouter and Memcache to make sure any node in our control
> plane can validate that token.
> —Joe
> From: Ajaya Agrawal <ajku.agr at gmail.com>
> Date: Friday, December 11, 2015 at 2:25 AM
> To: Matt Fischer <matt at mattfischer.com>
> Cc: "openstack-operators at lists.openstack.org" <
> openstack-operators at lists.openstack.org>
> Subject: Re: [Openstack-operators] Galera setup testing
> Thanks Matt. That surely is helpful. If you could share some numbers or
> problems you faced when you were storing UUID tokens in database, it would
> be awesome. In my test setup with Keystone Kilo, Fernet token creation and
> validation were way slower than UUID tokens. But UUID tokens come with a
> huge cost to database which is the pain point. I have never run Keystone
> with UUID tokens in Prod setup. So I am looking for perspective on Keystone
> with UUID in prod setup.
> Thanks to other people who also chimed in with advice.
> Cheers,
> Ajaya
> On Mon, Dec 7, 2015 at 8:34 PM, Matt Fischer <matt at mattfischer.com> wrote:
>> On Mon, Dec 7, 2015 at 3:54 AM, Ajaya Agrawal <ajku.agr at gmail.com> wrote:
>>> Hi everyone,
>>> We are deploying Openstack and planning to run multi-master Galera setup
>>> in production. My team is responsible for running a highly available
>>> Keystone. I have two questions when it comes to Galera with Keystone.
>>> 1. How do you test if a Galera cluster is setup properly?
>>> 2. Is there any Galera test specific to Keystone which you have found
>>> useful?
>> For 1 you could say that the clustercheck script which ships with
>> puppet-galera and is forked from
>> https://github.com/olafz/percona-clustercheck is a really simple check
>> that galera is up and the cluster is sync'd. It's main goal however is to
>> provide status to haproxy.
>> One thing you want to check is the turnaround time on operations, for
>> example, creating a user on a node and then immediately using them on
>> another node. We found that this is likely to sometimes (but rarely) fail.
>> The solution is two-fold, first, don't store tokens in mysql. Second,
>> designate one node as the primary in haproxy.
>> Other than that we've gotten good at reading the wsrep_ cluster status
>> info, but to be honest, once we removed tokens from the db, we've been in
>> way better shape.
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20151211/3b1d2949/attachment.html>

More information about the OpenStack-operators mailing list