[Openstack-operators] Openstack and mysql galera with haproxy

Sławek Kapłoński slawek at kaplonski.pl
Fri Sep 19 13:55:49 UTC 2014


Hello,

W dniu 2014-09-18 18:45, Clint Byrum napisał(a):
> Excerpts from Sławek Kapłoński's message of 2014-09-18 09:29:27 -0700:
>> Hello,
>> 
>> Is anyone here using openstack with mysql galera and haproxy? Have You 
>> got any
>> problems with that?
>> I was today installed such ha infra for database (two mysql servers in 
>> galera
>> cluster and haproxy on controller and neutron node, this haproxy is 
>> connecting
>> to one of galera servers with round robin algorithm). Generally all is 
>> working
>> fine but I have few problems:
>> 1. I have a lot of messages like:
>> WARNING neutron.openstack.common.db.sqlalchemy.session [-] Got mysql 
>> server
>> has gone away: (2006, 'MySQL server has gone away')
>> 2. I have (most on neutron) many errors like:
>> OperationalError: (OperationalError) (2013, 'Lost connection to MySQL 
>> server
>> during query') 'UPDATE ml2_port_bindings SET vif_type=%s, driver=%s,
>> segment=%s WHERE ml2_port_bindings.port_id =
> 
> 1 and 2 look like timeout issues. Check haproxy's timeouts. They need
> to be just a little longer than MySQL's connection timeouts.

But what exactly timeouts should I check in haproxy and mysql server? 
Maybe I should also adjust "pool_timeout" in neutron.conf (I don't sure 
is it also in nova but probably yes)?

> 
>> 3. Also errors:
>> StaleDataError: UPDATE statement on table 'ports' expected to update 1 
>> row(s);
>> 0 were matched.
>> 4. and errors:
>> DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to 
>> get lock;
>> try restarting transaction') 'UPDATE ipavailabilityranges SET 
>> first_ip=%s WHERE
>> ipavailabilityranges.allocation_pool_id =
> 
> 3 and 4 are a known issue. Our code doesn't always retry transactions,
> which is required to use Galera ACTIVE/ACTIVE. Basically, that doesn't
> work.
> 
> You can use ACTIVE/PASSIVE, and even do vertical partitioning where
> one of the servers is ACTIVE for Nova, but another one is ACTIVE for
> Neutron. But AFAIK, ACTIVE/ACTIVE isn't being tested and the work 
> hasn't
> been done to make the concurrent transactions work properly.
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-- 
Pozdrawiam
Sławek Kapłoński
slawek at kaplonski.pl



More information about the OpenStack-operators mailing list