Open Stack

Hi Li, comments inline.

On 08/08/2014 12:03 AM, Li Ma wrote:
> Getting a massive amount of information from data storage to be displayed is
> where most of the activity happens in OpenStack. The two activities of reading
> data and writing (creating, updating and deleting) data are fundamentally
> different.
>
> The optimization for these two opposite database activities can be done by
> physically separating the databases that service these two different
> activities. All the writes go to database servers, which then replicates the
> written data to the database server(s) dedicated to servicing the reads.
>
> Currently, AFAIK, many OpenStack deployment in production try to take
> advantage of MySQL (includes Percona or MariaDB) multi-master Galera cluster.
> It is possible to design and implement a read/write separation schema
> for such a DB cluster.

The above does not really make sense for MySQL Galera/PXC clusters *if 
only Galera nodes are used in the cluster*. Since Galera is 
synchronously replicated, there's no real point in segregating writers 
from readers, IMO. Better to just spread the write AND read load equally 
among all Galera cluster nodes.

However, if you have a Galera cluster that then slaves off to one or 
more standard MySQL slaves, then certainly doing writer/reader 
segregation could be useful, especially for directing readers of 
aggregate or report-type data to the read-only slaves.

> Actually, OpenStack has a method for read scalability via defining
> master_connection and slave_connection in configuration, but this method
> lacks of flexibility due to deciding master or slave in the logical
> context(code). It's not transparent for application developer.
> As a result, it is not widely used in all the OpenStack projects.
>
> So, I'd like to propose a transparent read/write separation method
> for oslo.db that every project may happily takes advantage of it
> without any code modification.

I've never seen a writer/reader segregation proxy or middleware piece 
that was properly able to send the "right" reads to the slaves. 
Unfortunately, determining what are the "right" reads to send to the 
slaves is highly application-dependent, since the application knows when 
it can tolerate slave lags.

> Moreover, I'd like to put it in the mailing list in advance to
> make sure it is acceptable for oslo.db.

I think oslo.db is not the right place for this. I believe the efforts 
that Mike Wilson has been doing in the "slavification" blueprints are 
the more appropriate place to add this slave-aware code.

Best,
-jay

> I'd appreciate any comments.
>
> br.
> Li Ma
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>