Open Stack

Matthew Booth <mbooth at redhat.com> wrote:

> A: start transaction;
> B: start transaction;
> A: insert into foo values(1);
> B: insert into foo values(1); <-- 'regular' DB would block here, and
>                                  report an error on A's commit
> A: commit; <-- success
> B: commit; <-- KABOOM
> 
> Confusingly, Galera will report a 'deadlock' to node B, despite this not
> being a deadlock by any definition I'm familiar with.

So, one of the entire points of the enginefacade work is that we will ensure that writes will continue to be made to exactly one node in the cluster.  Openstack does not have the problem defined above, because we only communicate with one node, even today.  The work that we are trying to proceed with is to at least have *reads* make full use of the cluster.

The above phenomenon is not a problem for openstack today except for the reduced efficiency, which enginefacade will partially solve.

> 
> As I understand it, these can be eliminated by sending all writes to a
> single node, although that obviously makes less efficient use of your
> cluster.

this is what we do right now and it continues to be the plan going forward.   Having single-master is in fact the traditional form of clustering.  In the Openstack case, this issue isn’t as bad as it seems, because openstack runs many different applications against the same database simultaneously.  Different applications should refer to different nodes in the cluster as their “master”.   There’s no conflict here because each app talks only to its own tables.

> During a commit, Galera replicates a transaction out to all other db
> nodes. Due to its design, Galera knows these transactions will be
> successfully committed to the remote node eventually[2], but it doesn't
> commit them straight away. The remote node will check these outstanding
> replication transactions for write conflicts on commit, but not for
> read. This means that you can do:
> 
> A: start transaction;
> A: insert into foo values(1)
> A: commit;
> B: select * from foo; <-- May not contain the value we inserted above[3]

will need to get more detail on this.   this would mean that galera is not in fact synchronous.