Open Stack

>
>
> Mike Bayer <mbayer at redhat.com> wrote on 11/19/2014 02:10:18 PM:
>
> > From: Mike Bayer <mbayer at redhat.com>
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev at lists.openstack.org>
> > Date: 11/19/2014 02:11 PM
> > Subject: Re: [openstack-dev] [Neutron] DB: transaction isolation and
> > related questions
> >
> > On Nov 19, 2014, at 1:49 PM, Ryan Moats <rmoats at us.ibm.com> wrote:
> >
> > I was waiting for this because I think I may have a slightly
> > different (and outside of the box) view on how to approach a solution
to this.
> >
> > Conceptually (at least in my mind) there isn't a whole lot of
> > difference between how the example below (i.e. updates from two
> > concurrent threads) is handled
> > and how/if neutron wants to support a multi-master database scenario
> > (which in turn lurks in the background when one starts thinking/
> > talking about multi-region support).
> >
> > If neutron wants to eventually support multi-master database
> > scenarios, I see two ways to go about it:
> >
> > 1) Defer multi-master support to the database itself.
> > 2) Take responsibility for managing the conflict resolution inherent
> > in multi-master scenarios itself.
> >
> > The first approach is certainly simpler in the near term, but it has
> > the down side of restricting the choice of databases to those that
> > have solved multi-master and further, may lead to code bifurcation
> > based on possibly different solutions to the conflict resolution
> > scenarios inherent in multi-master.
> > The second approach is certainly more complex as neutron assumes
> > more responsibility for its own actions, but it has the advantage
> > that (if done right) would be transparent to the underlying
> > databases (with all that implies)
>
> multi-master is a very advanced use case so I don’t see why it would
> be unreasonable to require a multi-master vendor database.
> Reinventing a complex system like that in the application layer is
> an unnecessary reinvention.
>
> As far as working across different conflict resolution scenarios,
> while there may be differences across backends, these differences
> will be much less significant compared to the differences against
> non-clustered backends in which we are inventing our own multi-
> master solution.   I doubt a home rolled solution would insulate us
> at all from “code bifurcation” as this is already a fact of life in
> targeting different backends even without any implication of
> clustering.   Even with simple things like transaction isolation, we
> see that different databases have different behavior, and if you
> look at the logic in oslo.db inside of https://github.com/openstack/
> oslo.db/blob/master/oslo/db/sqlalchemy/exc_filters.py you can see an
> example of just how complex it is to just do the most rudimental
> task of organizing exceptions into errors that mean the same thing.

I didn't say it was unreasonable, I only point out that there is an
alternative for consideration.

BTW, I view your examples from oslo as helping make my argument for
me (and I don't think that was your intent :) )

> > My reason for asking this question here is that if the community
> > wants to consider #2, then these problems are the place to start
> > crafting that solution - if we solve the conflicts inherent with the
> > two conncurrent thread scenarios, then I think we will find that
> > we've solved the multi-master problem essentially "for free”.
>
> Maybe I’m missing something, if we learn how to write out a row such
> that a concurrent transaction against the same row doesn’t throw us
> off, where is the part where that data is replicated to databases
> running concurrently on other IP numbers in a way that is atomic
> come out of that effort “for free” ?   A home-rolled “multi master”
> scenario would have to start with a system that has multiple
> create_engine() calls, since we need to communicate directly to
> multiple database servers. From there it gets really crazy.  Where’sall
that ?

Boiled down, what you are talking about here w.r.t. concurrent
transactions is really conflict resolution, which is the hardest
part of implementing multi-master (as a side note, using locking in
this case is the equivalent of option #1).

All I wished to point out is that there are other ways to solve the
conflict resolution that could then be leveraged into a multi-master
scenario.

As for the parts that I glossed over, once conflict resolution is
separated out, replication turns into a much simpler problem with
well understood patterns and so I view that part as coming
"for free."

Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141119/f0d2923e/attachment.html>