<html><body>
<p><font size="2" face="sans-serif">> </font><br>
<font size="2" face="sans-serif">> </font><br>
<tt><font size="2">> Mike Bayer <mbayer@redhat.com> wrote on 11/19/2014 02:10:18 PM:<br>
> <br>
> > From: Mike Bayer <mbayer@redhat.com></font></tt><br>
<tt><font size="2">> > To: "OpenStack Development Mailing List (not for usage questions)" <br>
> > <openstack-dev@lists.openstack.org></font></tt><br>
<tt><font size="2">> > Date: 11/19/2014 02:11 PM</font></tt><br>
<tt><font size="2">> > Subject: Re: [openstack-dev] [Neutron] DB: transaction isolation and<br>
> > related questions</font></tt><br>
<tt><font size="2">> > <br>
> > On Nov 19, 2014, at 1:49 PM, Ryan Moats <rmoats@us.ibm.com> wrote:</font></tt><br>
<tt><font size="2">> > <br>
> > I was waiting for this because I think I may have a slightly <br>
> > different (and outside of the box) view on how to approach a solution to this.<br>
> > <br>
> > Conceptually (at least in my mind) there isn't a whole lot of <br>
> > difference between how the example below (i.e. updates from two <br>
> > concurrent threads) is handled<br>
> > and how/if neutron wants to support a multi-master database scenario<br>
> > (which in turn lurks in the background when one starts thinking/<br>
> > talking about multi-region support).<br>
> > <br>
> > If neutron wants to eventually support multi-master database <br>
> > scenarios, I see two ways to go about it:<br>
> > <br>
> > 1) Defer multi-master support to the database itself.<br>
> > 2) Take responsibility for managing the conflict resolution inherent<br>
> > in multi-master scenarios itself.<br>
> > <br>
> > The first approach is certainly simpler in the near term, but it has<br>
> > the down side of restricting the choice of databases to those that <br>
> > have solved multi-master and further, may lead to code bifurcation <br>
> > based on possibly different solutions to the conflict resolution <br>
> > scenarios inherent in multi-master.</font></tt><br>
<tt><font size="2">> > The second approach is certainly more complex as neutron assumes <br>
> > more responsibility for its own actions, but it has the advantage <br>
> > that (if done right) would be transparent to the underlying <br>
> > databases (with all that implies)</font></tt><br>
<tt><font size="2">> </font></tt><br>
<tt><font size="2">> multi-master is a very advanced use case so I don’t see why it would<br>
> be unreasonable to require a multi-master vendor database. <br>
> Reinventing a complex system like that in the application layer is</font></tt><br>
<tt><font size="2">> an unnecessary reinvention.</font></tt><br>
<tt><font size="2">> <br>
> As far as working across different conflict resolution scenarios, <br>
> while there may be differences across backends, these differences <br>
> will be much less significant compared to the differences against <br>
> non-clustered backends in which we are inventing our own multi-<br>
> master solution. I doubt a home rolled solution would insulate us <br>
> at all from “code bifurcation” as this is already a fact of life in <br>
> targeting different backends even without any implication of <br>
> clustering. Even with simple things like transaction isolation, we<br>
> see that different databases have different behavior, and if you <br>
> look at the logic in oslo.db inside of <a href="https://github.com/openstack/">https://github.com/openstack/</a><br>
> oslo.db/blob/master/oslo/db/sqlalchemy/exc_filters.py you can see an<br>
> example of just how complex it is to just do the most rudimental <br>
> task of organizing exceptions into errors that mean the same thing.</font></tt><br>
<br>
<tt><font size="2">I didn't say it was unreasonable, I only point out that there is an </font></tt><br>
<tt><font size="2">alternative for consideration. </font></tt><br>
<br>
<tt><font size="2">BTW, I view your examples from oslo as helping make my argument for</font></tt><br>
<tt><font size="2">me (and I don't think that was your intent :) )</font></tt><br>
<br>
<tt><font size="2">> > My reason for asking this question here is that if the community <br>
> > wants to consider #2, then these problems are the place to start <br>
> > crafting that solution - if we solve the conflicts inherent with the<br>
> > two conncurrent thread scenarios, then I think we will find that <br>
> > we've solved the multi-master problem essentially "for free”.</font></tt><br>
<tt><font size="2">> <br>
> Maybe I’m missing something, if we learn how to write out a row such<br>
> that a concurrent transaction against the same row doesn’t throw us <br>
> off, where is the part where that data is replicated to databases <br>
> running concurrently on other IP numbers in a way that is atomic <br>
> come out of that effort “for free” ? A home-rolled “multi master” <br>
> scenario would have to start with a system that has multiple <br>
> create_engine() calls, since we need to communicate directly to <br>
> multiple database servers. From there it gets really crazy. Where’sall that ?</font></tt><br>
<br>
<tt><font size="2">Boiled down, what you are talking about here w.r.t. concurrent</font></tt><br>
<tt><font size="2">transactions is really conflict resolution, which is the hardest</font></tt><br>
<tt><font size="2">part of implementing multi-master (as a side note, using locking in</font></tt><br>
<tt><font size="2">this case is the equivalent of option #1). </font></tt><br>
<br>
<tt><font size="2">All I wished to point out is that there are other ways to solve the</font></tt><br>
<tt><font size="2">conflict resolution that could then be leveraged into a multi-master</font></tt><br>
<tt><font size="2">scenario.</font></tt><br>
<br>
<tt><font size="2">As for the parts that I glossed over, once conflict resolution is</font></tt><br>
<tt><font size="2">separated out, replication turns into a much simpler problem with</font></tt><br>
<tt><font size="2">well understood patterns and so I view that part as coming</font></tt><br>
<tt><font size="2">"for free."</font></tt><br>
<br>
<tt><font size="2">Ryan</font></tt></body></html>