[openstack-dev] [Fuel][MySQL][DLM][Oslo][DB][Trove][Galera][operators] Multi-master writes look OK, OCF RA and more things
bdobrelia at mirantis.com
Tue May 17 16:21:34 UTC 2016
On 04/22/2016 05:42 PM, Bogdan Dobrelya wrote:
> [crossposting to openstack-operators at lists.openstack.org]
> I wrote this paper  to demonstrate an approach how we can leverage a
> Jepsen framework for QA/CI/CD pipeline for OpenStack projects like Oslo
> (DB) or Trove, Tooz DLM and perhaps for any integration projects which
> rely on distributed systems. Although all tests are yet to be finished,
> results are quite visible, so I better off share early for a review,
> discussion and comments.
> I have similar tests done for the RabbitMQ OCF RA clusterers as well,
> although have yet wrote a report.
> PS. I'm sorry for so many tags I placed in the topic header, should I've
> used just "all" :) ? Have a nice weekends and take care!
>  https://goo.gl/VHyIIE
[ cross posting to operators ]
I added Appendix B there I made a few more tests dancing mostly around
that funny topic  full of interesting nuances, and there I tried to
cover some generic patterns OpenStack uses for transactions constructed
by sqlalchemy's ORM (I hope so).
Those test cases cover a5a read skews, SERIALIZABLE / RR / RC TI levels,
lock modes for select, and wsrep_sync_wait Galera settings. And reworked
conclusions and recommendations sections by the new tests results as well.
For now, I've finished all items I had on my TODO list for that paper.
If anyone would like to do more test runs, re-use the given approach to
the cluster-labs' upstream OCF RA or re-check with another configuration
tunings (mostly to wsrep& like things perhaps), you're welcome! I'm open
for questions, if any.
Also note, that with all submitted fixes to those multiple
testing-discovered bugs, cluster recovery after network partitions have
been working almost seamlessly, for me :-) For those who interested, the
full list of related bugs is easy to locate in this backport's commit
The link is the same .
More information about the OpenStack-dev