Open Stack

Hey list -

As many are aware, we’ve been looking for the one MySQL driver to rule them all.  As has been the case for some weeks now, that driver is PyMySQL, meeting all the critical requirements we have of:  1. pure python, so eventlet patchable, 2. Python 3 support 3. Is released on Pypi (which is what disqualifies MySQL-connector-Python).

I have experience with PyMySQL and I was at first a little skeptical that it’s ready for openstack’s level of activity, so I offered to have oslo.db write up a full review of everything we know about all the MySQL drivers, including that I’d evaluate PyMySQL and try to put some actual clue behind my vague notions.   

I finally got around to writing up the code review portion, so that we at least have awareness of what we’re getting with PyMySQL.  This is also a document that I’m very much hoping we can use to get the PyMySQL developers involved with.   Basically PyMySQL is fine, it lacks some polish and test coverage that can certainly be added, and as far as performance, we’re going to be really disappointed with pure-Python MySQL drivers in general, though PyMySQL itself can still be improved within the realm of pure Python.  It at least allows for eventlet monkey patching, so that we will for the first time be able to observe the benefits of some real database concurrency ever since Openstack decided not to use threads (if Openstack did in fact ever use normal threads…), which also means we will be able to observe the headaches of real database concurrency, especially the really scary ones I’ve already observed when working with eventlet/gevent style monkeypatching :).

While PyMySQL is lacking test coverage in some areas, has no external documentation, and has at least some areas where Python performance can be improved, the basic structure of the driver is perfectly fine and straightforward.  I can envision turning this driver into a total monster, adding C-speedups where needed but without getting in the way of async patching, adding new APIs for explicit async, and everything else.   However, I’ve no idea what the developers have an appetite for.

Please review the document at https://wiki.openstack.org/wiki/PyMySQL_evaluation.

- mike