Open Stack

Angus Lees <gus at inodes.org> wrote:

> 
> If we absolutely can't switch to another mysql driver, another option that was suggested recently (and passes the above test) is using eventlet.monkey_patch(MySQLdb=True).  I haven't done the investigation to find out why that isn't the default, or what the downsides are.  This obviously doesn't help us with other factors, like python3-ness either.

it throws the MySQLdb driver into a thread pool, just like Twisted’s DBAPI adapter does:

https://github.com/eventlet/eventlet/blob/master/eventlet/green/MySQLdb.py

Which leads into “tpool”, which we can confirm is native threads right here:  https://github.com/eventlet/eventlet/blob/master/eventlet/tpool.py#L33

So using MySQLdb patching with eventlet within database-centric processes, we gain:   absolutely nothing.   At the end of the day, we’re still using threads, or we’re doing it wrong.    Our connection pools are configured (or more accurately, are left at SQLAlchemy’s default setting) to only allow 15 concurrent connections at a time in any case, beyond which the pool blocks on new connection attempts; we need this in any case because when running a few dozen apps/processes, each with their own pool of 5-15 connections, that adds up to many dozens against the database overall.   It is not possible or even appropriate for us to get anywhere near the many dozens / hundreds of concurrent connections per-process that is really the only thing gevent/eventlet can do that regular threads cannot.

The entire eventlet / async game is utterly absurd.    It’s done absolutely nothing for Openstack except cause confusion, instability, and complexity.  It has never been done correctly, is incorrect in all of our applications right now, and if it were done “correctly”, our code could only run slower if anything, since both the GIL and eventlet are not IO-bound but both serialize CPU through a single straw within a process.