[openstack-dev] Asynchrounous programming: replace eventlet with asyncio

victor stinner victor.stinner at enovance.com
Thu Feb 6 09:52:02 UTC 2014


Hi,

Joshua Harlow:
> Any mysql DB drivers (I think the majority of openstack deployments use
> mysql?).

I don't know. Here are some asynchronous clients for MySQL:

https://github.com/PyMySQL/PyMySQL/
https://launchpad.net/myconnpy
https://github.com/hybridlogic/txMySQL
http://chartio.com/blog/2011/06/making-mysql-queries-asynchronous-in-tornado
http://www.arpalert.org/mysac.html
http://code.google.com/p/perl-mysql-async/

IMO to have an efficient driver for asyncio, it should give access to the socket / file descriptor, so asyncio can watch it can execute a callback when some data can be read on the socket. A pure Python connector should fit such requirements. Or the API should use a callback when the result is available.

> How about sqlalchemy (what would possibly need to change there for it to
> work)?

I found some projects using SQLAchemy asynchronously, but only with PostgreSQL.

> The pain that I see is that to connect all these libraries into
> asyncio they have to invert how they work (sqlalchemy would have to become
> asyncio compatible (?), which probably means a big rewrite).

There is no problem to call slow blocking functions in asyncio.

But if you want to have efficient code, it's better to run the blocking code asynchronously. For example, use loop.run_in_executor() with a thread pool.

> This is where
> it would be great to have a 'eventlet' like-thing built ontop of asyncio
> (letting existing libraries work without rewrites). Eventually I guess
> in-time (if tulip succeeds) then this 'eventlet' like-thing could be
> removed.

It's a little bit weird to design an abstraction on top of asyncio, since asyncio has be designed an an abstraction of existing event loops. But I wrote an asyncio executor for Oslo Messaging which has already such abstraction.

Victor



More information about the OpenStack-dev mailing list