[openstack-dev] [Olso][DB] Remove eventlet from oslo.db
markmc at redhat.com
Tue Dec 3 23:45:07 UTC 2013
On Mon, 2013-12-02 at 16:02 +0200, Victor Sergeyev wrote:
> Hi folks!
> At the moment I and Roman Podoliaka are working on splitting of
> openstack.common.db code into a separate library. And it would be nice to
> drop dependency on eventlet before oslo.db is released.
> Currently, there is only one place in oslo.db where we use eventlet -
> wrapping of DB API method calls to be executed by tpool threads. It wraps
> DB API calls to be executed by tpool threads. This is only needed when
> eventlet is used together with DB-API driver implemented as a Python C
> extension (eventlet can't monkey patch C code, so we end up with DB API
> calls blocking all green threads when using Python-MySQLdb). eventlet has a
> workaround known as 'tpool' which is basically a pool of real OS threads
> that can play nicely with eventlet event loop. tpool feature is
> experimental and known to have stability problems. There is a doubt that
> anyone is using it in production at all. Nova API (and probably other API
> services) has an option to prefork the process on start, so that they don't
> need to use tpool when using eventlet together Python-MySQLdb.
> We'd really like to drop tpool support from oslo.db, because as a library
> we should not be bound to any particular concurrency model. If a target
> project is using eventlet, we believe, it is its problem how to make it
> play nicely with Python-MySQLdb lib, but not the problem of oslo.db.
> Though, we could put tpool wrapper into another helper module within
> But we would really-really like not to have any eventlet related code in
> Are you using CONF.database.use_tpool in production? Does the approach with
> a separate tpool wrapper class seem reasonable? Or we can just drop tpool
> support at all, if no one is using it?
Another approach is to put the tpool wrapper class in a separate module
which would be completely optional for users of the library.
For example, you could imagine people who don't want this doing:
from oslo import db
dbapi = db.DBAPI()
but if you want the tpool thing, you might do:
from oslo import db
from oslo.db import eventlet as db_eventlet
dbapi = db_eventlet.TpoolWrapper(db.DBAPI())
(I'm just making stuff up, but you get the idea)
The key thing is that eventlet isn't a hard dependency of the library,
but the useful eventlet integration is still available in the library if
you want it.
We did something similar in oslo.messaging, and the issues there were
probably more difficult to deal with.
More information about the OpenStack-dev