Open Stack

Hey all, Mark,

Yes, that's exactly what we were going to do! (and there is a similar
class in eventlet itself, Victor is currently trying to run Nova with
it applied, so we might end up with no module/class at all, but rather
use the one from eventlet instead)

Thank you all for your feedback!

Roman

On Wed, Dec 4, 2013 at 1:45 AM, Mark McLoughlin <markmc at redhat.com> wrote:
> On Mon, 2013-12-02 at 16:02 +0200, Victor Sergeyev wrote:
>> Hi folks!
>>
>> At the moment I and Roman Podoliaka are working on splitting of
>> openstack.common.db code into a separate library. And it would be nice to
>> drop dependency on eventlet before oslo.db is released.
>>
>> Currently, there is only one place in oslo.db where we use eventlet -
>> wrapping of DB API method calls to be executed by tpool threads. It wraps
>> DB API calls to be executed by tpool threads. This is only needed when
>> eventlet is used together with DB-API driver implemented as a Python C
>> extension (eventlet can't monkey patch C code, so we end up with DB API
>> calls blocking all green threads when using Python-MySQLdb). eventlet has a
>> workaround known as 'tpool' which is basically a pool of real OS threads
>> that can play nicely with eventlet event loop. tpool feature is
>> experimental and known to have stability problems. There is a doubt that
>> anyone is using it in production at all. Nova API (and probably other API
>> services) has an option to prefork the process on start, so that they don't
>> need to use tpool when using eventlet together Python-MySQLdb.
>>
>> We'd really like to drop tpool support from oslo.db, because as a library
>> we should not be bound to any particular concurrency model. If a target
>> project is using eventlet, we believe, it is its problem how to make it
>> play nicely with Python-MySQLdb lib, but not the problem of oslo.db.
>> Though, we could put tpool wrapper into another helper module within
>> oslo-incubator.
>>
>> But we would really-really like not to have any eventlet related code in
>> oslo.db.
>>
>> Are you using CONF.database.use_tpool in production? Does the approach with
>> a separate tpool wrapper class seem reasonable? Or we can just drop tpool
>> support at all, if no one is using it?
>
> Another approach is to put the tpool wrapper class in a separate module
> which would be completely optional for users of the library.
>
> For example, you could imagine people who don't want this doing:
>
>   from oslo import db
>
>   dbapi = db.DBAPI()
>
> but if you want the tpool thing, you might do:
>
>   from oslo import db
>   from oslo.db import eventlet as db_eventlet
>
>   dbapi = db_eventlet.TpoolWrapper(db.DBAPI())
>
> (I'm just making stuff up, but you get the idea)
>
> The key thing is that eventlet isn't a hard dependency of the library,
> but the useful eventlet integration is still available in the library if
> you want it.
>
> We did something similar in oslo.messaging, and the issues there were
> probably more difficult to deal with.
>
> Mark.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev