Open Stack

> On Nov 24, 2014, at 12:40 PM, Doug Hellmann <doug at doughellmann.com> wrote:
> 
> 
> This is a good point. I’m not sure we can say “we’ll only use explicit/implicit async in certain cases" because most of our apps actually mix the cases. We have WSGI apps that send RPC messages and we have other apps that receive RPC messages and operate on the database. Can we mix explicit and implicit operating models, or are we going to have to pick one way? If we have to pick one, the implicit model we’re currently using seems more compatible with all of the various libraries and services we depend on, but maybe I’m wrong?

IMHO, in the ideal case, a single method shouldn’t be mixing calls to a set of database objects as well as calls to RPC APIs at the same time, there should be some kind of method boundary to cross.   There’s a lot of ways to achieve that.

What is really needed is some way that code can switch between explicit yields and implicit IO on a per-function basis.   Like a decorator for one or the other.

The approach that Twisted takes of just using thread pools for those IO-bound elements that aren’t compatible with explicit yields is one way to do this.     This might be the best way to go, if there are in fact issues with mixing in implicit async systems like eventlet.  I can imagine, vaguely, that the eventlet approach of monkey patching might get in the way of things in this more complicated setup.

Part of what makes this confusing for me is that there’s a lack of clarity over what benefits we’re trying to get from the async work.  If the idea is, the GIL is evil so we need to ban the use of all threads, and therefore must use defer for all IO, then that includes database IO which means we theoretically benefit from eventlet monkeypatching  - in the absence of truly async DBAPIs, this is the only way to have deferrable database IO.

If the idea instead is, the code we write that deals with messaging would be easier to produce, organize, and understand given an asyncio style approach, but otherwise we aren’t terribly concerned what highly sequential code like database code has to do, then a thread pool may be fine.