Open Stack

On Mon, 2014-07-07 at 12:48 +0200, Nikola Đipanov wrote:

> When I read all of this stuff and got my head around it (took some time
> :) ), a glaring drawback of such an approach, and as I mentioned on the
> spec proposing it [1] is that we would not really doing asyncio, we
> would just be pretending we are by using a subset of it's APIs, and
> having all of the really important stuff for overall design of the code
> (code that needs to do IO in the callbacks for example) and ultimately -
> performance, completely unavailable to us when porting.
> 
> So in Mark's example above:
> 
>   @asyncio.coroutine
>   def foo(self):
>     result = yield from some_async_op(...)
>     return do_stuff(result)
> 
> A developer would not need to do anything that asyncio requires like
> make sure that some_async_op() registers a callback with the eventloop
> (using for example event_loop.add_reader/writer methods) you could just
> simply make it use a 'greened' call and things would continue working
> happily.

Yes, Victor and I noticed this problem and wondered whether there was a
way to e.g. turn-off the monkey-patching at runtime in a single
greenthread, or even just make any attempt to context switch raise an
exception.

i.e. a way to run foo() coroutine above in a greenthread such that
context switching is disallowed, or logged, or whatever while the
function is running. The only way context switching would be allowed to
happen would be if the coroutine yielded.

>  I have a feeling this will in turn have a lot of people writing
> code that they don't understand, and as library writers - we are not
> doing an excellent job at that point.
> 
> Now porting an OpenStack project to another IO library with completely
> different design is a huge job and there is unlikely a single 'right'
> way to do it, so treat this as a discussion starter, that will hopefully
> give us a better understanding of the problem we are trying to tackle.
> 
> So I hacked up together a small POC of a different approach. In short -
> we actually use a real asyncio selector eventloop in a separate thread,
> and dispatch stuff to it when we figure out that our callback is in fact
> a coroutine. More will be clear form the code so:
> 
> (Warning - hacky code ahead): [2]
> 
> I will probably be updating it - but if you just clone the repo, all the
> history is there. I wrote it without the oslo.messaging abstractions
> like listener and dispatcher, but it is relatively easy to see which
> bits of code would go in those.
> 
> Several things worth noting as you read the above. First one is that we
> do not monkeypatch until we have fired of the asyncio thread (Victor
> correctly noticed this would be a problem in a comment on [1]). This may
> seem hacky (and it is) but if decide to go further down this road - we
> would probably not be 'greening the world' but rather importing patched
> non-ported modules when we need to dispatch to them. This may sound like
> a big deal, and it is, but it is critical to actually running ported
> code in a real asyncio evenloop. I have not yet tested this further, but
> from briefly reading eventlet code - it seems like ti should work.
> 
> Another interesting problem is (as I have briefly mentioned in [1]) -
> what happens when we need to synchronize between eventlet-run and
> asyncio-run callbacks while we are in the process of porting. I don't
> have a good answer to that yet, but it is worth noting that the proposed
> approach doesn't either, and this is a thing we should have some idea
> about before going in with a knife.
> 
> Now for some marketing :) - I can see several advantages of such an
> approach, the obvious one being as stated, that we are in fact doing
> asyncio, so we are all in. Also as you can see [2] the implementation is
> far from magical - it's (surprisingly?) simple, and requires no other
> additional dependencies apart from trollius itself (granted greenio is
> not too complex either). I am sure that we would hit some other problems
> that were not clear from this basic POC (it was done in ~3 hours on a
> bus), but it seems to me that those problems will likely need to be
> solved anyhow if we are to port Ceilometer (or any other project) to
> asyncio, we will just hit them sooner this way.
> 
> It was a fun approach to ponder anyway - so I am looking forward to
> comments and thoughts.

It's an interesting idea and I'd certainly welcome a more detailed
analysis of what the approach would mean for a service like Ceilometer.

My instinct is that adding an additional native thread where there is
only one native thread now will lead to tricky concurrency issues and a
more significant change of behavior than with the greenio approach. The
reason I like the greenio idea is that it allows us to make the
programming model changes without very significantly changing what
happens at runtime - the behavior, order of execution, concurrency
concerns, etc. shouldn't be all that different.

Mark.