Open Stack

On Thu, Jul 3, 2014 at 11:27 AM, Mark McLoughlin <markmc at redhat.com> wrote:
> Hey
>
> This is an attempt to summarize a really useful discussion that Victor,
> Flavio and I have been having today. At the bottom are some background
> links - basically what I have open in my browser right now thinking
> through all of this.
>
> We're attempting to take baby-steps towards moving completely from
> eventlet to asyncio/trollius. The thinking is for Ceilometer to be the
> first victim.
>
> Ceilometer's code is run in response to various I/O events like REST API
> requests, RPC calls, notifications received, etc. We eventually want the
> asyncio event loop to be what schedules Ceilometer's code in response to
> these events. Right now, it is eventlet doing that.
>
> Now, because we're using eventlet, the code that is run in response to
> these events looks like synchronous code that makes a bunch of
> synchronous calls. For example, the code might do some_sync_op() and
> that will cause a context switch to a different greenthread (within the
> same native thread) where we might handle another I/O event (like a REST
> API request) while we're waiting for some_sync_op() to return:
>
>   def foo(self):
>       result = some_sync_op()  # this may yield to another greenlet
>       return do_stuff(result)
>
> Eventlet's infamous monkey patching is what make this magic happen.
>
> When we switch to asyncio's event loop, all of this code needs to be
> ported to asyncio's explicitly asynchronous approach. We might do:
>
>   @asyncio.coroutine
>   def foo(self):
>       result = yield from some_async_op(...)
>       return do_stuff(result)
>
> or:
>
>   @asyncio.coroutine
>   def foo(self):
>       fut = Future()
>       some_async_op(callback=fut.set_result)
>       ...
>       result = yield from fut
>       return do_stuff(result)
>
> Porting from eventlet's implicit async approach to asyncio's explicit
> async API will be seriously time consuming and we need to be able to do
> it piece-by-piece.
>
> The question then becomes what do we need to do in order to port a
> single oslo.messaging RPC endpoint method in Ceilometer to asyncio's
> explicit async approach?
>
> The plan is:
>
>   - we stick with eventlet; everything gets monkey patched as normal
>
>   - we register the greenio event loop with asyncio - this means that
>     e.g. when you schedule an asyncio coroutine, greenio runs it in a
>     greenlet using eventlet's event loop
>
>   - oslo.messaging will need a new variant of eventlet executor which
>     knows how to dispatch an asyncio coroutine. For example:
>
>         while True:
>             incoming = self.listener.poll()
>             method = dispatcher.get_endpoint_method(incoming)
>             if asyncio.iscoroutinefunc(method):
>                 result = method()
>                 self._greenpool.spawn_n(incoming.reply, result)
>             else:
>                 self._greenpool.spawn_n(method)
>
>     it's important that even with a coroutine endpoint method, we send
>     the reply in a greenthread so that the dispatch greenthread doesn't
>     get blocked if the incoming.reply() call causes a greenlet context
>     switch
>
>   - when all of ceilometer has been ported over to asyncio coroutines,
>     we can stop monkey patching, stop using greenio and switch to the
>     asyncio event loop
>
>   - when we make this change, we'll want a completely native asyncio
>     oslo.messaging executor. Unless the oslo.messaging drivers support
>     asyncio themselves, that executor will probably need a separate
>     native thread to poll for messages and send replies.

We tried to keep eventlet out of the drivers. Does it make sense to do
the same for asyncio?

Does this change have any effect on the WSGI services, and the WSGI
container servers we can use to host them?

> If you're confused, that's normal. We had to take several breaks to get
> even this far because our brains kept getting fried.

I won't claim to understand all of the nuances, but it seems like a
good way to stage the changes. Thanks to everyone involved for working
it out!

>
> HTH,
> Mark.
>
> Victor's excellent docs on asyncio and trollius:
>
>   https://docs.python.org/3/library/asyncio.html
>   http://trollius.readthedocs.org/
>
> Victor's proposed asyncio executor:
>
>   https://review.openstack.org/70948
>
> The case for adopting asyncio in OpenStack:
>
>   https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio
>
> A previous email I wrote about an asyncio executor:
>
>  http://lists.openstack.org/pipermail/openstack-dev/2013-June/009934.html
>
> The mock-up of an asyncio executor I wrote:
>
>   https://github.com/markmc/oslo-incubator/blob/8509b8b/openstack/common/messaging/_executors/impl_tulip.py
>
> My blog post on async I/O and Python:
>
>   http://blogs.gnome.org/markmc/2013/06/04/async-io-and-python/
>
> greenio - greelets support for asyncio:
>
>   https://github.com/1st1/greenio/
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev