Open Stack

On 24 November 2014 at 11:01, victor stinner
<victor.stinner at enovance.com> wrote:
> Hi,
>
> I'm happy to announce you that I just finished the last piece of the puzzle to add support for trollius coroutines in Oslo Messaging! See my two changes:
>
> * Add a new aiogreen executor:
>   https://review.openstack.org/#/c/136653/
> * Add an optional executor callback to dispatcher:
>   https://review.openstack.org/#/c/136652/
>
> Related projects:
>
> * asyncio is an event loop which is now part of Python 3.4:
>   http://docs.python.org/dev/library/asyncio.html
> * trollius is the port of the new asyncio module to Python 2:
>   http://trollius.readthedocs.org/
> * aiogreen implements the asyncio API on top of eventlet:
>   http://aiogreen.readthedocs.org/
>
> For the long story and the full history of my work on asyncio in OpenStack since one year, read:
> http://aiogreen.readthedocs.org/openstack.html
>
> The last piece of the puzzle is the new aiogreen project that I released a few days ago. aiogreen is well integrated and fully compatible with eventlet, it can be used in OpenStack without having to modify code. It is almost fully based on trollius, it just has a small glue to reuse eventlet event loop (get read/write notifications of file descriptors).
>
> In the past, I tried to use the greenio project, which also implements the asyncio API, but it didn't fit well with eventlet. That's why I wrote a new project.
>
> Supporting trollius coroutines in Oslo Messaging is just the first part of the global project. Here is my full plan to replace eventlet with asyncio.

...

So - the technical bits of the plan sound fine.

On WSGI - if we're in an asyncio world, I don't think WSGI has any
relevance today - it has no async programming model. While is has
incremental apis and supports generators, thats not close enough to
the same thing: so we're going to have to port our glue code to
whatever container we end up with. As you know I'm pushing on a revamp
of WSGI right now, and I'd be delighted to help put together a
WSGI-for-asyncio PEP, but I think its best thought of as a separate
thing to WSGI per se. It might be a profile of WSGI2 though, since
there is quite some interest in truely async models.

However I've a bigger picture concern. OpenStack only relatively
recently switched away from an explicit async model (Twisted) to
eventlet.

I'm worried that this is switching back to something we switched away
from (in that Twisted and asyncio have much more in common than either
Twisted and eventlet w/magic, or asyncio and eventlet w/magic).

If Twisted was unacceptable to the community, what makes asyncio
acceptable? [Note, I don't really understand why Twisted was moved
away from, since our problem domain is such a great fit for reactor
style programming - lots of networking, lots of calling of processes
that may take some time to complete their work, and occasional DB
calls [which are equally problematic in eventlet and in
asyncio/Twisted]. So I'm not arguing against the move, I'm just
concerned that doing it without addressing whatever the underlying
thing was, will fail - and I'm also concerned that it will surprise
folk - since there doesn't seem to be a cross project blueprint
talking about this fairly fundamental shift in programming model.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologistste
HP Converged Cloud