Open Stack

Hi,

My patches for Oslo Messaging are naive and inefficient. They are just a first step to prepare OpenStack for Trollius. I chose to run asyncio event loop in its own dedicated thread and dispatch messages in a pool of threads. It would be nice to run asyncio event loop in the main thread, as the last blocker function of the "main()" function of applications. You can already pass your asyncio event loop with my patch, the thread is only created if you don't pass the loop.

I wrote that my patch is inefficient because it calls Olso Messaging listener with a short timeout (ex: 500 ms) in a busy-loop. The timeout is needed to be able to exit the threads. Later, each driver should be modified to give access to the low-level file descriptor, so asyncio can react to "ready to read" and "ready to write" events.

None of my Trollius patches has been merged yet. I have no experience of Trollius cooperating (or not) with eventlet yet.

There are different options:

- run asyncio tasks in an "eventlet event loop"
- run eventlet tasks in an asyncio event loop
- run asyncio and eventlet in two separated "event loops" (eventlet API is not written as an event loop), typically in two threads

The best would be to use asyncio with non-blocking file descriptors, without eventlet, and register these file descriptors in asyncio. But it cannot be done right now, it requires too much changes.

Sorry, I have no concrete code to show you right now.

I stopped working on Trollius in OpenStack because I was asked to wait after Icehouse release.

Victor

> In addition to the installation requirements, we also need to deal with the
> code-level changes. For example, when using the eventlet executor eventlet
> needs to have been imported very early by the application so it can
> monkeypatch the I/O libraries. When not using the eventlet executor, that
> monkeypatching should not be done because it will interfere with the
> regular I/O. So now we have an operation that needs to happen during code
> initialization that is dictated by a configuration option (which executor
> is being used) only available after all of the code initialization has
> happened.
> 
> My first impression is that when we have an executor that works with
> asyncio/trollius we will want to drop eventlet entirely, but that's just a
> first impression. There may be similar trade-offs with the other libraries.
> 
> Victor, what do you think?



> 
> Doug
> 
> 
> On Mon, Mar 17, 2014 at 9:52 PM, Davanum Srinivas <davanum at gmail.com> wrote:
> 
> > Adrian,
> >
> > We are too close to icehouse release candidates to bump up global
> > requirements with new rev of oslo.messaging. So even if we all agree
> > and cut a new rev of oslo.messaging it's too late for icehouse as
> > release candidates are rolling next week.
> >
> > I'd definitely support a way to get python33 support via trollius or
> > thread-exec that Joshua pointed out very soon. It may be a good idea
> > to keep solum's py33 non-voting till we nail down all other other
> > dependencies, so +1 for that as well.
> >
> > -- dims
> >
> > On Mon, Mar 17, 2014 at 7:29 PM, Adrian Otto <adrian.otto at rackspace.com>
> > wrote:
> > > Doug Hellmann and Victror Stinner (+ oslo cores),
> > >
> > > Solum currently depends on a py33 gate. We want to use oslo.messaging,
> > but are worried that in the current state, we will be stuck without py33
> > support. We hope that by adding the Trollius code[1], and getting a new
> > release number, that we can add the oslo.messaging library to our
> > requirements and proceed with our async messaging plan.
> > >
> > > I am seeking guidance from you on when the above might happen. If it's a
> > short time, we may just wait for it. If it's a long time, we may need to
> > relax our py33 gate to non-voting in order to prevent breakage of our
> > Stackforge CI while we work with the oslo.messaging code. We are also
> > considering doing an ugly workaround of creating a bunch of worker
> > processes on the same messaging topic until we can clear this concern.
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > >
> > > Adrian
> > >
> > > [1] https://review.openstack.org/70948
> > > _______________________________________________
> > > OpenStack-dev mailing list
> > > OpenStack-dev at lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> > --
> > Davanum Srinivas :: http://davanum.wordpress.com
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>