Hello, Thanks Itamar for your previous email and thanks for the useful information. FYI I just submitted a first draft (still WIP) of the related community goal proposal, it can be found here https://review.opendev.org/c/openstack/governance/+/902585. Thanks also Sean and John for the discussion about the aiohub. I included this part of the solution into the proposal. However I think we need more details about your vision. That would help to draft the specs of this solution. I think we could start to centralize this topic somewhere, so, I created a dedicated section into the wiki page I created to track the progress of this community goal. As you seem interested in this topic, do you mind giving us more details and thoughts about the aiohub and about how you imagine its design and its implementation. Do not hesitate to fullfile it with your observations and your discoveries. I think this kind of discussion can be run in parallel with the global goal proposal. I'll continue to work on the proposal in the coming days, and I'll remove the WIP flag once I think it has reached a good state, then we will be able to follow the normal process and the acceptance, or not, of this proposal. Thanks for your time. Le jeu. 30 nov. 2023 à 20:38, John Eckersberg <jeckersb@redhat.com> a écrit :
## A potential path for migrating off of eventlet
Like many networking frameworks, eventlet has pluggable event loops, in
itamar@pythonspeed.com writes: this case called a "hub". Typically these wrap things like select() and epoll(), but there also used to be a hub that ran on Twisted.
My hypothesis is that it should be possible to:
1. Create a hub that runs on asyncio. At this point asyncio and eventlet code can run in the same event
loop.
2. Create a little bit of glue so that eventlet code can wait for the result of a coroutine object (i.e. the result of calling an `async def` function). 3. Make `eventlet.greenlet.GreenThread` something you can `await` in an `async def` function.
The end result is that asyncio code can call eventlet code, and eventlet code can call asyncio code. There is an existing integration on this model for gevent, as reference: https://pypi.org/project/asyncio-gevent/
If this works, it would allow migrating from eventlet to asyncio in a gradual manner both within and across projects:
1. Within an OpenStack project, code could be gradually switched to async functions and asyncio libraries. 2. When an OpenStack library fully migrates to asyncio, it will still be usable by anything that is still running on eventlet.
Thanks for taking the time to write this part out in detail. I had an off-list conversation with Herve recently and I was starting to flesh out something quite similar to what you've proposed here. I'm not terribly familiar (yet) with all of the finer details surrounding this so this really helps to point me in the right direction and is motivating that you are thinking along the same lines.
I think it's going to be 100% necessary to run both asyncio and eventlet code simultaneously within OpenStack services for some time. Trying to maintain both in parallel and then hard-cut over to asyncio at some point sounds like an absolute nightmare and effectively impossible to pull off. We'll have to do it bit by bit over a rather extended timeframe, and even then I suspect there is untold pain ahead.
eck
-- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/