On Thu, 2024-05-16 at 10:27 +0200, Herve Beraud wrote:
Thanks Jay for this thread and for summarizing the T.C feeling. My following sentences are not directly addressed to you (Jay), but to others T.C members.
Le mar. 14 mai 2024 à 18:28, Jay Faulkner jay@gr-oss.io a écrit :
Hi all,
As you may have read in Goutham's TC summary, the existing eventlet goal[0] is unlikely to gain consensus to merge. This appears to be due to the difference in needs between OpenStack projects, and belief that it's unlikely a single solution would serve them all.
I might agree with the fact it's unlikely a single solution would serve them all. I was going to update some sentence to make this optionality more stronger: https://review.opendev.org/c/openstack/governance/+/902585/comment/bb363bda_...
Also, I was going to propose some adaptations to introduce futurist and executors: https://review.opendev.org/c/openstack/governance/+/902585/comment/6b23377c_...
speaking of executors for nova we have approved a specless bluepirnt to adress some of the low hanging fruit https://blueprints.launchpad.net/nova/+spec/eventlet-removal-part-1
which is implemented here https://review.opendev.org/q/topic:%22eventlet-removal-part-1%22 although i will be revising that at least once more before its ready to merge.
the last two patches in the series remove eventlet.tpool and replace its use with futurist a thread pool executor instead
for our specific usage https://review.opendev.org/c/openstack/nova/+/917962/4/nova/storage/rbd_util... that's a relatively simple transformation.
there are one our too outer uses of eventlet.tpool in nova and i will also adress the usage in the guestfs module but i will likely no touch its usage in the libvirt driver as we have a rather complex code dedicated to how we interact libvirt that need careful thought to untangle.
But, rather than speaking of "a single solution", I'd suggest speaking about "a solution by default".
Indeed, many comments made in this proposal strongly adopt Nova's perspective. Nova is a major piece of Openstack, but Openstack is not only Nova. By updating that sentence (see the previous links), we would make this proposal optional for teams who have the capabilities to design their own alternatives. Nova.
Teams like Nova have resources that other teams may not have.
yes and no. the nova teams capacity is fairly limited these days. we have between 5-10 active contributors depending on how you look at it. refining that by the people that understand our use of eventlet and can review the code and factor in have the time rewrite the relevnet code while being upstream consistently enought to actually make progress that probably means about 3-5 people could actually do this.
at lesat 2 of those people are actively working on finisnhing multi cycle efforts and the rest including myself are heavily engaged in dowstream work or work in other projects that limits our upstream review and development time for at least the next 3-6 months.
Nova is trying to be pragmatic that we do not have enough contributor to rewrite the part of nova that would require it to asyncio without heavily impacting the projects viablity.
so we are looking at using futurist executors instead because its a lot less work to achieve the goal of not using eventlet while not prohibiting us form exploring asyncio or other options eventually if it makes sense.
if we had to replace eventlet with asyncio im not confident we could complete that for nova in the next 12 -18 months. not without effectively stopping all other development. we just don't have the review or code writing capacity to do something that large and complex while actually ensuring we don't introduce regressions if we were doing anything else.
T.C members should consider this difference in resources between teams.
T.C members are elected to adopt a global point of view, and not a team based point of view: https://governance.openstack.org/tc/reference/principles.html#openstack-firs...
Else, without any default optional solution, teams with lower resources will be deprived of any solutions...
Teams should not consider the use of their surplus as legitimate when other teams are deprived of what is necessary.
As a community member, this is the kind of mindset that I expect from the Openstack governance.
Our current situation is:
- Eventlet usage must be discontinued. The temporary maintainers are just
that -- temporary -- and we should not expect eventlet to continue to be a valid path moving forward. 2) We have basic agreement that shared tooling, such as oslo libraries, should not dictate a specific threading model.
If by "should not dictate a specific threading model", you mean that Oslo forced the use of async/await, then I STRONGLY disagree about 2)... Please show me where, in the proposed goal, oslo libraries dictate a specific threading model? => https://review.opendev.org/c/openstack/governance/+/902585
that more a refence ot not forcing async await rather then the internal impelmations. i.e. dont require your users to use asynio
FYI the libs migration is detailed in the "How to migrate a library" section (around line 646 of the proposed goal). And a living example of how the libs would work is available at https://github.com/4383/snippets/blob/main/python/facade/facade.py
that snipit seams to assume its oke to create an asyncio eventlop on the fly in any geiven invocation which is proably going to be too slow for many usecases
i.e. if oslo log did that every time we logged that would be a problem. i have not messuered the startup time of the asynio eventloop but i expect its more expensive then a trival python function call.
if that loop was reused between calls to the lib or even kicked out into its own thread perhaps with a graceful shutdown if noting is run for an extended period of time like privsep, then i can see that variation fo the facade pattern working. we ould have to mussure the overhad of _iter_coroutine
https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_running... does appear to be thread safe so in principal it should not cause any cross thread interactions. that was one of my concerns when lookign at the snipit initally.
To conclude, is it still worth it to spend time updating this proposal again or is your decision definitive?