<div dir="ltr"><div>To add insult to injury, the services that would benefit the most (think nova) would require the most work.</div><div>And, since the goal is to have NATS support and not remove eventlet (because there is like no force to do it now), it seems best to focus on porting NATS support to eventlet.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 21 Jul 2023 at 23:31, Tobias Urdin <<a href="mailto:tobias.urdin@binero.com">tobias.urdin@binero.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">



<div>
Hello,
<div><br>
</div>
<div>And just to iterate on this, just look at the code duplication (double the maintenance for an async and sync with eventlet code path).</div>
<div>
<div style="display:block">
<div style="display:inline-block" role="link">
<a style="border-radius:10px;font-family:-apple-system,Helvetica,Arial,sans-serif;display:block;width:300px;overflow:hidden;text-decoration:none" rel="nofollow" href="https://review.opendev.org/q/topic:asyncio-nats" dir="ltr" role="button" width="300" target="_blank">
<table style="table-layout:fixed;border-collapse:collapse;width:300px;background-color:rgb(229,230,233);font-family:-apple-system,Helvetica,Arial,sans-serif" width="300" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td>
<table style="font-family:-apple-system,Helvetica,Arial,sans-serif;table-layout:fixed;background-color:rgb(229,230,233)" width="300" cellspacing="0" cellpadding="0" bgcolor="#E5E6E9">
<tbody>
<tr>
<td style="padding:8px 0px">
<div style="max-width:100%;margin:0px 16px;overflow:hidden">
<div style="font-weight:400;font-size:11px;overflow:hidden;text-overflow:ellipsis;text-align:left">
<a rel="nofollow" href="https://review.opendev.org/q/topic:asyncio-nats" style="text-decoration:none" target="_blank"><font style="color:rgba(0,0,0,0.498)" color="#808080">review.opendev.org</font></a></div>
</div>
</td>
<td style="padding:6px 12px 6px 0px" width="36">
<a rel="nofollow" href="https://review.opendev.org/q/topic:asyncio-nats" target="_blank"></a></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</a></div>
</div>
</div>
<div><br>
</div>
<div>We’d need to duplicate oslo.service, oslo.messaging and all other common libraries being called in a asyncio code path, and not only</div>
<div>that we’d need to consolidate everyone on a Pecan (REST API library) replacement that would support asyncio (a new oslo library?) and</div>
<div>just imagine that migration and consolidation effort for multiple projects.</div>
<div><br>
</div>
<div>Even then we’d also need to port every single project to using it without regressions in performance.</div>
<div><br>
</div>
<div>It’s a crazy massive effort, but I fully agree that eventlet should have been gone long ago.</div>
<div><br>
</div>
<div>Best regards</div>
<div>Tobias</div>
<div>
<div style="display:block"><br>
</div>
<div>
<blockquote type="cite">
<div>On 21 Jul 2023, at 16:16, <a href="mailto:smooney@redhat.com" target="_blank">smooney@redhat.com</a> wrote:</div>
<br>
<div><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">On
 Fri, 2023-07-21 at 15:40 +0200, Thomas Goirand wrote:</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<blockquote type="cite" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
On 7/21/23 13:32, <a href="mailto:smooney@redhat.com" target="_blank">smooney@redhat.com</a> wrote:<br>
<blockquote type="cite">On Fri, 2023-07-21 at 11:59 +0200, Radosław Piliszek wrote:<br>
<blockquote type="cite">On Fri, 21 Jul 2023 at 09:03, Thomas Goirand <<a href="mailto:zigo@debian.org" target="_blank">zigo@debian.org</a>> wrote:<br>
<blockquote type="cite"><br>
On 7/15/23 16:27, Tobias Urdin wrote:<br>
<blockquote type="cite">The effort is mostly now to get/find a supported third-party Python library that does not<br>
use asyncio for usage in a oslo.messaging driver.<br>
</blockquote>
<br>
Sorry to ask, but what's the problem with asyncio? Do we want something<br>
*not* async? If yes, why?<br>
</blockquote>
<br>
Large swaths of OpenStack code are not ready for async.<br>
We want to take baby steps and get to some better place with tangible benefits.<br>
</blockquote>
that not entirly correct.<br>
much of openstack is already async but we made a decision ot use eventlet as our async model many many<br>
years ago after we ported form twisted.<br>
<br>
The problem is you cannot really mix eventlet code and asyncio safely in the python process.<br>
That means you cannot have the async nats lib used in nova,swift,cinder,neutron, glance and progrably other<br>
projects today.<br>
<br>
i proposed one option a long time ago which is to take a privsep like approch where we spawn the oslo messaging<br>
driver<br>
into a sperate process and comunicate with it over a unix socket.<br>
</blockquote>
<br>
IRRC, back in 2014, at the Hongkong summit, there was a large consensus<span> </span><br>
that Eventlet was a wrong choice. I can still remember Jay Pipes<span> </span><br>
shouting "die Eventlet, die...". Fast forward nearly 10 years later,<span> </span><br>
nothing has moved in that direction, and we're still stuck with that<span> </span><br>
horrible hack. Why don't we just get rid of Eventlet and switch to<span> </span><br>
asyncio instead? That'd be a *HUGE* improvement in terms of ... well<span> </span><br>
many things ... less bugs, more maintainability, less breakage on each<span> </span><br>
minor Python release, etc. That'd be a lot of work, I'm sure... At the<span> </span><br>
same time, everyone would be happy to finally get rid of it.<br>
</blockquote>
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">we
 have consider that in the past and the reality is it would be a massive rewrite</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">we
 considerd doing this when we were dicussiong moving to python3 and at various points since</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">the
 problem is two fold. 1 we dont have contibutors able to work on this currently. 2 we dont have</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">a
 good solution for how to do this incrementally.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">in
 nova at least we did breicly consider tyring to add asyncio console-script entries points</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">so
 that we couuld have an eventlet version and non eventlet version of each service to act as a bridge</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">its
 somethign we could discuss again but makign such a change without introducing bugs or performance regressions</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">would
 require a lot of careful engenieing work.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">i
 would love to regarin the ablity ot run nova under a debugger and drop the dep on eventlet but i dont want</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">to
 do that at the expense of stablity or perfromace for our customer.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">some
 services liek the api and schduler would be relitivly simple to port. the compute agent and conductor on the other</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">hand
 rely on the implict acync behavior to fucntion at all. nova-comptue with libvirt as at lest 3 while true loops<span> </span></span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">to
 compunticate with rabbit, libvirt and privsep to name just one of the the issues tha towuld need to be resolved.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<blockquote type="cite" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<br>
Cheers,<br>
<br>
Thomas Goirand (zigo)</blockquote>
</div>
</blockquote>
</div>
<br>
</div>
</div>

</blockquote></div>