<div dir="ltr">Concurrency is hard, let's blame the tools!<div><br></div><div>Any lib that we use in python is going to have a set of trade-offs. Looking at a couple of the options on the table:</div><div><br></div><div>
1. Threads: Great! code doesn't have to change too much, but now that code *will* be preempted at any time, so now we have to worry about locking and we have even more race conditions that are difficult to debug.</div>
<div><br></div><div>2. Asyncio: Explicit FTW! Except now that big list of dependencies has to also support the same form of explicit concurrency. This is a trade-off that twisted makes as well. Any library that might block has to have a separate library made for it.</div>
<div><br></div><div>We could dig deeper, but hopefully you see what I mean. Changing tools may solve one problem, but at the same time introduce a different set of problems.</div><div><br></div><div>I think the biggest issue with using Eventlet is that developers want to treat it like magic, and you can't do that. If you are monkey patching the world, then you are doing it wrong. How about we take a moment to learn how to use the tools we have effectively, rather than just blaming them. Many projects have managed to use Eventlet effectively (including some in Openstack).</div>
<div><br></div><div>Eventlet isn't perfect, but it has gotten us quite a ways. If you do choose to use another library, please make sure you are trading for the right set of problems.</div><div><br></div><div>--</div>
<div>Chuck</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Feb 7, 2014 at 1:07 AM, Joshua Harlow <span dir="ltr"><<a href="mailto:harlowja@yahoo-inc.com" target="_blank">harlowja@yahoo-inc.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">
<div>
<div><span style="background-color:rgba(255,255,255,0)">+1<br>
<br>
To give an example as to why eventlet implicit monkey patch the world isn't especially great (although it's what we are currently using throughout openstack).<br>
<br>
The way I think about how it works is to think about what libraries that a single piece of code calls and how it is very hard to predict whether that code will trigger a implicit switch (conceptually similar to a context switch).<br>
<br>
Let's take a simple naive piece of code.<br>
<br>
>>> import logging<br>
>>> LOG = logging.getLogger(__name__)<br>
>>> LOG.info("hi")<br>
<br>
This seems rather straightforward, write 'hi' to some log location. With eventlets implicitness (via ye-old monkey patch everything) it is entirely possible that somewhere inside the logging code there will be a write to a socket (say perhaps this person enabled
syslog/socket logger or something like that) and that will block. This causes an implicit switch to another greenthread (and so-on for the applications life-cycle). Now just magnify the amount of understanding required to reason about how the logging library
(which is pretty well understood) works with eventlet by the number of libraries in <a href="https://github.com/openstack/requirements/blob/master/global-requirements.txt" target="_blank">https://github.com/openstack/requirements/blob/master/global-requirements.txt</a>. To
understand how all these libraries interact with I/O, threading or other locations where things can implicitly switch is pretty much near impossible. It becomes even more 'hairy' when those libraries themselves acquire some type of locks (did you as an eventlet
user remember to monkey patch the threading module?)...<br>
<br>
IMHO eventlet has 'seduced' many developers into thinking that it magically makes an application C10K ready even though it easily makes it possible to 'crash and burn' without to much trouble. Is the benefit worth it? Maybe, maybe not...<br>
<br>
I'm not saying we should abandon eventlet (likely we can't easily pull this off even if we wanted to), but I do agree that the randomness it provides is not easy to follow, debug, analyze... It gets even more complicated when you start to mix threads (which
do exist in python, but are GIL handicapped, although this has been getting better in 3.2+ with GIL improvements) with greenthreads (try figuring out which one is causing race conditions in a gdb session for example).<br>
<br>
Anyways, the future of this whole situation looks bright, it will be an interesting balance between making it easy to read/understand (eventlet tries to make it look so easy and no-changes needed, see above seduction) vs. requiring a "big" mind-set change in
how libraries and applications are currently written.<br>
<br>
Which is the right path to get to the final destination, only time will tell :-)<br>
<br>
</span></div>
<span style="background-color:rgba(255,255,255,0)">-Josh</span><div class=""><br>
<br>
<span>Sent from my really tiny device...</span></div></div><div><div class="h5">
<div><br>
On Feb 6, 2014, at 6:55 PM, "Zane Bitter" <<a href="mailto:zbitter@redhat.com" target="_blank">zbitter@redhat.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div><span>On 04/02/14 13:53, Kevin Conway wrote:</span><br>
<blockquote type="cite"><span>On 2/4/14 12:07 PM, "victor stinner"<<a href="mailto:victor.stinner@enovance.com" target="_blank">victor.stinner@enovance.com</a>> wrote:</span><br>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span>>The purpose of replacing eventlet with asyncio is to get a well defined</span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span>>control flow, no more surprising task switching at random points.</span><br>
</blockquote>
</blockquote>
<blockquote type="cite"><span></span><br>
</blockquote>
<blockquote type="cite"><span>I disagree with this. Eventlet and gevent yield the execution context</span><br>
</blockquote>
<blockquote type="cite"><span>anytime an IO call is made or the 'sleep()' function is called explicitly.</span><br>
</blockquote>
<blockquote type="cite"><span>The order in which greenthreads grain execution context is deterministic</span><br>
</blockquote>
<blockquote type="cite"><span>even if not entirely obvious. There is no context switching at random.</span><br>
</blockquote>
<span></span><br>
<span>This is technically correct of course, but in reality there's no way to know whether a particular piece of code is safe from context switches unless you have the entire codebase of the program and all of its libraries in your head at the same time. So
no, it's not *random*, but it might as well be. And it's certainly not explicit in the way that asyncio is explicit; it's very much implicit in other operations.</span><br>
<span></span><br>
<blockquote type="cite"><span>What's more is it shouldn't matter when the context switch happens. When</span><br>
</blockquote>
<blockquote type="cite"><span>writing green threaded code you just pretend you have real threads and</span><br>
</blockquote>
<blockquote type="cite"><span>understand that things happen in an order other than A => B => C.</span><br>
</blockquote>
<span></span><br>
<span>If you like pretending you have real threads, you could just use Python threads. Greenthreads exist because people don't want to deal with actual pretend threads.</span><br>
<span></span><br>
<blockquote type="cite"><span>One of the great benefits of using a green thread abstraction, like</span><br>
</blockquote>
<blockquote type="cite"><span>eventlet or gevent, is that it lets you write normal Python code and slap</span><br>
</blockquote>
<blockquote type="cite"><span>your concurrency management over the top.</span><br>
</blockquote>
<span></span><br>
<span>Right, it lets you do that and neglects to mention that it doesn't actually work.</span><br>
<span></span><br>
<span>The whole premise of eventlet is that it allows you to write your code without thinking about thread safety, except that you *do* still have to think about thread safety. So the whole reason for its existence is to write cheques that it can't cash. It's
conceptually unsound at the most fundamental level.</span><br>
<span></span><br>
<span>I'm not suggesting for a second that it would be an easy change - I'm not even sure it would be a good change - but let's not kid ourselves that everything is fine here in happy-land and there's nothing to discuss.</span><br>
<span></span><br>
<span>cheers,</span><br>
<span>Zane.</span><br>
<span></span><br>
<span>_______________________________________________</span><br>
<span>OpenStack-dev mailing list</span><br>
<span><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a></span><br>
<span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span><br>
</div>
</blockquote>
</div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>