Open Stack

I want to address some of Chuck’s post, but I think we should come up with a list of requirements. Replies to Chuck inline, and then some requirements below:

On Feb 7, 2014, at 8:38 AM, Chuck Thier <cthier at gmail.com> wrote:

> Concurrency is hard, let's blame the tools!
> 
> Any lib that we use in python is going to have a set of trade-offs.  Looking at a couple of the options on the table:
> 
> 1.  Threads:  Great! code doesn't have to change too much, but now that code *will* be preempted at any time, so now we have to worry about locking and we have even more race conditions that are difficult to debug.

Yes. I mean, as was pointed out earlier in this thread, there are also some gotchas when using eventlet, but there are a lot of cases that you 100% know will not result in a context switch. We’ve been able to avoid locks for this reason. (Although I also feel like if there’s cases where locking would be necessary when using Threads, we should look at how we can re-factor to avoid them. It tends to mean we’re sharing too much globally.)

Besides the locking issue, our current model of creating a million greenthreads would not work well if we simply converted them to Threads. Our processes are already using way too much memory as it is (a separate issue that needs investigation). This becomes even worse if we only support async by using worker processes, as was suggested and commented on earlier in this thread.

> 
> 2.  Asyncio:  Explicit FTW!  Except now that big list of dependencies has to also support the same form of explicit concurrency.  This is a trade-off that twisted makes as well.  Any library that might block has to have a separate library made for it.
> 
> We could dig deeper, but hopefully you see what I mean.  Changing tools may solve one problem, but at the same time introduce a different set of problems.

Yeah, exactly what I was trying to point out last night in my quick reply before bed. :)  This should really be amended to say ‘not monkey-patching’ instead of ‘asyncio’. I realized that as soon as I hit Send last night. An implementation that would monkey patch and use asyncio underneath doesn’t have this issue.

> 
> I think the biggest issue with using Eventlet is that developers want to treat it like magic, and you can't do that.  If you are monkey patching the world, then you are doing it wrong.  How about we take a moment to learn how to use the tools we have effectively, rather than just blaming them.  Many projects have managed to use Eventlet effectively (including some in Openstack).

In general, I agree with the ‘monkey patching the world’ statement. Except that tests are exempt from that argument. ;) But it may be a necessary evil.

> 
> Eventlet isn't perfect, but it has gotten us quite a ways.  If you do choose to use another library, please make sure you are trading for the right set of problems.
> 

Which is what leads me to wanting us to get a list of our requirements before we make any decisions.

1) Socket/fifo/pipe I/O cannot block ‘other work’.
2) Currently executing code that has potential to block for long periods of time need the ability to easily yield for ‘other work’ to be done. This statement is general, but I’m thinking about file I/O here. For example, if a block of code needs to copy a large file, it needs to be able to yield now and then.
3) Semaphores/locks/etc cannot block ‘other work’ that is not trying to acquire the same lock.
4) OS calls such as ‘wait’ or ‘waitpid’ need to not block ‘other work’.
5) The solution needs to perform reasonably well.
6) The solution needs to be reasonably resource efficient.
7) The solution needs to fulfill the above requirements even when using 3rd party modules.
8) Clients and libraries that we produce need to support the above in a way that arbitrary implementations could be used.

I’m debating whether File I/O in #2 should be combined with #1 such that #1 becomes ‘any I/O’. I might only be separating File I/O out by thinking about possible implementations. And I’ve probably missed something.

Anyway, I have opinions on what does and doesn’t satisfy the above, but I’ll reply separately. :)

- Chris