Open Stack

On 06/07/2016 06:09 PM, Samuel Merritt wrote:
> On 6/7/16 12:00 PM, Monty Taylor wrote:
>> [snip]
> >
>> I'd rather see us focus energy on Python3, asyncio and its pluggable
>> event loops. The work in:
>>
>> http://magic.io/blog/uvloop-blazing-fast-python-networking/
>>
>> is a great indication in an actual apples-to-apples comparison of what
>> can be accomplished in python doing IO-bound activities by using modern
>> Python techniques. I think that comparing python2+eventlet to a fresh
>> rewrite in Go isn't 100% of the story. A TON of work has gone in to
>> Python that we're not taking advantage of because we're still supporting
>> Python2. So what I've love to see in the realm of comparative
>> experimentation is to see if the existing Python we already have can be
>> leveraged as we adopt newer and more modern things.
>
> Asyncio, eventlet, and other similar libraries are all very good for
> performing asynchronous IO on sockets and pipes. However, none of them
> help for filesystem IO. That's why Swift needs a golang object server:
> the go runtime will keep some goroutines running even though some
> other goroutines are performing filesystem IO, whereas filesystem IO
> in Python blocks the entire process, asyncio or no asyncio.

That can be modified. gevent has a tool
(http://www.gevent.org/gevent.fileobject.html) that enables the File IO
to be async  as well by putting the file into non-blocking mode. I've
used it, and it works and scales well.

Sadly, Python doesn't offer this by default; perhaps OpenStack can get
that changed.

$0.02

Ben