Open Stack

On Thu, May 7, 2015 at 7:05 PM, Adam Lawson <alawson at aqorn.com> wrote:

> Chuck (and/or others who understand tor have experienced the limits of
> Python)
>
> I found this comment of yours incredibly intriguing: "we are running out
> of incremental improvements that can be made with Python".
>
> Given your work with Swift thus far, what sort of limitations have you
> discovered that had to do specifically with the fact we're using Python? I
> haven't run into real-life limitations specific to a particular language
> before (I usually run into issues with my approach rather than limitations
> with the language itself) so i find this to be a fascinating (and perhaps
> accidental) consideration.
>


Well, Swift is sort of different from provisioning services like most
Openstack projects.  We handle hundreds of times as many requests as big
Nova installations, and the backend servers like this one handle some
multiplier on top of that.  Our users care a lot about performance because
it affects things like their page load times.

Then Python turns out to be kind of a bad choice for writing a
high-performance file server.  It's slow.  Its concurrency model really
punishes you for having workloads that mix disk and network i/o and CPU.
Swift's mix of worker processes, eventlet, and thread pools mostly works
but it's complicated and inefficient.  Blocking disk operations and
CPU-heavy tasks are still prone to either locking up event loops or
thrashing the GIL.

Python 3 and pypy would both make some aspects of that better, but not fix
it (yet).

- Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150507/9bd0a739/attachment.html>