Open Stack

On 9 May 2016, at 13:16, Gregory Haynes wrote:
>
> This is a bit of an aside but I am sure others are wondering the same
> thing - Is there some info (specs/etherpad/ML thread/etc) that has more
> details on the bottleneck you're running in to? Given that the only
> clients of your service are the public facing DNS servers I am now even
> more surprised that you're hitting a python-inherent bottleneck.

In Swift's case, the summary is that it's hard[0] to write a network
service in Python that shuffles data between the network and a block
device (hard drive) and effectively utilizes all of the hardware
available. So far, we've done very well by fork()'ing child processes,
using cooperative concurrency via eventlet, and basic "write more
efficient code" optimizations. However, when it comes down to it,
managing all of the async operations across many cores and many drives
is really hard, and there just isn't a good, efficient interface for
that in Python.

Initial results from a golang reimplementation of the object server in
Python are very positive[1]. We're not proposing to rewrite Swift
entirely in Golang. Specifically, we're looking at improving object
replication time in Swift. This service must discover what data is on
a drive, talk to other servers in the cluster about what they have,
and coordinate any data sync process that's needed.

[0] Hard, not impossible. Of course, given enough time, we can do
 anything in a Turing-complete language, right? But we're not talking
 about possible, we're talking about efficient tools for the job at
 hand.

[1] http://d.not.mn/python_vs_golang_gets.png and
 http://d.not.mn/python_vs_golang_puts.png


--John




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160509/88543570/attachment.pgp>