[Openstack-operators] Swift for high throughput

Kevin Sweeney kevin at yieldex.com
Mon Dec 19 18:44:06 UTC 2011


Hi,

I'm testing out swift for an application that requires high bandwidth
access to data on the system. My test environment is following the
Openstack documentation for a multi-node cluster and my proxy/auth node has
two 10g interfaces (one to the "public" and one to the private storage
network).

Two questions:
1. I'm getting a lot of chunk read timeouts when streaming larger groups of
files (50 500MB files). This doesn't occur when I do smaller groups of the
same files (10-25). I've increased my timeouts to 60s, which reduces the
number of failed reads but this also increases the time to retrieve the
data by quite a bit. I don't want to keep increasing the timeout, 60s seems
too long anyway. Has anyone else experienced this?

2. In general, does anyone have advice on using Swift for this purpose? I'm
starting to get the feeling this product is more for small random file
retrieval vs. pulling large batches at a time.

thanks in advance,
Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20111219/01d9e86c/attachment-0002.html>


More information about the Openstack-operators mailing list