Open Stack

correct. a single container is replicated like other data in the system (typically 3x). This means that a single container is on only 3 spindles, and an nymber of concurrent writes to objects in that container will attempt to update the container listing (with graceful failure handling). This means that under significant concurrency, the concurrent object write speed is limited by the time it takes to update one of those container replicas.

There are two easy "fixes" for this: (1) shard your data across containers and (2) use faster, dedicated drives for the containers (eg SSDs).

The hard fix for this is to implement container sharding within swift, but this is a hard problem to solve (although nobody would be opposed to a successful solution).

--John





On Dec 4, 2013, at 3:01 PM, Stephen Wood <smwood4 at gmail.com> wrote:

> Can someone explain to me (or point me to some good literature) about why sharding across containers is such a big deal in terms of performance? Is it that a single container is typically localized across a small number of shards?
> 
> -- 
> Stephen Wood
> www.heystephenwood.com
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131204/8fa8e6f1/attachment.sig>