[Openstack] [OpenStack][Swift] Fast way of uploading 200GB of 200KB files to Swift

Robert van Leeuwen Robert.vanLeeuwen at spilgames.com
Mon Jan 14 15:21:16 UTC 2013


> Allow me to rephrase.
> I've read somewhere (can't remember where) that it would be faster to upload files if they would be uploaded to separate containeres.
> This was suggested for a standard swift installation with a certain replication factor.
> Since I'll be uploading the files with the replicators turned off, does it really matter if I insert a group of them in separate containeres?

My guess is this concerns the SQLite database load distribution.
So yes, it still matters.

Just to be clear: turning replicators off does not matter at ALL when putting files in a healthy cluster.
Files will be "replicated" / put on all required nodes at the moment the put request is done.
The put request will only give an OK when there is quorum writing the file (the file is stored on more than half of the required object nodes)
The replicator daemons do not have anything to do with this.

Cheers,
Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20130114/c251946c/attachment.html>


More information about the Openstack mailing list