[Openstack] Swift sharding across containers
cthier at gmail.com
Thu Oct 10 18:46:17 UTC 2013
On Thu, Oct 10, 2013 at 1:31 PM, Shrinand Javadekar <shrinand at maginatics.com
> Thanks for the inputs Chuck. Please see my responses inline.
> On Thu, Oct 10, 2013 at 7:56 AM, Chuck Thier <cthier at gmail.com> wrote:
>> Hi Shri,
>> I think your observations are fairly spot on. Here are a couple of
>> 1. I wonder if you are maxing out how much your client can push at 128
>> threads. If you were to increase the number threads (or number of clients)
>> for the higher container counts, you could get more transactions through.
> [SJ] In this experiment I simply wanted to see how sharding across
> containers helps gives the same input data rate. I will run some numbers
> with higher number of threads to see what's the max number of operations
> per second I can get.
>> 2. Cloudfiles rate limits PUTs at 100 per second to a single container.
>> This helps ensure fairly consistent performance to a single container. We
>> also put our container data on SSD drives to help drive better performance.
>> So your max theoretical performance is 100xNUM_CONTAINERS PUTs/sec.
> [SJ] Great to know about the SSDs and the rate limits. Is it also possible
> to know what version of Swift has been deployed at Rackspace Cloudfiles?
We stay pretty close to master. I'm working on a different project at
Rackspace now, so I don't know what exact version we are running right now.
>> 3. It would be worthwhile to test even larger containers to test how
>> much container size affects performance. I don't think your sample size is
>> large enough.
> [SJ] Yeah, especially with SSDs, this is definitely not a large enough
> sample. I guess, I should start at 1M and go upto 10M or so.
Or even 100M :)
> Will keep you'll posted.
Cool, I look forward to it :)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Openstack