[Openstack] Swift sharding across containers
Chuck Thier
cthier at gmail.com
Thu Oct 10 18:46:17 UTC 2013
Hi Shri,
On Thu, Oct 10, 2013 at 1:31 PM, Shrinand Javadekar <shrinand at maginatics.com
> wrote:
> Thanks for the inputs Chuck. Please see my responses inline.
>
>
> On Thu, Oct 10, 2013 at 7:56 AM, Chuck Thier <cthier at gmail.com> wrote:
>
>> Hi Shri,
>>
>> I think your observations are fairly spot on. Here are a couple of
>> thoughts/comments.
>>
>> 1. I wonder if you are maxing out how much your client can push at 128
>> threads. If you were to increase the number threads (or number of clients)
>> for the higher container counts, you could get more transactions through.
>>
>
> [SJ] In this experiment I simply wanted to see how sharding across
> containers helps gives the same input data rate. I will run some numbers
> with higher number of threads to see what's the max number of operations
> per second I can get.
>
Cool
>
>
>>
>> 2. Cloudfiles rate limits PUTs at 100 per second to a single container.
>> This helps ensure fairly consistent performance to a single container. We
>> also put our container data on SSD drives to help drive better performance.
>> So your max theoretical performance is 100xNUM_CONTAINERS PUTs/sec.
>>
>
> [SJ] Great to know about the SSDs and the rate limits. Is it also possible
> to know what version of Swift has been deployed at Rackspace Cloudfiles?
>
We stay pretty close to master. I'm working on a different project at
Rackspace now, so I don't know what exact version we are running right now.
>
>
>> 3. It would be worthwhile to test even larger containers to test how
>> much container size affects performance. I don't think your sample size is
>> large enough.
>>
>
> [SJ] Yeah, especially with SSDs, this is definitely not a large enough
> sample. I guess, I should start at 1M and go upto 10M or so.
>
Or even 100M :)
> Will keep you'll posted.
>
> -Shri
>
Cool, I look forward to it :)
--
Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131010/9c105a6f/attachment.html>
More information about the Openstack
mailing list