[Openstack] Swift sharding across containers

Chuck Thier cthier at gmail.com
Thu Oct 10 18:46:17 UTC 2013


Hi Shri,

On Thu, Oct 10, 2013 at 1:31 PM, Shrinand Javadekar <shrinand at maginatics.com
> wrote:

> Thanks for the inputs Chuck. Please see my responses inline.
>
>
> On Thu, Oct 10, 2013 at 7:56 AM, Chuck Thier <cthier at gmail.com> wrote:
>
>> Hi Shri,
>>
>> I think your observations are fairly spot on.  Here are a couple of
>> thoughts/comments.
>>
>> 1.  I wonder if you are maxing out how much your client can push at 128
>> threads.  If you were to increase the number threads (or number of clients)
>> for the higher container counts, you could get more transactions through.
>>
>
> [SJ] In this experiment I simply wanted to see how sharding across
> containers helps gives the same input data rate. I will run some numbers
> with higher number of threads to see what's the max number of operations
> per second I can get.
>

Cool


>
>
>>
>> 2.  Cloudfiles rate limits PUTs at 100 per second to a single container.
>>  This helps ensure fairly consistent performance to a single container.  We
>> also put our container data on SSD drives to help drive better performance.
>>  So your max theoretical performance is 100xNUM_CONTAINERS PUTs/sec.
>>
>
> [SJ] Great to know about the SSDs and the rate limits. Is it also possible
> to know what version of Swift has been deployed at Rackspace Cloudfiles?
>

We stay pretty close to master.  I'm working on a different project at
Rackspace now, so I don't know what exact version we are running right now.


>
>
>> 3.  It would be worthwhile to test even larger containers to test how
>> much container size affects performance.  I don't think your sample size is
>> large enough.
>>
>
> [SJ] Yeah, especially with SSDs, this is definitely not a large enough
> sample. I guess, I should start at 1M and go upto 10M or so.
>

Or even 100M :)


> Will keep you'll posted.
>
> -Shri
>

Cool, I look forward to it :)

--
Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131010/9c105a6f/attachment.html>


More information about the Openstack mailing list