[Openstack] Ceph vs swift

Vincenzo Pii piiv at zhaw.ch
Tue Jun 3 13:32:04 UTC 2014


Hi Remo,

We are doing a performance evaluation study on Ceph vs Swift for small
storage clusters.
The results should be published soon, so if the use case is of interest to
you you will have some material to analyze :).

Concerning the partition power, I think this article [1] (which is a bit
old btw) offers a good explanation of what the partitions are in Swift and
how they are used.
The swift deployment guide [2] also provides some indications on how to
determine the number of partitions for the ring.

[1]:
https://julien.danjou.info/blog/2012/openstack-swift-consistency-analysis
[2]:
http://docs.openstack.org/developer/swift/deployment_guide.html#preparing-the-ring

2014-06-02 20:52 GMT+02:00 Chuck Thier <cthier at gmail.com>:

> Hi Remo,
>
> I have heard that there has been some work to integrate Hadoop with Swift,
> but know very little about it.  Integration with MS exchange, but could be
> an interesting use case.
>
> Partitions can be thought of as virtual buckets that objects are assigned
> to.  They are an abstract concept and don't have any inherent size.
>  Buckets then are assigned to storage devices.  The partition power just
> determines how many partitions are distributed across the storage devices.
>
> One of my coworkers wrote the following tool to help calculate the
> partition power: http://rackerlabs.github.io/swift-ppc/
>
> --
> Chuck
>
>
> On Fri, May 30, 2014 at 1:03 AM, Remo Mattei <remo at italy1.com> wrote:
>
>> Thanks Chuck
>> This is great I want to use an object store which allows me to work well
>> with Hadoop and if possible with MS exchange IIS etc. If you do have tips
>> on this that will be great. I also would love your point of partition power
>> in how you describe it.  I have my own ideas on it so I m curious to see
>> how to get an idea on object calculation on a partition. One of the
>> question I have always been asked was if the partition is sizex but the
>> object is sizez which is bigger how is that handle?
>>
>> Thanks for you sharing I m sure everyone will take a good lesson.
>>
>> Ciao
>>
>> Inviato da iPhone ()
>>
>> Il giorno May 29, 2014, alle ore 22:39, Chuck Thier <cthier at gmail.com>
>> ha scritto:
>>
>> Hello Remo,
>>
>> That is quite an open ended question :)  If you could share a bit more
>> about your use case, then it would be easier to provide more detailed
>> information, but I'll try to cover some of the basics.
>>
>> First, a disclaimer.  I am one of the original Openstack Swift
>> developers, so I *may* be a bit biased.
>>
>> Both Swift and Ceph are capable object storage systems.  Swift focuses
>> purely on object storage, while Ceph provides object, block and filesystem
>> storage.  This leads to, what I believe is, the biggest fundamental
>> difference between Swift and Ceph.  Since Ceph also provides block and
>> filesystem storage, it chooses  consistency and partition tolerance over
>> availability.  Swift was designed to be eventually consistent (similar to
>> S3) favoring availability and partition tolerance over consistency.  If you
>> are not familiar with the CAP theorem, then I suggest starting with the
>> Wikipedia article about it [1] (oh and btw, all the images that you see on
>> Wikipedia are stored in a Swift cluster ;))
>>
>> I can't speak directly to the performance characteristics of Ceph, but I
>> gave a talk [2] at the Hong Kong summit where I go into quite a bit of
>> detail about how well swift has performed at Rackspace over the past 5
>> years, and how far we have been able to scale it.  Openstack Swift is also
>> in use by a substantial number of other companies and institutions all over
>> the world.
>>
>> All of that said, you can't trust anything you read on the internet, or
>> even me.  If you are interested in running either Swift or Ceph, then I
>> would recommend evaluating your use case with both systems.
>>
>> --
>> Chuck
>>
>> [1]  http://en.wikipedia.org/wiki/CAP_theorem
>> [2]  https://www.youtube.com/watch?v=YxnFUQHIwNs
>>
>>
>> On Thu, May 29, 2014 at 9:34 PM, Remo Mattei <remo at italy1.com> wrote:
>>
>>> Hi all does anyone have done any testing or comparison between swift and
>>> ceph I want to get some others people prospectives.
>>>
>>> Thanks.
>>>
>>> Inviato da iPhone ()
>>> _______________________________________________
>>> Mailing list:
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>> Post to     : openstack at lists.openstack.org
>>> Unsubscribe :
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>
>>
>> !DSPAM:1,53881b75141238639619343!
>>
>>
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>


-- 
Vincenzo Pii
Researcher, InIT Cloud Computing Lab
Zurich University of Applied Sciences (ZHAW)
http://www.cloudcomp.ch/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140603/47b2edbc/attachment.html>


More information about the Openstack mailing list