Open Stack

Hi,

Thanks very much for the sharing.
Really interesting data.

I just have a stupid question:
Your workload is “wordcount”, right ?
Why “rename” would happen ??

Thanks.
-chen

From: Johnson MDevadoss [mailto:johnson.mdevadoss at gmail.com]
Sent: Wednesday, March 25, 2015 6:04 AM
To: Dariusz Chrząścik
Cc: openstack at lists.openstack.org
Subject: Re: [Openstack] [Sahara] Swift performance

We had identified a problem of running MapReduce on top of Swift mainly on the objects that cannot be renamed without a data copy. Also, there are some significant impact on job completion time for large input data and latency sensitive jobs.


You can refer the following research paper which talks about performance study on swift & Hadoop.



http://sc14.supercomputing.org/sites/all/themes/sc14/files/archive/tech_poster/poster_files/post192s2-file2.pdf

On Mar 23, 2015, at 2:17 PM, Dariusz Chrząścik <dariusz at chrzascik.com<mailto:dariusz at chrzascik.com>> wrote:
Hello,

In my Sahara deployment, I am considering using a swift as an input/output data store. However, I am wondering if swift is eligible for big data processing. Does anyone have some experiences with such configuration? Is it efficient? Can you possibly point me to any articles, reports that compare hdfs performance with swift when running Hadoop Jobs over it? I have done some research in that matter but without success. I'd be grateful for any piece of advice.

Regards,
Dariusz Chrząścik

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150325/576b8101/attachment.html>