We had identified a problem of running MapReduce on top of Swift mainly on the objects that cannot be renamed without a data copy. Also, there are some significant impact on job completion time for large input data and latency sensitive jobs. You can refer the following research paper which talks about performance study on swift & Hadoop. http://sc14.supercomputing.org/sites/all/themes/sc14/files/archive/tech_poster/poster_files/post192s2-file2.pdf > On Mar 23, 2015, at 2:17 PM, Dariusz Chrząścik <dariusz at chrzascik.com> wrote: > > Hello, > > In my Sahara deployment, I am considering using a swift as an input/output data store. However, I am wondering if swift is eligible for big data processing. Does anyone have some experiences with such configuration? Is it efficient? Can you possibly point me to any articles, reports that compare hdfs performance with swift when running Hadoop Jobs over it? I have done some research in that matter but without success. I'd be grateful for any piece of advice. > > Regards, > Dariusz Chrząścik > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack at lists.openstack.org > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150324/75119750/attachment.html>