<div dir="ltr"><font color="#cc0000"><b>REF :</b> <a href="http://www.quora.com/What-features-differentiate-HDFS-and-OpenStack-Object-Storage">http://www.quora.com/What-features-differentiate-HDFS-and-OpenStack-Object-Storage</a></font> by <font color="#990000">ChunkThier</font><div>
<br></div><div><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">While there are some similarities between HDFS and Openstack Object Storage (Swift), the overall design of the systems are very different.</span><br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">
<br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">1. HDFS uses a central system to maintain file metadata (Namenode), where as in Swift the metadata is distributed and replicated across the cluster. Having a central meta-data system is a single point of failure for HDFS, and makes it more difficult to scale to very large sizes.</span><br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">
<br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">2. Swift is designed with multi-tenancy in mind, where HDFS has no notion of multi-tenancy</span><br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">
<br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">3. HDFS is optimized for larger files (as is typical for processing data), where Swift is designed to store any sized files.</span><br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">
<br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">4. Files in HDFS are write once, and can only have one writer at a time, in Swift files can be written many times, and under concurrency, the last write wins.</span><br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">
<br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">5. HDFS is written in Java, where Swift is written in Python</span><br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">
<br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">TLDR: HDFS is designed to store a medium number of larges files to support data processing, where Swift is designed as a more generic storage solution to reliably store very large numbers of varying sized files.</span><br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">
<br style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">(HDFS Architecture information attained from </span><span class="" style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><a href="http://hadoop.apache.org/hdfs/docs/current/hdfs_design.html" rel="nofollow" class="" target="_blank" style="margin:0px;padding:0px 12px 0px 0px;text-decoration:none;color:rgb(25,85,141);background-image:url(http://d1vgw4v7ja2ido.cloudfront.net/-d00b84133c0b47df.gif);background-repeat:no-repeat no-repeat">http://hadoop.apache.org/hdfs/do...</a></span><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">)</span><br>
</div><div><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px"><br></span></div><div style><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Arial,sans-serif;font-size:13px;line-height:17.90625px">Hope it help</span></div>
<div class="gmail_extra"><br clear="all"><div><div dir="ltr"><div>+Hugo Kuo+</div><div><a href="mailto:hugo@swiftstack.com" target="_blank">hugo@swiftstack.com</a><br></div><div><a href="mailto:tonytkdk@gmail.com" target="_blank">tonytkdk@gmail.com<br>
</a></div><div>+886 935004793<br></div></div></div>
<br><br><div class="gmail_quote">2013/7/3 Li, Leon <span dir="ltr"><<a href="mailto:Leon.Li2@emc.com" target="_blank">Leon.Li2@emc.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi,<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I have googled and found some answers but they are not to the point I think. Why foundation choice object storage other than scale-out NAS?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I see some points about benefit of object storage(swift)<u></u><u></u></span></p><p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Storing billions of files.<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Storing Petabytes (millions of Gigabytes) of data.<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Use cheap servers<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Can have several copy for each file<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">However a scale-out NAS could also have these benefits, if you build the scale-out NAS with open source cluster FS(for example HDFS), just like many Internet company did.<span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span></span></p>
<span class="HOEnZb"><font color="#888888"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Leon<u></u><u></u></span></p>
</font></span></div></div><br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div></div>