<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>Hi Razique,<div><br></div><div>Thanks for the link !</div><div>I read the full discussion and as I tought there is no real perfect solution so far.</div><div>I think i'll continue to use nexenta because it's a great solution and i'll set up multi back end storage for cinder in order to test ceph block storage.</div><div>For meta data storage i'll do some test with CephFS because not production ready mean a lot and nothing at the same time. </div><div>I your previsous mail you said <span style="font-family: Helvetica, Arial; font-size: 13px;"> "</span><span style="font-family: Helvetica, Arial; font-size: 13px;">the FS kept hanging on high load, so I considered it to be pretty unstable for OpenStack", but if it was </span><span style="font-family: Helvetica, Arial; font-size: 13px;"> </span><span style="font-family: Helvetica, Arial; font-size: 13px;">kept hanging on high load it should be pretty stable ? what was the load ? Can you share more detail with us ?</span></div><div>That's a pity that we could not find any neutral heavy test out there.</div><div><br></div><div>Thanks,</div><div><br></div><div>Julien<br><br><div><hr id="stopSpelling">Date: Sun, 3 Nov 2013 12:02:41 -0800<br>From: razique.mahroua@gmail.com<br>To: openstack@lists.openstack.org; bada.boum@outlook.com<br>Subject: Re: [Openstack] Storage decision<br><br><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">Hi Julien, </div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">we do discussed that topic many times. With Havana, things are a bit different, but on our previous discussion, we challenged a couple of technologies for the shared storage.</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">It boils down in the end to :</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">- the resources you have</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">- what you are trying to achieve</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;"><br></div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">•Ceph cluster <b>IS</b> production ready, that’s the CephFS which is not, and that’s that FS which is a shared one. In my testing (supported by others), the FS kept hanging on high load, so I considered it to be pretty unstable for OpenStack. </div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">• iSCSI gave me the best performance so far, what you need to do is to create first the iSCSI LUN on you SAN and map is as a block device. Libvirt is able to use that as storage.</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">• NFS was too slow, and I ended up having locks, and a stalled FS</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">• MooseFS will give you good performance, but it’s not adviced to use it for storing and manipulating big images. Make sure to have a solid network backend as well for that cluster :)</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">• GlusterFS is easy to manage, but I only had bad experience with Windows instances (aka big instances :D) the replication process was eating all the cpu and the I/O were very slow. </div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;"><br></div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">Here is a previous topic:</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;"><a href="http://lists.openstack.org/pipermail/openstack-operators/2013-July/003310.html" target="_blank">http://lists.openstack.org/pipermail/openstack-operators/2013-July/003310.html</a></div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;"><br></div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;"><br></div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">regards,</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;">Razique</div><div id="ecxbloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);line-height:auto;"><br></div> <br><p style="color:#A0A0A8;">On November 3, 2013 at 7:17:19, Julien De Freitas (<a href="mailto://bada.boum@outlook.com">bada.boum@outlook.com</a>) wrote:</p> <blockquote class="ecxclean_bq"><span><div class="ecxhmmessage"><div>
<title></title>
<div dir="ltr">
<div><span style="font-size:12pt;">Hi guys,</span></div>
<div>I knows that question has been treated hundreds of times but i
cannot find 1 good answer.</div>
<div>Moreover, since Havana extend support for cinder and gluster
it could be nice to review the question.</div>
<div><br></div>
<div><span style="font-size:12pt;">What I currently use on my
plateform :</span></div>
<div><br></div>
<div>I configured Nexenta to provide NFS and iScsi target :</div>
<div><span style="font-size:12pt;">NFS Instance disk : i mounted a
volume on each compute node and configured the NFS on
nova.conf. </span></div>
<div><span style="font-size:12pt;">Iscsi for cinder back end : I
configured iScsi so when i create a volume it create an iScsi
volume and i'm then able to mount in inside instance.</span></div>
<div><span style="font-size:12pt;">But the problem is that the
replication module for nexenta to get a HA storage system is
expensive and it's not a distributed file system.</span></div>
<div><br></div>
<div>
<div>My goal : store instances ephemeral storage on a performant,
highly available and cheap storage system configured for live
migration :D</div>
<div><br></div>
<div>To achieve this, I studied read about <span style="font-size:12pt;">CephFS and glusterFS.</span></div>
<div><span style="font-size:12pt;">But Ceph is marked has not
ready for production and GlusterFS seems to have some concerned
about performance.</span></div>
</div>
<div><span style="font-size:12pt;"><br></span></div>
<div><span style="font-size:12pt;">What do you think ? Does anyone
have production experiences on GlusterFS or Ceph ?</span></div>
<div><span style="font-size:12pt;"><br></span></div>
<div><span style="font-size:12pt;">Thanks</span></div>
<div><br></div>
</div>
_______________________________________________
<br>Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<br>Post to : openstack@lists.openstack.org
<br>Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<br></div></div></span></blockquote> <div id="ecxbloop_sign_1383508420412806912"><span style="font-family:helvetica,arial;font-size:13px;"></span>-- <br>Razique Mahroua<br><br></div></div></div> </div></body>
</html>