At Zadara Storage, we are working on a block storage system for the cloud. We didn't published much info yet but if somebody is interested I will be happy to be in a call and explain what we are doing.<br><br>Nelson Nahum<br>
CTO<br><a href="mailto:nelson@zadarastorage.com">nelson@zadarastorage.com</a><br><br><br><div class="gmail_quote">2011/5/27 Oleg Gelbukh <span dir="ltr"><<a href="mailto:ogelbukh@mirantis.com">ogelbukh@mirantis.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hi<br>Our approach was defined by need to combine storage and compute on the same hosts.<br>Our configuration is dual-primary, so we can run nova-compute and virtual servers on both nodes and have them with write access to volumes. DRBD allows this mode out-of-box now, but it requires clustered file system or great caution when runnning LVM on it.<br>
But nova-volume must run on one node of drbd-connected pair, while the second gets copy of lvm data via drbd. The tricky part is that it seems we must activate volumes and volume groups on the peer node, but automation of this is relatively easy.<br>
For now, we are not going to share volumes outside of drbd-peers pair for live migration or as attachable volumes, except some special cases like migrating VMs between drbd pairs.<div>Looking forward to read couple of words on your approach.</div>
<div>
<br><div class="gmail_quote"><div class="im">2011/5/26 Diego Parrilla Santamarķa <span dir="ltr"><<a href="mailto:diego.parrilla.santamaria@gmail.com" target="_blank">diego.parrilla.santamaria@gmail.com</a>></span><br>
</div><div><div></div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Oleg,<div><br></div><div>thank you very much for your post, it's really didactic. We are taking a different approach for HA at storage level, but I have worked formerly with DRBD and I think it's a very good choice.</div>
<div><br></div><div>I'm curious about how you have deployed nova-volume nodes in your architecture. You don't specify if the two nodes of the DRBD cluster run one or two instances of nova-volume. If you run one instance probably you have implemented some kind of fault-tolerant active-passive service if the nova-volume process fails in the active node, but I would like to know if you can run an active-active two nova-volume instances on two different physical nodes on top of the DRBD shared resource.</div>
<div><br></div><div>Regards</div><div>Diego </div><div><br></div><div>-- <br><span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Times;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><div align="left">
<div><font size="2"><span style="font-family:Arial" lang="ES">Diego Parrilla<br><a href="http://www.stackops.com" title="file:///C:/Documents%20and%20Settings/carolina.capsir.per1/Application%20Data/Microsoft/Signatures/www.garrigues.com
www.garrigues.com" style="color:rgb(7, 77, 143)" target="_blank"><span title="file:///C:/Documents%20and%20Settings/carolina.capsir.per1/Application%20Data/Microsoft/Signatures/www.garrigues.com"></span></a></span></font><font style="font-family:arial,helvetica,sans-serif" size="2" face="Arial"><b>CEO</b><font size="1"><br>
</font></font><span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Times;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"></span></span><b><font style="font-family:arial,helvetica,sans-serif" size="2" face="Arial"><a href="http://www.stackops.com/" target="_blank"><b>www.stackops.com</b></a> | </font></b><font style="font-family:arial,helvetica,sans-serif" size="2" face="Arial"><font size="1"> <a href="mailto:diego.parrilla@stackops.com" target="_blank">diego.parrilla@stackops.com</a></font></font><span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Times;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><font size="2" face="Arial" color="#004438"><b><b><span style="font-size:10pt;color:rgb(0, 68, 56);font-family:Arial" lang="EN-GB"></span></b></b></font></span></span><font style="font-family:arial,helvetica,sans-serif" size="2" face="Arial"><font size="1"> | </font></font><span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Times;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><font style="font-family:arial,helvetica,sans-serif" size="2" face="Arial"><font size="1"><a href="tel:%2B34%20649%2094%2043%2029" value="+34649944329" target="_blank">+34 649 94 43 29</a> | <a>skype:diegoparrilla</a></font></font></span></span><font style="font-family:arial,helvetica,sans-serif" size="2" face="Arial"><a href="http://www.stackops.com" target="_blank"><b><br>
</b></a></font></div></div></span></span></div><div><br>
<br><br><div class="gmail_quote"><div><div></div><div>On Thu, May 26, 2011 at 1:29 PM, Oleg Gelbukh <span dir="ltr"><<a href="mailto:ogelbukh@mirantis.com" target="_blank">ogelbukh@mirantis.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div></div><div>
<div>Hi,</div>We were researching Openstack for our private cloud, and want to share experience and get tips from community as we go on. <div><br></div><div>We have settled on DRBD as shared storage platform for our installation. LVM is used over the drbd device to mange logical volumes. OCFS2 file system is created on one of volumes, mounted and set up as <b>image_path</b> and <b>instance_path</b> in the <i>nova.conf</i>, other space is reserved for storage volumes (managed by nova-volume). <div>
<div><br></div><div>As a result, we have shared storage suitable for features such as live migration and snapshots. We also have some level of fault-tolerance, with DRBD I/O error handling, which automatically redirects I/O requests to peer node over network in case of primary node failure. We created <a href="https://github.com/Mirantis/openstack-utils/blob/master/recovery_instance_by_id.py" target="_blank">script</a> for bootstrapping lost VMs in two crash scenarios:</div>
<div>* dom0 host restart/domU failure: restore VMs on the same host</div><div>* dom0 host failure: restore VMs on peer node<br><div>We are considering such pair of servers with shared storage as a basic block for the cloud structure.</div>
<div><br></div><div>For whom it may interest, the details of DRBD installation are <a href="http://mirantis.blogspot.com/2011/05/shared-storage-for-openstack-based-on.html" target="_blank">here</a>. I'll be glad to answer any questions and highly appreciate feedback on this.</div>
<div><br></div><div>Oleg S. Gelbukh,</div><div>Mirantis Inc.</div><div><a href="http://www.mirantis.com" target="_blank">www.mirantis.com</a></div>
</div></div></div>
<br></div></div>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div>
</blockquote></div></div></div><br>
</div>
<br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br>