The current OpenStack paradigm seems to be built around external storage, which contains user data on attached volumes. However, we wanted to create distributed storage on the same nodes we are running nova-compute on.<br>
<br><div class="gmail_quote">2011/5/26 Peter J. Pouliot <span dir="ltr"><<a href="mailto:ppouliot@novell.com">ppouliot@novell.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Greetings Programs,<br>
<br>
We to have been toying with a similar idea in our lab. We are using the same model as Oleg, for existing clouds. The current OpenStack paradigm is a bit different. Having not read all his info yet, I hope they include service resources for the openstacks bits configured into his CIB.<br>
<br>
We have been toying with the idea, of doing linux-ha clusters under the openstack services for service availability across the cloud.<br>
<br>
p<br>
<div class="im"><br>
<br>
<br>
<br>
On Thu, May 26, 2011 at 03:11:34PM +0200, Diego Parrilla Santamaría wrote:<br>
> Hi Oleg,<br>
> thank you very much for your post, it's really didactic. We are taking a<br>
> different approach for HA at storage level, but I have worked formerly<br>
> with DRBD and I think it's a very good choice.<br>
> I'm curious about how you have deployed nova-volume nodes in your<br>
> architecture. You don't specify if the two nodes of the DRBD cluster run<br>
> one or two instances of nova-volume. If you run one instance probably you<br>
> have implemented some kind of fault-tolerant active-passive service if the<br>
> nova-volume process fails in the active node, but I would like to know if<br>
> you can run an active-active two nova-volume instances on two different<br>
> physical nodes on top of the DRBD shared resource.<br>
> Regards<br>
> Diego�<br>
> --<br>
> Diego Parrilla<br>
> CEO<br>
</div>> [1]<a href="http://www.stackops.com" target="_blank">www.stackops.com</a> |� [2]<a href="mailto:diego.parrilla@stackops.com">diego.parrilla@stackops.com</a> | +34 649 94 43 29 |<br>
> skype:diegoparrilla<br>
><br>
> On Thu, May 26, 2011 at 1:29 PM, Oleg Gelbukh <[3]<a href="mailto:ogelbukh@mirantis.com">ogelbukh@mirantis.com</a>><br>
<div class="im">> wrote:<br>
><br>
> Hi,<br>
> We were researching Openstack for our private cloud, and want to share<br>
> experience and get tips from community as we go on.�<br>
> We have settled on DRBD as shared storage platform for our installation.<br>
> LVM is used over the drbd device to mange logical volumes. OCFS2 file<br>
> system is created on one of volumes, mounted and set up as<br>
> image_path�and�instance_path in the nova.conf, other space is reserved<br>
> for storage volumes (managed by nova-volume).�<br>
> As a result, we have shared storage suitable for features such as live<br>
> migration and snapshots. We also have some level of fault-tolerance,<br>
> with DRBD I/O error handling, which automatically redirects I/O requests<br>
> to peer node over network in case of primary node failure. We created<br>
</div>> [4]script for bootstrapping lost VMs in two crash scenarios:<br>
<div class="im">> * dom0 host restart/domU failure: restore VMs on the same host<br>
> * dom0 host failure: restore VMs on peer node<br>
> We are considering such pair of servers with shared storage as a basic<br>
> block for the cloud structure.<br>
</div>> For whom it may interest, the details of DRBD installation are [5]here.<br>
<div class="im">> I'll be glad to answer any questions and highly appreciate feedback on<br>
> this.<br>
> Oleg S. Gelbukh,<br>
> Mirantis Inc.<br>
</div>> [6]<a href="http://www.mirantis.com" target="_blank">www.mirantis.com</a><br>
> _______________________________________________<br>
> Mailing list: [7]<a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> Post to � � : [8]<a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
> Unsubscribe : [9]<a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> More help � : [10]<a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
><br>
> References<br>
><br>
> Visible links<br>
> 1. <a href="http://www.stackops.com/" target="_blank">http://www.stackops.com/</a><br>
> 2. mailto:<a href="mailto:diego.parrilla@stackops.com">diego.parrilla@stackops.com</a><br>
> 3. mailto:<a href="mailto:ogelbukh@mirantis.com">ogelbukh@mirantis.com</a><br>
> 4. <a href="https://github.com/Mirantis/openstack-utils/blob/master/recovery_instance_by_id.py" target="_blank">https://github.com/Mirantis/openstack-utils/blob/master/recovery_instance_by_id.py</a><br>
> 5. <a href="http://mirantis.blogspot.com/2011/05/shared-storage-for-openstack-based-on.html" target="_blank">http://mirantis.blogspot.com/2011/05/shared-storage-for-openstack-based-on.html</a><br>
> 6. <a href="http://www.mirantis.com/" target="_blank">http://www.mirantis.com/</a><br>
> 7. <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> 8. mailto:<a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
> 9. <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> 10. <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<div><div></div><div class="h5"><br>
> _______________________________________________<br>
> Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
> Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br>
</div></div></blockquote></div><br>