<div dir="ltr">I've been meaning to see if there's an existing blueprint for this functionality. <div><br></div><div>Along with Gluster, qemu can launch instances directly in Ceph.<div><br></div><div style>Right now, Nova has hard-coded using flat-files under /var/lib/nova/instances.</div>
</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 25, 2013 at 3:48 AM, Sylvain Bauza <span dir="ltr"><<a href="mailto:sylvain.bauza@digimind.com" target="_blank">sylvain.bauza@digimind.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>GlusterFS support is fully available
since qemu-1.3, which makes possible to direcly address block
devices in a glusterfs namespace.<br>
<a href="http://raobharata.wordpress.com/2012/10/29/qemu-glusterfs-native-integration/" target="_blank">http://raobharata.wordpress.com/2012/10/29/qemu-glusterfs-native-integration/</a><br>
<br>
Unfortunately, I'm not aware of this integration effort in Nova.
It would be great if one could specify GlusterFS as backend for
instances and let Nova use qemu capabilities. Maybe it's already
done or in progress, but I haven't heard any BP about of.<br>
<br>
Benchmarks are very impressive with glusterfs native support in
qemu. I'll try to find out one.<br>
<br>
-Sylvain<br>
<br>
Le 25/04/2013 09:51, Razique Mahroua a écrit :<br>
</div>
<blockquote type="cite"><div><div class="h5">
Oh heck John that is one of a feature :D
<div><br>
</div>
<div>Jacob, that's what I do - I mount glusterFS on
/var/lib/nova/instances and the backend is a distributed
replicated volume (replica 4) that the nodes themselves are part
of.</div>
<div>The only thing you need to make sure is to allocate enough
space - and disable the auto suppression of the unused base
images.</div>
<div>Performance-wise, I came up with good figures - the network
was the bottleneck, not so much the Gluster FS itself. I'll run
soon extra benches (I use iozone3 everytime) on the production
environment that will have the exact same setup). Make sure to
use the last stable version as well :)</div>
<div><br>
</div>
<div>regards,</div>
<div>
<div><br>
<div>
<span><span style="font-weight:normal;font-family:Helvetica"><b style="color:rgb(19,112,138)">Razique Mahroua</b></span><span style="font-weight:normal;font-family:Helvetica;color:rgb(19,112,138)"><b> - </b></span><span style="font-family:Helvetica"><span style="font-weight:normal;font-family:Helvetica"><b style="color:rgb(19,112,138)">Nuage & Co</b></span><span style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;border-spacing:0px;font-size:medium"><span style="border-spacing:0px;text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><span style="border-collapse:separate;font-variant:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;border-spacing:0px">
<div style="font-style:normal;font-size:medium;font-family:Helvetica;font-weight:normal"><font color="#13708a"><a href="mailto:razique.mahroua@gmail.com" target="_blank">razique.mahroua@gmail.com</a></font></div>
<div style="font-style:normal;font-size:medium;font-family:Helvetica"><font color="#13708a">Tel : <a href="tel:%2B33%209%2072%2037%2094%2015" value="+33972379415" target="_blank">+33 9 72 37 94 15</a></font></div>
</span></span></span></span></span><br style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;text-transform:none;font-size:medium;white-space:normal;font-family:Arial;word-spacing:0px">
<span><img src="cid:part2.05040509.02060604@digimind.com" height="125" width="125"></span>
</div>
<br>
<div>
<div>Le 25 avr. 2013 à 02:41, Jacob Godin <<a href="mailto:jacobgodin@gmail.com" target="_blank">jacobgodin@gmail.com</a>>
a écrit :</div>
<br>
<blockquote type="cite">
<div dir="ltr">Hey John,
<div><br>
</div>
<div>Thanks for the info! Have you had anyone simply
using GlusterFS mounted to /var/lib/nova and running
instances that way? I'd be interested in hearing some
results if any.</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Jacob</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Apr 24, 2013 at 4:51
PM, John Mark Walker <span dir="ltr"><<a href="mailto:johnmark@redhat.com" target="_blank">johnmark@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:12pt;font-family:times new roman,new york,times,serif">
<div><br>
</div>
<div><br>
</div>
<hr>
<div>
<blockquote style="padding-left:5px;font-size:12pt;font-style:normal;margin-left:5px;font-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:normal;border-left:2px solid #1010ff">I ended up using Gluster as a
shared storage for instance and Ceph for
Cinder/ Nova-volume and admin storage as well.
<div>
works perfectly!</div>
</blockquote>
<div><br>
</div>
</div>
<div>For the record, this is what I usually
recommend. Whenever someone asks me "Gluster or
Ceph?" I ask them "for which part?" We (being
the Gluster team) devoted almost all of our time
to developing a scale-out NAS solution, which
works well for shared storage for tenants in the
cloud. We didn't really start to work on
GlusterFS for hosting VMs until last year. <br>
</div>
<div><br>
</div>
<div>For the curious among you, feel free to try
out our new KVM/QEMU integration pieces which
are available in GlusterFS 3.4, which is
currently in alpha with a beta coming soon:</div>
<div><a href="http://download.gluster.org/pub/gluster/glusterfs/qa-releases/" target="_blank">http://download.gluster.org/pub/gluster/glusterfs/qa-releases/</a>
- NOTE: it's in alpha! It will probably break!
Do not run in production!<br>
</div>
<div><br>
</div>
<div>I'd be very curious to hear how this works
out for openstack users, particularly given our
recent Cinder integration. Note that this
requires QEMU >= 1.3 to utilize the new
integration bits. To read a bit more about it,
along with some links to howtos, see this blog
post: </div>
<div><br>
</div>
<div> <a href="http://www.gluster.org/2012/11/integration-with-kvmqemu/" target="_blank">http://www.gluster.org/2012/11/integration-with-kvmqemu/</a></div>
<div><br>
</div>
<div>Again - this is alpha code, so I can't
recommend using it for any production system,
anywhere. But if you're interested in putting it
through its paces, I'd love to hear how it goes.<br>
</div>
<div><br>
</div>
<div>Happy hacking,<br>
</div>
<div>John Mark Walker<br>
</div>
<div>Gluster Community Lead<br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><div class="im"><pre>_______________________________________________
OpenStack-operators mailing list
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a>
</pre>
</div></blockquote>
<br>
</div>
<br>_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Joe Topjian<div>Systems Administrator</div><div>Cybera Inc.</div><div><br></div><div><a href="http://www.cybera.ca" target="_blank">www.cybera.ca</a></div>
<div><br></div><div><font color="#666666"><span>Cybera</span><span> is a not-for-profit organization that works to spur and support innovation, for the economic benefit of Alberta, through the use of cyberinfrastructure.</span></font></div>
</div>