<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">A few months ago at a meetup, I asked DH about that, they said they start a row with computes on the left, ceph on the right, and they meet in the middle.<div><br><div><div>On Aug 19, 2013, at 6:04 AM, Lorin Hochstein <<a href="mailto:lorin@nimbisservices.com">lorin@nimbisservices.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">Abel:<div><br></div><div>I vaguely recall from one of the summits that DreamHost deploys ceph on their compute nodes. Are there any DreamHost folks on this list who could comment? </div><div><br></div><div>
Lorin</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Aug 17, 2013 at 12:33 PM, Abel Lopez <span dir="ltr"><<a href="mailto:alopgeek@gmail.com" target="_blank">alopgeek@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I believe the general consensus for production systems is to not run ceph on compute nodes. Compute nodes should be solely used as instance resources. Plus, compute nodes should be 'failure expectant', you should be able to just pull one out and replace it with a blank box. Adding storage cluster to the mix just complicates maintenance planning, etc. Plus, rule-of-thumb for ceph is 1GHz per OSD, which can be significant depending on the number of disks you're planning on.<br>
<br>
Since you're starting from scratch, I would recommend having your glance utilize the ceph cluster you're planning. You get added benefits by using qcow2 disk images in ceph, as new instances are launched as COW clones.<br>
<br>
As for 'minimal storage' on your compute nodes, I assume that you're intending to have a shared '/var/lib/nova/instances/' directory, as each vm will need a disk file. This has the added benefit of being a prerequisite for vm migration.<br>
<br>
Hope that helps.<br>
<div><div class="h5"><br>
On Aug 16, 2013, at 11:05 PM, Mitch Anderson <<a href="mailto:mitch@metauser.net">mitch@metauser.net</a>> wrote:<br>
<br>
> aHi all,<br>
><br>
> I've been looking around for example architectures for types of sytems and numbers for an HA setup of Openstack Grizzly (Probably won't go live until after havana is released).<br>
><br>
> I've done a Simple Non-HA setup with Mirantis' Fuel. Which has worked out well. And they're documented Production HA setup is with 3 Controllers and N compute nodes...<br>
><br>
> If I were to use Ceph for storage I would need a minimum of atleast 3 nodes. I was looking to make my compute nodes have minimal disk space so only my Controllers would have storage (for Glance, DB's, etc..) and the Ceph storage nodes would have the rest. Is this solution preferred? Or, do I run Ceph on the compute nodes? If so, what size nodes should they be then? I'd like to run 40-60 VM's per compute node of varying sizes and needs.<br>
><br>
> Any pointers would be much appreciated!<br>
><br>
> -Mitch Anderson<br>
</div></div>> _______________________________________________<br>
> OpenStack-operators mailing list<br>
> <a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Lorin Hochstein<br><div>Lead Architect - Cloud Services</div><div>Nimbis Services, Inc.</div><div><a href="http://www.nimbisservices.com/" target="_blank">www.nimbisservices.com</a></div>
</div>
</div></div>
</blockquote></div><br></div></body></html>