[Openstack-operators] Shared storage for live-migration with NFS, Ceph and Lustre
Miguel A Diaz Corchero
miguelangel.diaz at externos.ciemat.es
Tue Jun 23 07:27:22 UTC 2015
After reading CephRDB the impressions were extremely good and even
better than CephFS to ephemeral storage. Are you using qcow2 or raw
type? I prefer qcow2, but in this case we cannot enable the writing
cache in the cluster reducing a bit the performance. I should test the
CephRDB performance of both (qcow2 and raw) before migrating to production.
Thanks for sharing your experience.
El 20/06/15 22:49, Dmitry Borodaenko escribió:
> With Ceph, you'll want to use RBD instead of CephFS, we had OpenStack
> live migration working with Ceph RBD for about a year and a half now,
> here's a PDF slide deck with some details:
> If you take CephFS and the bottlenecks associated with POSIX metadata
> (which you don't need to manage your boot volumes which are just block
> devices) out of the way, the need to partition your storage cluster
> disappears, a single Ceph cluster can serve all 40 nodes.
> It may be tempting to combine compute and storage on the same nodes,
> but there's a gotcha associated with that. Ceph OSD processes may be
> fairly CPU-heavy at high IOPS loads or when rebalancing data after an
> disk dies or a node goes offline, you'd have to figure out a way to
> isolated their CPU usage from that of your workloads. Which is why,
> for example, Fuel allows you to combine ceph-osd and compute roles on
> the same node, but Fuel documentation discourages you from doing so.
> On Wed, Jun 17, 2015 at 2:11 AM Miguel A Diaz Corchero
> <miguelangel.diaz at externos.ciemat.es
> <mailto:miguelangel.diaz at externos.ciemat.es>> wrote:
> Hi friends.
> I'm evaluating different DFS to increase our infrastructure from
> 10 nodes to 40 nodes approximately. One of the bottleneck is the
> shared storage installed to enable the live-migration.
> Well, the selected candidate are NFS, Ceph or Lustre (which is
> already installed for HPC purpose).
> Creating a brief planning and avoiding network connectivities:
> *a)* with NFS and Ceph, I think it is possible but dividing the
> whole infrastructure (40 nodes) in smaller clusters, for instance;
> 10 nodes with 1 storage each one. Obviously, the live-migration is
> only possible between nodes on the same cluster (or zone)
> *b) *with Lustre, my idea is to connect all the nodes (40 nodes)
> to the same lustre (MDS) and use all the concurrency advantages of
> the storage. In this case, the live migration could be possible
> among all the nodes.
> I would like to ask you for any idea, comment or experience. I
> think the most untested case is b), but has anyone tried to use
> Lustre in a similar scenario? Any comment in any case a) o b) are
> /Miguel Angel Díaz Corchero/
> /*System Administrator / Researcher*/
> /c/ Sola nº 1; 10200 TRUJILLO, SPAIN/
> /Tel: +34 927 65 93 17 Fax: +34 927 32 32 37/
> CETA-Ciemat logo <http://www.ceta-ciemat.es/>
> Este mensaje y sus ficheros adjuntos se dirige exclusivamente a su destinatario y puede contener información privilegiada o
> confidencial. Si no es vd. el destinatario indicado, queda notificado de que la utilización, divulgación y/o copia sin autorización está
> prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique
> inmediatamente respondiendo al mensaje y proceda a su destrucción.
> This message and its attached files is intended exclusively for its recipients and may contain confidential information. If you received
> this e-mail in error you are hereby notified that any dissemination, copy or disclosure of this communication is strictly prohibited and
> may be unlawful. In this case, please notify us by a reply and delete this email and its contents immediately.
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> <mailto:OpenStack-operators at lists.openstack.org>
-------------- next part --------------
An HTML attachment was scrubbed...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 26213 bytes
Desc: not available
More information about the OpenStack-operators