Open Stack

Ceph is basically a ‘swiss army knife of storage’.  It can play multiple roles in an Openstack deployment, which is one reason why it is so popular among this crowd.  It can be used as storage for:

- nova ephemeral disks (Ceph RBD)
- replacement for swift (Ceph Object)
- cinder volume backend (Ceph RBD)
- glance image backend (Ceph RBD)
- gnocchi metrics storage (Ceph Object)
- generic filesystem (CephFS)

…and probably a few more that I’m missing.

The combination of Ceph as backend for glance and nova ephemeral and/or cinder volumes is gorgeous because it’s an ‘instance clone’ of the glance image into the disk/volume which means very fast VM provisioning.  Some people boot instances off of nova ephemeral storage, some prefer to boot off of cinder volumes.  It depends if you want features like QoS (I/O limiting), snapshots, backup, and whether you want the data to be able to ‘persist’ as a volume after the VM that uses it is removed or if you want it to disappear when the VM is deleted (i.e. ‘ephemeral’)

I’m not a ‘fuel/mirantis guy’ so I can’t tell you specifically what those options in their installer do, but generally Ceph storage is often housed on separate servers dedicated to Ceph regardless of how you want to use it.  Some people to colocate Ceph onto their compute nodes and have them perform double duty (i.e. ‘hyperconverged’)

Hopefully this gives you a little bit of information regarding how Ceph is used.


Mike Smith
Lead Cloud System Architect
Overstock.com<http://Overstock.com>



On Aug 24, 2017, at 9:22 PM, Jim Okken <jim at jokken.com<mailto:jim at jokken.com>> wrote:

Ive been learning a bit more about storage. Let me share what think I know and ask a more specific question. Please correct me if I am off on what I think I know.

Glance Images and Cinder Volumes are traditionally stored on the storage node. Ephemeral volumes (Nova managed, traditionally on the compute node) are the copy of the Glance image that has been copied to the compute node and booted as an instances' vHD. Cinder volumes can (among other things) be added to an instance as additional storage besides this Glance Image.

In Fuel I set the "Ceph RBD for volumes (Cinder)" and "Ceph RBD for images (Glance)" settings, which will setup Glance and Cinder on the CEPH OSD storage nodes.
But I am not sure about what the setting "Ceph RBD for ephemeral volumes (Nova)" will do.
Would selecting it move the running instances' vHD off the hypervisors and onto the storage node? (aka: move ephemeral from local to over the network?

Thanks

--jim



On Thu, Aug 24, 2017 at 12:14 PM, Jim Okken <jim at jokken.com<mailto:jim at jokken.com>> wrote:
Hi all,

We have a pretty complicated storage setup and I am not sure how to configure Fuel for deployment of the storage nodes. I'm using Fuel 10/Newton. Plus i'm a bit confused on some of the storage aspects (image/glance, volume/cinder, ephemeral/?.)

We have 3 nodes dedicated to be storage nodes, for HA.
We’re using fiber channel extents and need to use the CEPH filesystem.

I’ll try to simplify the storage situation at first to ask my initial question without too many details.

We have a fast and a slow storage location. Management tells me they want the slow location for the Glance images and the fast location for the place where the instances actually run. (assume compute nodes with slow hard drives but access to a fast fiber channel volume.)

Where is “the place where the instances actually run”. It isn’t via Glance nor Cinder is it?
When I configure the storage for CEPH OSD node I see volume settings for Base System, CEPH and CEPH journal. (I see my slow storage and my fast storage disks).

When I configure  the storage for a Compute node I see volume settings for Base system and Virtual Storage. Is this Ephemeral storage? How does a Virtual Storage volume here compare to the CEPH volume on the CEPH OSD?

I have seen an openstack instance who’s .xml file on the compute node shows the vHD as a CEPH path (ie: rbd:compute/f63e4d30-7706-40be-8eda-b74e91b9dac1_disk. Is this a CEPH local to the compute node or CEPH on the storage node? (Is this Ephemeral storage?)

Thanks for any help you might have, I’m a bit confused

thanks


-- Jim

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


________________________________

CONFIDENTIALITY NOTICE: This message is intended only for the use and review of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message solely to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify sender immediately by telephone or return email. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170825/75e5e54f/attachment.html>