Open Stack

Hi Jim,

"Ceph RBD for ephemeral volumes (Nova)" means put Nova ephemeral on the
CEPH.
That means all VM will use network as vHD I/O on the CEPH cluster.

If you're not select it, Nova ephemeral will put on the local disk of
compute node.
(For example, if instance created at "node-1" compute node, it will use
node-1 disk space to put vHD file.)

But note that if you create instances by ephemeral, and Nova ephemeral
space is not set on storage cluster (like CEPH, Swift , etc.)
You can't evacuate these instances if compute node crashes.

BTW, here is few tips that may helpful if you decided to build CEPH OSD as
OpenStack storage backend.

1. Normally, 1G ethernet is enough for many cases, but I suggest that let
Storage network running on independent ethernet port.
If still worry, can also try build LACP for Storage network.

2. For disks that will be use for CEPH, it's better that don't build RAID
for them, let them as one OSD per disk will be a better performance.

3. It's better that using SSD as CEPH Journal, it will speed up OSD's
performance.

Just my 2 cents, and sorry about my bad English.


Eddie.

2017-08-25 23:38 GMT+08:00 Jim Okken <jim at jokken.com>:

> thanks Mike for the info
>
> yes I do want very fast VM provisioning and all the useful features that
> comes with having all 3 glance/cinder/ephemeral in CEPH on the storage node.
>
> But I can't afford to have my vHD (either as a cinder volume, or as a
> ephemeral volume) over the network on the storage node.
>
> Do any FUEL experts know exactly what the "Ceph RBD for ephemeral volumes
> (Nova)" option in Fuel 10  does?
> Does it move the running instances vHD off the hypervisors, and onto the
> storage node? (aka: move ephemeral from local IO to, network IO?)
>
> thanks!
>
>
> -- Jim
>
> On Fri, Aug 25, 2017 at 12:08 AM, Mike Smith <mismith at overstock.com>
> wrote:
>
>> Ceph is basically a ‘swiss army knife of storage’.  It can play multiple
>> roles in an Openstack deployment, which is one reason why it is so popular
>> among this crowd.  It can be used as storage for:
>>
>> - nova ephemeral disks (Ceph RBD)
>> - replacement for swift (Ceph Object)
>> - cinder volume backend (Ceph RBD)
>> - glance image backend (Ceph RBD)
>> - gnocchi metrics storage (Ceph Object)
>> - generic filesystem (CephFS)
>>
>> …and probably a few more that I’m missing.
>>
>> The combination of Ceph as backend for glance and nova ephemeral and/or
>> cinder volumes is gorgeous because it’s an ‘instance clone’ of the glance
>> image into the disk/volume which means very fast VM provisioning.  Some
>> people boot instances off of nova ephemeral storage, some prefer to boot
>> off of cinder volumes.  It depends if you want features like QoS (I/O
>> limiting), snapshots, backup, and whether you want the data to be able to
>> ‘persist’ as a volume after the VM that uses it is removed or if you want
>> it to disappear when the VM is deleted (i.e. ‘ephemeral’)
>>
>> I’m not a ‘fuel/mirantis guy’ so I can’t tell you specifically what those
>> options in their installer do, but generally Ceph storage is often housed
>> on separate servers dedicated to Ceph regardless of how you want to use
>> it.  Some people to colocate Ceph onto their compute nodes and have them
>> perform double duty (i.e. ‘hyperconverged’)
>>
>> Hopefully this gives you a little bit of information regarding how Ceph
>> is used.
>>
>>
>> Mike Smith
>> Lead Cloud System Architect
>> Overstock.com
>>
>>
>>
>> On Aug 24, 2017, at 9:22 PM, Jim Okken <jim at jokken.com> wrote:
>>
>> Ive been learning a bit more about storage. Let me share what think I
>> know and ask a more specific question. Please correct me if I am off on
>> what I think I know.
>>
>>
>> Glance Images and Cinder Volumes are traditionally stored on the storage
>> node. Ephemeral volumes (Nova managed, traditionally on the compute node)
>> are the copy of the Glance image that has been copied to the compute node
>> and booted as an instances' vHD. Cinder volumes can (among other things) be
>> added to an instance as additional storage besides this Glance Image.
>>
>>
>> In Fuel I set the "Ceph RBD for volumes (Cinder)" and "Ceph RBD for
>> images (Glance)" settings, which will setup Glance and Cinder on the CEPH
>> OSD storage nodes.
>>
>> But I am not sure about what the setting "Ceph RBD for ephemeral volumes
>> (Nova)" will do.
>>
>> Would selecting it move the running instances' vHD off the hypervisors
>> and onto the storage node? (aka: move ephemeral from local to over the
>> network?
>>
>>
>> Thanks
>>
>>
>> --jim
>>
>>
>>
>> On Thu, Aug 24, 2017 at 12:14 PM, Jim Okken <jim at jokken.com> wrote:
>>
>>> Hi all,
>>>
>>>
>>> We have a pretty complicated storage setup and I am not sure how to
>>> configure Fuel for deployment of the storage nodes. I'm using Fuel
>>> 10/Newton. Plus i'm a bit confused on some of the storage aspects
>>> (image/glance, volume/cinder, ephemeral/?.)
>>>
>>>
>>> We have 3 nodes dedicated to be storage nodes, for HA.
>>>
>>> We’re using fiber channel extents and need to use the CEPH filesystem.
>>>
>>>
>>> I’ll try to simplify the storage situation at first to ask my initial
>>> question without too many details.
>>>
>>>
>>> We have a fast and a slow storage location. Management tells me they
>>> want the slow location for the Glance images and the fast location for the
>>> place where the instances actually run. (assume compute nodes with slow
>>> hard drives but access to a fast fiber channel volume.)
>>>
>>>
>>> Where is “the place where the instances actually run”. It isn’t via
>>> Glance nor Cinder is it?
>>>
>>> When I configure the storage for CEPH OSD node I see volume settings for
>>> Base System, CEPH and CEPH journal. (I see my slow storage and my fast
>>> storage disks).
>>>
>>>
>>> When I configure  the storage for a Compute node I see volume settings
>>> for Base system and Virtual Storage. Is this Ephemeral storage? How does a
>>> Virtual Storage volume here compare to the CEPH volume on the CEPH OSD?
>>>
>>>
>>> I have seen an openstack instance who’s .xml file on the compute node
>>> shows the vHD as a CEPH path (ie: rbd:compute/f63e4d30-7706-40be-8eda-b74e91b9dac1_disk.
>>> Is this a CEPH local to the compute node or CEPH on the storage node? (Is
>>> this Ephemeral storage?)
>>>
>>>
>>> Thanks for any help you might have, I’m a bit confused
>>>
>>>
>>> thanks
>>>
>>>
>>> -- Jim
>>>
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>>
>>
>>
>> ------------------------------
>>
>> CONFIDENTIALITY NOTICE: This message is intended only for the use and
>> review of the individual or entity to which it is addressed and may contain
>> information that is privileged and confidential. If the reader of this
>> message is not the intended recipient, or the employee or agent responsible
>> for delivering the message solely to the intended recipient, you are hereby
>> notified that any dissemination, distribution or copying of this
>> communication is strictly prohibited. If you have received this
>> communication in error, please notify sender immediately by telephone or
>> return email. Thank you.
>>
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
> openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
> openstack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170830/583ccb2f/attachment.html>