Open Stack

On Mon, 2021-06-28 at 18:30 +0200, Gorka Eguileor wrote:
> On 26/06, pradyumna borge wrote:
> > Hi,
> > 
> > In a mulit-node setup do we need to provide shared storage via Cinder
> > when setting up the second compute node?
> > 
> > In a typical mulit-node setup we will have:
> > 1. First node as Controller node acting as a Compute node too. This
> >    will have Cinder *lvm*.
> > 2. Second node as Compute node.
> >    1. Will this node have any storage via lvm? If yes then how will
> >       the first compute node access storage on the second node?
> >    2. Likewise, how can the VMs on this second node access storage on
> >       the first compute node?
> > 
> > My other questions are:
> > 1. So if I spawn a VM on the second Compute node, where will the disks
> >    of the VM reside?
> > 2. Can I attach attach a disk on the first node to a VM on the second
> >    node?
> > 3. Do I have to configure NFS storge as shared storage for Cinder?
> > 4. Does Cinder take care of sharing the disks (I dont think so)
> > 5. What are the steps to setup devstack for multi-node and multi
> >    storage (nfs and lvm)
> > 
> > ~ shree
> > 
> 
> Hi,
> 
> I believe there may be some misunderstandings on how OpenStack operates.
> 
> Some clarifications:
> 
> Nova:
> 
> - Can run VMs without Cinder volumes, using only ephemeral volumes that
>   are stored in compute's local disk.
> - Can run with ephemeral local boot volumes and attach Cinder external
>   volumes.
> - Can run with Cinder boot volumes.
> 
> Cinder:
> 
> Cinder-volume usually connects to an external storage solution that is
> not running on the controller node itself, except when LVM is used.  In
> that case the volume is local no the node where cinder-volume is running
> and the host exports the volume via iSCSI so any compute node can
> connect or the cinder-backup service running on any controller node can
> connect.
> 
> But since the volume data is only stored in that specific node, it means
> that when the node is offline no cinder volume can be used, so it's
> usually only used for POCs.
well pocs or small scale deployment like 3-5 node cluseter that might be deployed
at the edge or labs.

what i have seen people suggest in the past was to use a drbd volume
https://linbit.com/drbd/ for the lvm PV  or similarly use a san/disk shleve to provide the storage for lvm  with reduntant connections
to multiple hosts  and use pacemakeer to manage the cinder volume processs in active backup but in pratice
you should really only use it if our ok wiht only one copy of your data. you can hack aroudn its lack fo ha support
but its not the right direction 


cinder does not explictly garentee that if the host runing cinder volume goes down that you will be
aboule to still acess your data but in pratice you often can. as such its often implictly assumed that
cinder storage is some how redunant.

e.g. with ceph if you have only one instance of cinder volumen and that host goes down the vms will
still be ablel to connect to the ceph cluster assuming it was also not on that host. its only the managblity
that would be impacted and that is mitigated by the fact you can have multiple instnace of cinder volume runing
managing the same ceph cluster.

trust gorka when they advise against the use of lvm in production. while possible it wont fulfil the expectaion of
consomer of cinder that assume there data is safe.
> 
> There are multiple ways to deploying devstack with multiple backends,
> but one way to do it is using the CINDER_ENABLED_BACKENDS variable in
> your local.conf file.
> 
> I never use NFS, but for example, to have 2 LVM backends:
> 
>   CINDER_ENABLED_BACKENDS=lvm:lvmdriver-1,lvm:lvmdriver-2
> 
> To enable the Ceph plugin and have lvm and ceph with:
> 
>   enable_plugin devstack-plugin-ceph git://git.openstack.org/openstack/devstack-plugin-ceph
> 
>   CINDER_ENABLED_BACKENDS=lvm,ceph
> 
> Cheers,
> Gorka.
> 
>