Open Stack

On Sun, Nov 1, 2020 at 4:32 AM Oliver Weinmann <oliver.weinmann at me.com> wrote:
>
> Hi Jerome,
>
> thanks for your reply. Ok, but can rook also be used outside of
> kubernetes? Can I provision e.g. volumes for openstack cinder?

Rook provides storage for k8s and Ceph is one of its storage
providers. I don't think a Ceph cluster as orchestrated by rook was
designed for access outside of k8s. I've seen people expose parts of
it outside of k8s [0] but that type of thing doesn't seem to be a
standard rook pattern. I wonder how many people are doing that.

If you want Ceph to provide pools to both k8s and OpenStack, then one
approach I see is the following:

1. Install Ceph on its own independent system so it's external to both
OpenStack and k8s and use cephadm to do that deployment [1]
2. Install rook for k8s access to Ceph but direct rook to the external
Ceph cluster from step 1 [2]
3. Install OpenStack but direct it to the external Ceph cluster from
step 1, e.g. TripleO can do this [3]

I'm suggesting a tool to install Ceph, cephadm [1], which isn't in the
current list of tools in the subject. One benefit of this tool is that
you'd have the new Orchestrator APIs that Jerome mentioned but you
wouldn't have to use k8s if you don't want to. The downside is that
you don't have k8s operators directly managing Ceph. An upside of
using the 3-step approach above is that one Ceph cluster can be used
as the same backend to both OpenStack and k8s services. I haven't done
this myself but I don't see why this wouldn't work. You'd just manage
different pools/cephx keys for different services.

If you're not planning to use k8s, the above (without step 2) would
also work well for a large deployment. If you're interested in a
smaller footprint with just OpenStack and Ceph, perhaps where you
collocate OSD and Compute containers (aka "hyperconverged"), then
TripleO will deploy that too and it uses ceph-ansible to deploy Ceph
behind the scenes [4]. TripleO can also deploy CephFS for NFS access
from OpenStack tenants via ganesha. This can work even if the Ceph
cluster itself is external [6] though access to the NFS service is
meant for OpenStack tenants, not e.g. VMWare tenants. This method of
deployment will likely evolve over the next two TripleO cycles to use
cephadm in place of ceph-ansible [7].

  John

[0] https://www.adaltas.com/en/2020/04/16/expose-ceph-from-rook-kubernetes/
[1] https://docs.ceph.com/en/latest/cephadm
[2] https://github.com/rook/rook/blob/master/design/ceph/ceph-external-cluster.md
[3] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/ceph_external.html
[4] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/ceph_config.html
[5] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/deploy_manila.html
[6] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/deploy_manila.html#deploying-the-overcloud-with-an-external-backend
[7] https://review.opendev.org/#/c/723108/

>
> Am 01.11.2020 um 09:49 schrieb Jerome Pansanel:
> > Hi Oliver,
> >
> > Due to recent changes in CEPH Nautilus and Octopus (new Orchestrator
> > APIs), we decided to switch to Rook. It is still in an experimental
> > stage on our platform, but it seems promising.
> >
> > Cheers,
> >
> > Jerome
> >
> > Le 01/11/2020 à 08:55, Oliver Weinmann a écrit :
> >> Hi,
> >>
> >> I'm still in the process of preparing a OpenStack POC. I'm 100% sure
> >> that I want to use CEPH and so I purchased the book Mastering CEPH 2nd
> >> edition. First of all, It's a really good book. It basically explains
> >> the various methods how ceph can be deployed and also the components
> >> that CEPH is build of. So I played around a lot with ceph-ansible and
> >> rook in my virtualbox environment. I also started to play with tripleo
> >> ceph deployment, although I haven't had the time yet to sucessfully
> >> deploy a openstack cluster with CEPH. Now I'm wondering, which of these
> >> 3 methods should I use?
> >>
> >> rook
> >>
> >> ceph-ansible
> >>
> >> tripleo
> >>
> >> I also want to use CEPH for NFS and CIFS (Samba) as we have plenty of
> >> VMs running under vSphere that currently consume storage from a ZFS
> >> storage via CIFS and NFS. I don't know if rook can be used for this. I
> >> have the feeling that it is purely meant to be used for kubernetes? And
> >> If I would like to have CIFS and NFS maybe tripleo is not capable of
> >> enabling these features in CEPH? So I would be left with ceph-ansible?
> >>
> >> Any suggestions are very welcome.
> >>
> >> Best Regards,
> >>
> >> Oliver
> >>
> >>
> >
>