Open Stack

Hi Romain,

On Mon, Jun 13, 2022 at 6:20 PM CHANU ROMAIN <romain.chanu at univ-lyon1.fr> wrote:
>
> Hello,
>
> I added Manila service with CephFS NFS driver to my openstack cluster.
> Everything works fine but I would like to add 2 nfs-ganesha servers to
> ensure high availability to the service.
>
> I configured haproxy to forward 2049 to ganesha backend but Manila
> cephFS NFS provides only IP restriction and see only haproxy's IP
> address. To make it work you have to add haproxy to allowed ip but it
> means everyone can access the share.

True; HAProxy terminates client connections and NFS Ganesha sees the
HAProxy's IP address instead of the client's IP address. This causes
the client's mount operations to be denied since manila explicitly
requests client restrictions to exports according to the share's
access rules. Presumably, setting up haproxy in "transparent" mode may
allow the client source IP to be preserved. We have found that this is
infeasible in deployments such as the Red Hat OpenStack Platform.
We're discussing with the nfs-ganesha community if they would support
the PROXY protocol [1][2].

How are you orchestrating your nfs-ganesha solution with haproxy? Via
a custom script? Or are you using cephadm? [3]. I ask also because the
CephFS-NFS driver in manila currently communicates with NFS-Ganesha
via dbus, and we're looking to support the use of new Ceph Manager
APIs to setup, update and teardown exports in the Zed release - this
should make configuring multiple NFS-Ganesha servers much more
efficient and easy.


[1] https://www.haproxy.com/blog/haproxy/proxy-protocol/
[2] https://github.com/nfs-ganesha/ntirpc/issues/252
[3] https://docs.ceph.com/en/latest/cephadm/services/nfs/
[4] https://docs.ceph.com/en/octopus/cephfs/fs-nfs-exports/#create-cephfs-export



>
> So currently the only way I found out is to use pacemaker to set public
> vip to a running nfs-ganesha node. Could you confirm that is not
> possible to provide an active/active nfs-ganesha cluster with manila
> cephfs NFS driver?
>
> Best regards,
> Romain