Hi Romain,
On Mon, Jun 13, 2022 at 6:20 PM CHANU ROMAIN romain.chanu@univ-lyon1.fr wrote:
Hello,
I added Manila service with CephFS NFS driver to my openstack cluster. Everything works fine but I would like to add 2 nfs-ganesha servers to ensure high availability to the service.
I configured haproxy to forward 2049 to ganesha backend but Manila cephFS NFS provides only IP restriction and see only haproxy's IP address. To make it work you have to add haproxy to allowed ip but it means everyone can access the share.
True; HAProxy terminates client connections and NFS Ganesha sees the HAProxy's IP address instead of the client's IP address. This causes the client's mount operations to be denied since manila explicitly requests client restrictions to exports according to the share's access rules. Presumably, setting up haproxy in "transparent" mode may allow the client source IP to be preserved. We have found that this is infeasible in deployments such as the Red Hat OpenStack Platform. We're discussing with the nfs-ganesha community if they would support the PROXY protocol [1][2].
How are you orchestrating your nfs-ganesha solution with haproxy? Via a custom script? Or are you using cephadm? [3]. I ask also because the CephFS-NFS driver in manila currently communicates with NFS-Ganesha via dbus, and we're looking to support the use of new Ceph Manager APIs to setup, update and teardown exports in the Zed release - this should make configuring multiple NFS-Ganesha servers much more efficient and easy.
[1] https://www.haproxy.com/blog/haproxy/proxy-protocol/ [2] https://github.com/nfs-ganesha/ntirpc/issues/252 [3] https://docs.ceph.com/en/latest/cephadm/services/nfs/ [4] https://docs.ceph.com/en/octopus/cephfs/fs-nfs-exports/#create-cephfs-export
So currently the only way I found out is to use pacemaker to set public vip to a running nfs-ganesha node. Could you confirm that is not possible to provide an active/active nfs-ganesha cluster with manila cephfs NFS driver?
Best regards, Romain