hello, 
I'm trying to install manila in my openstack cloud (deployed on k8s cluster) but having problem.

1 -> i created the auth using 

ceph auth get-or-create client.manila -o manila.keyring \
  mgr 'allow rw' \
  mon 'allow r'
source - https://docs.openstack.org/manila/latest/configuration/shared-file-systems/drivers/cephfs_driver.html

2 -> in manila-share pod i see the error

2024-10-08 07:37:25.090 7 INFO manila.share.drivers.cephfs.driver [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] [CEPHFSNATIVE1] Ceph client found, connecting...
2024-10-08 07:37:25.104 7 ERROR manila.share.manager [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] Error encountered during initialization of driver CephFSDriver@lab-11053-26006-ceph-1@cephfsnative1: manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect.
2024-10-08 07:37:25.104 7 ERROR manila.share.manager Traceback (most recent call last):
2024-10-08 07:37:25.104 7 ERROR manila.share.manager   File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 421, in rados_client
2024-10-08 07:37:25.104 7 ERROR manila.share.manager     self._rados_client.connect()
2024-10-08 07:37:25.104 7 ERROR manila.share.manager   File "rados.pyx", line 690, in rados.Rados.connect
2024-10-08 07:37:25.104 7 ERROR manila.share.manager rados.IOError: [errno 5] RADOS I/O error (error connecting to the cluster)
2024-10-08 07:37:25.104 7 ERROR manila.share.manager
2024-10-08 07:37:25.104 7 ERROR manila.share.manager During handling of the above exception, another exception occurred:
2024-10-08 07:37:25.104 7 ERROR manila.share.manager
2024-10-08 07:37:25.104 7 ERROR manila.share.manager Traceback (most recent call last):
2024-10-08 07:37:25.104 7 ERROR manila.share.manager   File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/manager.py", line 371, in _driver_setup
2024-10-08 07:37:25.104 7 ERROR manila.share.manager     self.driver.do_setup(ctxt)
2024-10-08 07:37:25.104 7 ERROR manila.share.manager   File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 274, in do_setup
2024-10-08 07:37:25.104 7 ERROR manila.share.manager     rados_client=self.rados_client,
2024-10-08 07:37:25.104 7 ERROR manila.share.manager   File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 424, in rados_client
2024-10-08 07:37:25.104 7 ERROR manila.share.manager     raise exception.ShareBackendException(
2024-10-08 07:37:25.104 7 ERROR manila.share.manager manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect.
2024-10-08 07:37:25.104 7 ERROR manila.share.manager
2024-10-08 07:37:25.105 7 DEBUG manila.utils [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] Finished call to 'manila.share.manager.ShareManager.init_host.<locals>._driver_setup' after 62.106(s), this was the 6th time calling it. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/after.py:44
2024-10-08 07:37:25.105 7 DEBUG manila.utils [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] Retrying manila.share.manager.ShareManager.init_host.<locals>._driver_setup in 64.0 seconds as it raised ShareBackendException: [%(be)s] Ceph client failed to connect.. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/before_sleep.py:65


3 -> when i try to use the maila keyring i get the error 

[rook@cmp-2 ceph]$ ceph --key AQBip/9mCMf4N/jFNKFlwl/hA4j64esWuA== osd ls
[errno 13] RADOS permission denied (error connecting to the cluster)
this is what i get after running that not sure what permissions any idea?

-> i also tried with admin rights to the auth 

[rook@cmp-2 ceph]$ ceph auth get client.manila
[client.manila]
        key = AQBip/9mCMf4N/jFNKFlwl/hA4j64esWuA==
        caps mds = "allow *"
        caps mgr = "allow *"
        caps mon = "allow *"
        caps osd = "allow *"
exported keyring for client.manila 

4-> my manila conf looks like 
    conf:   
      manila:
        DEFAULT:
          debug: true
          enabled_share_protocols: NFS,CIFS,CEPHFS
          enabled_share_backends: cephfsnative1
        cephfsnative1:
          driver_handles_share_servers: False
          share_backend_name: CEPHFSNATIVE1
          share_driver: manila.share.drivers.cephfs.driver.CephFSDriver
          cephfs_conf_path: /etc/ceph/ceph.conf
          cephfs_protocol_helper_type: CEPHFS
          cephfs_auth_id: manila
          cephfs_cluster_name: rook-ceph
          cephfs_filesystem_name: myfs

any idea what could be wrong? 

Hope to hear from you soon
Thank you!