Hi,
On Thu, Jun 27, 2024 at 12:26 AM Shubham Kumar Yadav shubham.kumar.yadav369@gmail.com wrote:
Hi, these are the outputs you asked for
1 -> ls -l /etc/ceph total 12 -rw-r--r-- 1 root root 279 Jun 26 08:02 ceph.conf -rw-r--r-- 1 root root 62 Jun 26 08:02 keyring -rw-r--r-- 1 root root 92 Mar 4 20:27 rbdmap
The manila keyring is missing here; Can you add it here and retry?
2 -> ceph --verbose fs ls -n client.manila parsed_args: Namespace(admin_socket=None, block=False, cephconf=None, client_id=None, client_name='client.manila', cluster=None, cluster_timeout=None, completion=False, help=False, input_file=None, output_file=None, output_format=None, period=1, setgroup=None, setuser=None, status=False, verbose=True, version=False, watch=False, watch_channel=None, watch_debug=False, watch_error=False, watch_info=False, watch_sec=False, watch_warn=False), childargs: ['fs', 'ls'] [errno 1] RADOS permission error (error connecting to the cluster)
3 -> i created keyring using
ceph auth get-or-create client.manila -o manila.keyring \ mgr 'allow rw' \ mon 'allow r'
Hope to hear from you soon Thank you!
On Thu, 27 Jun 2024 at 05:25, Goutham Pacha Ravi gouthampravi@gmail.com wrote:
The client capabilities look correct; Can you paste the output of "ls -l /etc/ceph" from the manila-share pod? Also attempt to use "--verbose" with the Ceph CLI; maybe there'll be a helpful log: "ceph --verbose fs ls -n client.manila"
I'm unsure if you're creating the keyring file by hand.. but if you are, you can use this command to generate a file instead:
ceph auth get-or-create client.manila -o /etc/ceph/ceph.client.manila.keyring
You can use this ^ command to generate it and then make sure to copy it into the /etc/ceph folder that's mounted into the manila-share pod.
On Wed, Jun 26, 2024 at 12:02 PM Alvaro Soto alsotoes@gmail.com wrote:
Mmm wait, I believe I'm mistaken, I didn't read the entire config file. Please omit my comment.
On Wed, Jun 26, 2024, 12:54 PM Alvaro Soto alsotoes@gmail.com wrote:
Hi again, It seems you are missing osd permission on manila user authorization:
Allow manila user to read/write on osd's
Cheers.
On Wed, Jun 26, 2024, 7:36 AM Shubham Kumar Yadav shubham.kumar.yadav369@gmail.com wrote:
hello, I'm trying to install manila in my openstack cloud but having problem with "Authorizing the driver to communicate with Ceph"
1 -> i have created a configmap with these confs and mounted it on manila-share pod
[client.manila] client mount uid = 0 client mount gid = 0 log file = /opt/stack/logs/ceph-client.manila.log admin socket = /opt/stack/status/stack/ceph-$name.$pid.asok keyring = /etc/ceph/manila.keyring
2 -> then i have created a secret with client.manila keyring and attached it to the manila-share pod at
keyring = /etc/ceph/manila.keyring
3 -> inside manila-share pod i see the logs
2024-06-03 09:00:53.682 6 INFO manila.share.drivers.cephfs.driver [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] [CEPHFSNATIVE1] Ceph client found, connecting... 2024-06-03 09:00:53.713 6 ERROR manila.share.manager [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Error encountered during initialization of driver CephFSDriver@lab-11053-26006-ceph-2@cephfsnative1: manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. 2024-06-03 09:00:53.713 6 ERROR manila.share.manager Traceback (most recent call last): 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 421, in rados_client 2024-06-03 09:00:53.713 6 ERROR manila.share.manager self._rados_client.connect() 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "rados.pyx", line 690, in rados.Rados.connect 2024-06-03 09:00:53.713 6 ERROR manila.share.manager rados.IOError: [errno 5] RADOS I/O error (error connecting to the cluster) 2024-06-03 09:00:53.713 6 ERROR manila.share.manager 2024-06-03 09:00:53.713 6 ERROR manila.share.manager During handling of the above exception, another exception occurred: 2024-06-03 09:00:53.713 6 ERROR manila.share.manager 2024-06-03 09:00:53.713 6 ERROR manila.share.manager Traceback (most recent call last): 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/manager.py", line 371, in _driver_setup 2024-06-03 09:00:53.713 6 ERROR manila.share.manager self.driver.do_setup(ctxt) 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 274, in do_setup 2024-06-03 09:00:53.713 6 ERROR manila.share.manager rados_client=self.rados_client, 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 424, in rados_client 2024-06-03 09:00:53.713 6 ERROR manila.share.manager raise exception.ShareBackendException( 2024-06-03 09:00:53.713 6 ERROR manila.share.manager manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. 2024-06-03 09:00:53.713 6 ERROR manila.share.manager 2024-06-03 09:00:53.714 6 DEBUG manila.utils [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Finished call to 'manila.share.manager.ShareManager.init_host.<locals>._driver_setup' after 2222.651(s), this was the 12th time calling it. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/after.py:44 2024-06-03 09:00:53.714 6 DEBUG manila.utils [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Retrying manila.share.manager.ShareManager.init_host.<locals>._driver_setup in 600.0 seconds as it raised ShareBackendException: [%(be)s] Ceph client failed to connect.. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/before_sleep.py:65
4 -> My ceph cluster is healthy
5 -> My manila confs look like (what could be wrong here)?
conf: manila: DEFAULT: debug: true enabled_share_protocols: NFS,CIFS,CEPHFS enabled_share_backends: cephfsnative1 cephfsnative1: driver_handles_share_servers: False share_backend_name: CEPHFSNATIVE1 share_driver: manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path: /etc/ceph/ceph.conf cephfs_protocol_helper_type: CEPHFS cephfs_auth_id: manila cephfs_cluster_name: rook-ceph ( my rook-ceph cluster name) cephfs_filesystem_name: myfs (i created this fs manually)6 -> Permissions for the user manila
ceph auth get-or-create client.manila -o manila.keyring \ mgr 'allow rw' \ mon 'allow r'
7 -> ceph --version ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy (stable)
openstackhelm Manila Chart name: manila version: 0.1.78 -> ceph -n client.manila fs ls [errno 1] RADOS permission error (error connecting to the cluster) (while it works with admin)
Hope to hear from you soon with a solution Thank you!