Hey Shubham,
Please keep the conversation on the list.
On Thu, Jun 27, 2024 at 10:36 PM Shubham Kumar Yadav shubham.kumar.yadav369@gmail.com wrote:
Hi, actually the keyring file contains the manila keyring which is mounted on manila-share pod
1 -> ls -l /etc/ceph (from manila-share pod) total 12 -rw-r--r-- 1 root root 279 Jun 26 08:02 ceph.conf -rw-r--r-- 1 root root 62 Jun 26 08:02 keyring -rw-r--r-- 1 root root 92 Mar 4 20:27 rbdmap
2 -> i have already generated manila keyring
Do you mean the content of the keyring is in the "keyring" file? In your earlier email you showed the contents of your ceph.conf, it referred to the file name of the manila keyring as "/etc/ceph/manila.keyring". I don't see that file in the output above.
On Fri, 28 Jun 2024 at 00:46, Goutham Pacha Ravi gouthampravi@gmail.com wrote:
Hi,
On Thu, Jun 27, 2024 at 12:26 AM Shubham Kumar Yadav shubham.kumar.yadav369@gmail.com wrote:
Hi, these are the outputs you asked for
1 -> ls -l /etc/ceph total 12 -rw-r--r-- 1 root root 279 Jun 26 08:02 ceph.conf -rw-r--r-- 1 root root 62 Jun 26 08:02 keyring -rw-r--r-- 1 root root 92 Mar 4 20:27 rbdmap
The manila keyring is missing here; Can you add it here and retry?
2 -> ceph --verbose fs ls -n client.manila parsed_args: Namespace(admin_socket=None, block=False, cephconf=None, client_id=None, client_name='client.manila', cluster=None, cluster_timeout=None, completion=False, help=False, input_file=None, output_file=None, output_format=None, period=1, setgroup=None, setuser=None, status=False, verbose=True, version=False, watch=False, watch_channel=None, watch_debug=False, watch_error=False, watch_info=False, watch_sec=False, watch_warn=False), childargs: ['fs', 'ls'] [errno 1] RADOS permission error (error connecting to the cluster)
3 -> i created keyring using
ceph auth get-or-create client.manila -o manila.keyring \ mgr 'allow rw' \ mon 'allow r'
Hope to hear from you soon Thank you!
On Thu, 27 Jun 2024 at 05:25, Goutham Pacha Ravi gouthampravi@gmail.com wrote:
The client capabilities look correct; Can you paste the output of "ls -l /etc/ceph" from the manila-share pod? Also attempt to use "--verbose" with the Ceph CLI; maybe there'll be a helpful log: "ceph --verbose fs ls -n client.manila"
I'm unsure if you're creating the keyring file by hand.. but if you are, you can use this command to generate a file instead:
ceph auth get-or-create client.manila -o /etc/ceph/ceph.client.manila.keyring
You can use this ^ command to generate it and then make sure to copy it into the /etc/ceph folder that's mounted into the manila-share pod.
On Wed, Jun 26, 2024 at 12:02 PM Alvaro Soto alsotoes@gmail.com wrote:
Mmm wait, I believe I'm mistaken, I didn't read the entire config file. Please omit my comment.
On Wed, Jun 26, 2024, 12:54 PM Alvaro Soto alsotoes@gmail.com wrote:
Hi again, It seems you are missing osd permission on manila user authorization:
Allow manila user to read/write on osd's
Cheers.
On Wed, Jun 26, 2024, 7:36 AM Shubham Kumar Yadav shubham.kumar.yadav369@gmail.com wrote: > > hello, > I'm trying to install manila in my openstack cloud but having problem with "Authorizing the driver to communicate with Ceph" > > 1 -> i have created a configmap with these confs and mounted it on manila-share pod > > [client.manila] > client mount uid = 0 > client mount gid = 0 > log file = /opt/stack/logs/ceph-client.manila.log > admin socket = /opt/stack/status/stack/ceph-$name.$pid.asok > keyring = /etc/ceph/manila.keyring > > > 2 -> then i have created a secret with client.manila keyring and attached it to the manila-share pod at > > keyring = /etc/ceph/manila.keyring > > 3 -> inside manila-share pod i see the logs > > 2024-06-03 09:00:53.682 6 INFO manila.share.drivers.cephfs.driver [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] [CEPHFSNATIVE1] Ceph client found, connecting... > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Error encountered during initialization of driver CephFSDriver@lab-11053-26006-ceph-2@cephfsnative1: manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager Traceback (most recent call last): > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 421, in rados_client > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager self._rados_client.connect() > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "rados.pyx", line 690, in rados.Rados.connect > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager rados.IOError: [errno 5] RADOS I/O error (error connecting to the cluster) > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager During handling of the above exception, another exception occurred: > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager Traceback (most recent call last): > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/manager.py", line 371, in _driver_setup > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager self.driver.do_setup(ctxt) > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 274, in do_setup > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager rados_client=self.rados_client, > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 424, in rados_client > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager raise exception.ShareBackendException( > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. > 2024-06-03 09:00:53.713 6 ERROR manila.share.manager > 2024-06-03 09:00:53.714 6 DEBUG manila.utils [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Finished call to 'manila.share.manager.ShareManager.init_host.<locals>._driver_setup' after 2222.651(s), this was the 12th time calling it. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/after.py:44 > 2024-06-03 09:00:53.714 6 DEBUG manila.utils [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Retrying manila.share.manager.ShareManager.init_host.<locals>._driver_setup in 600.0 seconds as it raised ShareBackendException: [%(be)s] Ceph client failed to connect.. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/before_sleep.py:65 > > > 4 -> My ceph cluster is healthy > > 5 -> My manila confs look like (what could be wrong here)? > > conf: > manila: > DEFAULT: > debug: true > enabled_share_protocols: NFS,CIFS,CEPHFS > enabled_share_backends: cephfsnative1 > cephfsnative1: > driver_handles_share_servers: False > share_backend_name: CEPHFSNATIVE1 > share_driver: manila.share.drivers.cephfs.driver.CephFSDriver > cephfs_conf_path: /etc/ceph/ceph.conf > cephfs_protocol_helper_type: CEPHFS > cephfs_auth_id: manila > cephfs_cluster_name: rook-ceph ( my rook-ceph cluster name) > cephfs_filesystem_name: myfs (i created this fs manually) > > 6 -> Permissions for the user manila > > ceph auth get-or-create client.manila -o manila.keyring \ > mgr 'allow rw' \ > mon 'allow r' > > > 7 -> ceph --version > ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy (stable) > > openstackhelm Manila Chart > name: manila > version: 0.1.7 > > 8 -> ceph -n client.manila fs ls > [errno 1] RADOS permission error (error connecting to the cluster) > (while it works with admin) > > Hope to hear from you soon with a solution > Thank you!