[kolla-ansible][manila]Permission denied when using cephfs nfs
Hi all,
I have met an issue when I use manila with cephfs nfs backend.
Environment Information: - OS: ubuntu 18.04 - OpenStack Version: Train - Kolla-ansible Version: 9.3.2
I used kolla-ansible to deploy my OpenStack environment AIO.
Here is the info of my globals.yml as followed: --- kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "train" kolla_internal_vip_address: "172.16.150.67" network_interface: "ens3" neutron_external_interface: "ens4" enable_haproxy: "no" enable_ceph: "yes" enable_ceph_mds: "yes" enable_ceph_rgw: "yes" enable_ceph_nfs: "yes" enable_cinder: "yes" enable_manila: "yes" enable_manila_backend_cephfs_nfs: "yes" enable_ceph_rgw_keystone: "yes" ceph_osd_store_type: "filestore"
I try to use cephfs nfs backend for manila. All containers are running and the services of manila is good. +----+------------------+-------------------+------+---------+-------+----------------------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | +----+------------------+-------------------+------+---------+-------+----------------------------+ | 1 | manila-data | manila | nova | enabled | up | 2022-02-16T03:25:34.000000 | | 2 | manila-scheduler | manila | nova | enabled | up | 2022-02-16T03:25:35.000000 | | 3 | manila-share | manila@cephfsnfs1 | nova | enabled | up | 2022-02-16T03:25:40.000000 | +----+------------------+-------------------+------+---------+-------+----------------------------+
Here is my share type: +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | 265a6637-0322-4c4a-9185-f30f01b96d12 | default_share_type | public | - | driver_handles_share_servers : False | | None | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+
When I create a share, error information appears in manila-share.log file:
2022-02-16 09:30:04.615 20 INFO manila.share.manager [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share status 2022-02-16 09:30:04.616 20 DEBUG manila.share.driver [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share stats. _update_share_stats /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/driver.py:1232 2022-02-16 09:30:54.675 20 DEBUG manila.share.drivers.cephfs.driver [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff - - -] create_share CEPHFSNFS1 name=449a52a4-f19c-4d1b-b437-7dc2443e040c size=1 share_group_id=None create_share /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py:262 2022-02-16 09:30:54.682 20 INFO ceph_volume_client [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff - - -] create_volume: /volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c2022-02-16 09:30:54.927 20 ERROR manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff - - -] Share instance 449a52a4-f19c-4d1b-b437-7dc2443e040c failed on creation.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.929 20 WARNING manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff - - -] Share instance information in exception can not be written to db because it contains {} and it is not a dictionary.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.955 20 INFO manila.message.api [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff - - -] Creating message record for request_id = req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff - - -] Exception during message handling: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 656, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.stat(subpath) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 1257, in cephfs.LibCephFS.stat 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.ObjectNotFound: error in stat: volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: No such file or directory [Errno 2] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 187, in wrapped 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/utils.py", line 568, in wrapper 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1790, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server exception=e) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.force_reraise() 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/six.py", line 693, in reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server raise value 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1753, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server context, share_instance, share_server=share_server) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py", line 272, in create_share 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server mode=self._cephfs_volume_mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 677, in create_volume 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self._mkdir_p(path, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 658, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.mkdir(subpath, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 887, in cephfs.LibCephFS.mkdir 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server
This sounds a lot like this bug: https://bugs.launchpad.net/charm-manila-ganesha/+bug/1901570
What are your ceph auth caps for the required pool(s)? I'm not familiar with Manila or Kolla, but maybe as a workaround you can change the directory permissions as described in the report.
Zitat von Boxiang Zhu bxzhu_5355@163.com:
Hi all,
I have met an issue when I use manila with cephfs nfs backend.
Environment Information:
- OS: ubuntu 18.04
- OpenStack Version: Train
- Kolla-ansible Version: 9.3.2
I used kolla-ansible to deploy my OpenStack environment AIO.
Here is the info of my globals.yml as followed:
kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "train" kolla_internal_vip_address: "172.16.150.67" network_interface: "ens3" neutron_external_interface: "ens4" enable_haproxy: "no" enable_ceph: "yes" enable_ceph_mds: "yes" enable_ceph_rgw: "yes" enable_ceph_nfs: "yes" enable_cinder: "yes" enable_manila: "yes" enable_manila_backend_cephfs_nfs: "yes" enable_ceph_rgw_keystone: "yes" ceph_osd_store_type: "filestore"
I try to use cephfs nfs backend for manila. All containers are running and the services of manila is good. +----+------------------+-------------------+------+---------+-------+----------------------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | +----+------------------+-------------------+------+---------+-------+----------------------------+ | 1 | manila-data | manila | nova | enabled | up | 2022-02-16T03:25:34.000000 | | 2 | manila-scheduler | manila | nova | enabled | up | 2022-02-16T03:25:35.000000 | | 3 | manila-share | manila@cephfsnfs1 | nova | enabled | up | 2022-02-16T03:25:40.000000 | +----+------------------+-------------------+------+---------+-------+----------------------------+
Here is my share type: +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | 265a6637-0322-4c4a-9185-f30f01b96d12 | default_share_type | public | - | driver_handles_share_servers : False | | None | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+
When I create a share, error information appears in manila-share.log file:
2022-02-16 09:30:04.615 20 INFO manila.share.manager [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share status 2022-02-16 09:30:04.616 20 DEBUG manila.share.driver [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share stats. _update_share_stats /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/driver.py:1232 2022-02-16 09:30:54.675 20 DEBUG manila.share.drivers.cephfs.driver [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] create_share CEPHFSNFS1
name=449a52a4-f19c-4d1b-b437-7dc2443e040c size=1 share_group_id=None create_share /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py:262 2022-02-16 09:30:54.682 20 INFO ceph_volume_client [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] create_volume:
/volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c2022-02-16 09:30:54.927 20 ERROR manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Share instance 449a52a4-f19c-4d1b-b437-7dc2443e040c failed on
creation.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.929 20 WARNING manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Share instance information in exception can not be written to
db because it contains {} and it is not a dictionary.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.955 20 INFO manila.message.api [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Creating message record for request_id =
req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Exception during message handling: cephfs.OSError: error in
mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 656, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.stat(subpath) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 1257, in cephfs.LibCephFS.stat 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.ObjectNotFound: error in stat: volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: No such file or directory [Errno 2] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 187, in wrapped 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/utils.py", line 568, in wrapper 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1790, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server exception=e) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.force_reraise() 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/six.py", line 693, in reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server raise value 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1753, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server context, share_instance, share_server=share_server) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py", line 272, in create_share 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server mode=self._cephfs_volume_mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 677, in create_volume 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self._mkdir_p(path, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 658, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.mkdir(subpath, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 887, in cephfs.LibCephFS.mkdir 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server
eblock, Thanks for your reply. I have read the report [1] but it can not fix my issue.
In our manila-share container, the manila-share service is running by manila user not root user.
When I run as root user into manila-share container with command like "docker exec -it -uroot manila_share bash". And then run "python" into interactive model. Then run code as followed: importceph_volume_client share={"share_group_id":None,"id":"449a52a4-f19c-4d1b-b437-7dc2443e040c"} c=ceph_volume_client.CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", volume_prefix="/volumes") c.connect(premount_evict="manila") v=c.create_volume(ceph_volume_client.VolumePath(share['share_group_id'], share['id']), size=1073741824, data_isolated=False, mode=int('755',8)) I can succeed to create the volume.
But when I run as manila user into manila-share container with command like "docker exec -it manila_share bash". And then run the same python codes, it will raise the same error which appears in the manila-share.log file.
[1]: https://bugs.launchpad.net/charm-manila-ganesha/+bug/1901570
Regards
Boxiang
At 2022-02-16 21:41:15, "Eugen Block" eblock@nde.ag wrote:
This sounds a lot like this bug: https://bugs.launchpad.net/charm-manila-ganesha/+bug/1901570
What are your ceph auth caps for the required pool(s)? I'm not familiar with Manila or Kolla, but maybe as a workaround you can change the directory permissions as described in the report.
Zitat von Boxiang Zhu bxzhu_5355@163.com:
Hi all,
I have met an issue when I use manila with cephfs nfs backend.
Environment Information:
- OS: ubuntu 18.04
- OpenStack Version: Train
- Kolla-ansible Version: 9.3.2
I used kolla-ansible to deploy my OpenStack environment AIO.
Here is the info of my globals.yml as followed:
kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "train" kolla_internal_vip_address: "172.16.150.67" network_interface: "ens3" neutron_external_interface: "ens4" enable_haproxy: "no" enable_ceph: "yes" enable_ceph_mds: "yes" enable_ceph_rgw: "yes" enable_ceph_nfs: "yes" enable_cinder: "yes" enable_manila: "yes" enable_manila_backend_cephfs_nfs: "yes" enable_ceph_rgw_keystone: "yes" ceph_osd_store_type: "filestore"
I try to use cephfs nfs backend for manila. All containers are running and the services of manila is good. +----+------------------+-------------------+------+---------+-------+----------------------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | +----+------------------+-------------------+------+---------+-------+----------------------------+ | 1 | manila-data | manila | nova | enabled | up | 2022-02-16T03:25:34.000000 | | 2 | manila-scheduler | manila | nova | enabled | up | 2022-02-16T03:25:35.000000 | | 3 | manila-share | manila@cephfsnfs1 | nova | enabled | up | 2022-02-16T03:25:40.000000 | +----+------------------+-------------------+------+---------+-------+----------------------------+
Here is my share type: +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | 265a6637-0322-4c4a-9185-f30f01b96d12 | default_share_type | public | - | driver_handles_share_servers : False | | None | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+
When I create a share, error information appears in manila-share.log file:
2022-02-16 09:30:04.615 20 INFO manila.share.manager [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share status 2022-02-16 09:30:04.616 20 DEBUG manila.share.driver [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share stats. _update_share_stats /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/driver.py:1232 2022-02-16 09:30:54.675 20 DEBUG manila.share.drivers.cephfs.driver [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] create_share CEPHFSNFS1
name=449a52a4-f19c-4d1b-b437-7dc2443e040c size=1 share_group_id=None create_share /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py:262 2022-02-16 09:30:54.682 20 INFO ceph_volume_client [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] create_volume:
/volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c2022-02-16 09:30:54.927 20 ERROR manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Share instance 449a52a4-f19c-4d1b-b437-7dc2443e040c failed on
creation.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.929 20 WARNING manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Share instance information in exception can not be written to
db because it contains {} and it is not a dictionary.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.955 20 INFO manila.message.api [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Creating message record for request_id =
req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Exception during message handling: cephfs.OSError: error in
mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 656, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.stat(subpath) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 1257, in cephfs.LibCephFS.stat 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.ObjectNotFound: error in stat: volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: No such file or directory [Errno 2] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 187, in wrapped 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/utils.py", line 568, in wrapper 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1790, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server exception=e) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.force_reraise() 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/six.py", line 693, in reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server raise value 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1753, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server context, share_instance, share_server=share_server) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py", line 272, in create_share 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server mode=self._cephfs_volume_mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 677, in create_volume 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self._mkdir_p(path, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 658, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.mkdir(subpath, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 887, in cephfs.LibCephFS.mkdir 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server
Manila interacts with ceph using a ceph client user (this user is called “manila”by default, but is customizable with config option [1]). It’s likely that user doesn’t have permission to use the data pool that’s assigned to the underlying cephfs file system; can you check these permissions? (called “caps”)
In Train, these are the permissions the ceph client user needs: https://docs.openstack.org/manila/train/admin/cephfs_driver.html#authorizing...
Can you check that these permissions match in your deployment?
If they do, check the file permissions for the ceph keyring associated with the user.
[1] https://github.com/openstack/manila/blob/191e8c9634d606c16ca0c1882cc01f4eb31...
On Wed, Feb 16, 2022 at 6:08 PM Boxiang Zhu bxzhu_5355@163.com wrote:
eblock, Thanks for your reply. I have read the report [1] but it can not fix my issue.
In our manila-share container, the manila-share service is running by *manila* *user not root user*.
When I run as *root user* into manila-share container with command like "docker exec -it -uroot manila_share bash". And then run "python" into interactive model. Then run code as followed: import ceph_volume_client share={"share_group_id":None,"id":"449a52a4-f19c-4d1b-b437-7dc2443e040c"} c=ceph_volume_client.CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", volume_prefix="/volumes") c.connect(premount_evict="manila") v=c.create_volume(ceph_volume_client.VolumePath(share['share_group_id'], share['id']), size=1073741824, data_isolated=False, mode=int('755',8)) I can *succeed *to create the volume.
But when I run as manila user into manila-share container with command like "docker exec -it manila_share bash". And then run the same python codes, it will raise the same error which appears in the manila-share.log file.
Regards
Boxiang
At 2022-02-16 21:41:15, "Eugen Block" eblock@nde.ag wrote:
This sounds a lot like this bug: https://bugs.launchpad.net/charm-manila-ganesha/+bug/1901570
What are your ceph auth caps for the required pool(s)? I'm not familiar with Manila or Kolla, but maybe as a workaround you can change the directory permissions as described in the report.
Zitat von Boxiang Zhu bxzhu_5355@163.com:
Hi all,
I have met an issue when I use manila with cephfs nfs backend.
Environment Information:
- OS: ubuntu 18.04
- OpenStack Version: Train
- Kolla-ansible Version: 9.3.2
I used kolla-ansible to deploy my OpenStack environment AIO.
Here is the info of my globals.yml as followed:
kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "train" kolla_internal_vip_address: "172.16.150.67" network_interface: "ens3" neutron_external_interface: "ens4" enable_haproxy: "no" enable_ceph: "yes" enable_ceph_mds: "yes" enable_ceph_rgw: "yes" enable_ceph_nfs: "yes" enable_cinder: "yes" enable_manila: "yes" enable_manila_backend_cephfs_nfs: "yes" enable_ceph_rgw_keystone: "yes" ceph_osd_store_type: "filestore"
I try to use cephfs nfs backend for manila. All containers are running and the services of manila is good. +----+------------------+-------------------+------+---------+-------+----------------------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | +----+------------------+-------------------+------+---------+-------+----------------------------+ | 1 | manila-data | manila | nova | enabled | up | 2022-02-16T03:25:34.000000 | | 2 | manila-scheduler | manila | nova | enabled | up | 2022-02-16T03:25:35.000000 | | 3 | manila-share | manila@cephfsnfs1 | nova | enabled | up | 2022-02-16T03:25:40.000000 | +----+------------------+-------------------+------+---------+-------+----------------------------+
Here is my share type: +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | ID | Name | visibility | is_default | required_extra_specs | optional_extra_specs | Description | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+ | 265a6637-0322-4c4a-9185-f30f01b96d12 | default_share_type | public | - | driver_handles_share_servers : False | | None | +--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+
When I create a share, error information appears in manila-share.log file:
2022-02-16 09:30:04.615 20 INFO manila.share.manager [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share status 2022-02-16 09:30:04.616 20 DEBUG manila.share.driver [req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share stats. _update_share_stats /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/driver.py:1232 2022-02-16 09:30:54.675 20 DEBUG manila.share.drivers.cephfs.driver [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] create_share CEPHFSNFS1
name=449a52a4-f19c-4d1b-b437-7dc2443e040c size=1 share_group_id=None create_share /var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py:262 2022-02-16 09:30:54.682 20 INFO ceph_volume_client [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] create_volume:
/volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c2022-02-16 09:30:54.927 20 ERROR manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Share instance 449a52a4-f19c-4d1b-b437-7dc2443e040c failed on
creation.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.929 20 WARNING manila.share.manager [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Share instance information in exception can not be written to
db because it contains {} and it is not a dictionary.: cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.955 20 INFO manila.message.api [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Creating message record for request_id =
req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server [req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -
- -] Exception during message handling: cephfs.OSError: error in
mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 656, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.stat(subpath) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 1257, in cephfs.LibCephFS.stat 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.ObjectNotFound: error in stat: volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: No such file or directory [Errno 2] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 187, in wrapped 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/utils.py", line 568, in wrapper 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1790, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server exception=e) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.force_reraise() 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/six.py", line 693, in reraise 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server raise value 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py", line 1753, in create_share_instance 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server context, share_instance, share_server=share_server) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py", line 272, in create_share 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server mode=self._cephfs_volume_mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 677, in create_volume 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self._mkdir_p(path, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/ceph_volume_client.py", line 658, in _mkdir_p 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server self.fs.mkdir(subpath, mode) 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server File "cephfs.pyx", line 887, in cephfs.LibCephFS.mkdir 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server cephfs.OSError: error in mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission denied [Errno 13] 2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server
Hi gouthampravi,
Thanks for your reply. I have recheck the issue[1][2] and I found some commands. I run commands as followed: - docker exec -it -uroot manila_share bash - mount -t ceph 172.20.0.33:6789:/ /mnt -o name=manila,secret=<manila client key> - chown 42429:42429 /mnt/ - umount /mnt/
At last, it resolved my issue.
[1]: https://bugs.launchpad.net/charm-manila-ganesha/+bug/1901570
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1639823
Regards
Boxiang
At 2022-02-17 11:41:34,"Goutham Pacha Ravi" gouthampravi@gmail.com, said:
Manila interacts with ceph using a ceph client user (this user is called “manila”by default, but is customizable with config option [1]). It’s likely that user doesn’t have permission to use the data pool that’s assigned to the underlying cephfs file system; can you check these permissions? (called “caps”)
In Train, these are the permissions the ceph client user needs: https://docs.openstack.org/manila/train/admin/cephfs_driver.html#authorizing...
Can you check that these permissions match in your deployment?
If they do, check the file permissions for the ceph keyring associated with the user.
[1] https://github.com/openstack/manila/blob/191e8c9634d606c16ca0c1882cc01f4eb31...
participants (3)
-
Boxiang Zhu
-
Eugen Block
-
Goutham Pacha Ravi