回复:Re: [kolla-ansible][manila]Permission denied when using cephfs nfs

17 Feb 2022


      eblock, Thanks for your reply. I have read the report [1] but it can not fix my issue.
In our manila-share container, the manila-share service is running by manila
user not root user.
When I run as root user into manila-share container with command like
"docker exec -it -uroot manila_share bash". And then run "python" into
interactive model. Then run code as followed:
importceph_volume_client
share={"share_group_id":None,"id":"449a52a4-f19c-4d1b-b437-7dc2443e040c"}
c=ceph_volume_client.CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", volume_prefix="/volumes")
c.connect(premount_evict="manila")
v=c.create_volume(ceph_volume_client.VolumePath(share['share_group_id'], share['id']), size=1073741824, data_isolated=False, mode=int('755',8))
I can succeed to create the volume.
But when I run as manila user into manila-share container with command
like "docker exec -it manila_share bash". And then run the same python
codes, it will raise the same error which appears in the manila-share.log file.
[1]: https://bugs.launchpad.net/charm-manila-ganesha/+bug/1901570
Regards
Boxiang
At 2022-02-16 21:41:15, "Eugen Block" eblock@nde.ag wrote:
...
This sounds a lot like this bug:
https://bugs.launchpad.net/charm-manila-ganesha/+bug/1901570
What are your ceph auth caps for the required pool(s)? I'm not  
familiar with Manila or Kolla, but maybe as a workaround you can  
change the directory permissions as described in the report.
Zitat von Boxiang Zhu bxzhu_5355@163.com:
...
Hi all,
I have met an issue when I use manila with cephfs nfs backend.
Environment Information:

OS: ubuntu 18.04
OpenStack Version: Train
Kolla-ansible Version: 9.3.2

I used kolla-ansible to deploy my OpenStack environment AIO.
Here is the info of my globals.yml as followed:
kolla_base_distro: "ubuntu"
kolla_install_type: "source"
openstack_release: "train"
kolla_internal_vip_address: "172.16.150.67"
network_interface: "ens3"
neutron_external_interface: "ens4"
enable_haproxy: "no"
enable_ceph: "yes"
enable_ceph_mds: "yes"
enable_ceph_rgw: "yes"
enable_ceph_nfs: "yes"
enable_cinder: "yes"
enable_manila: "yes"
enable_manila_backend_cephfs_nfs: "yes"
enable_ceph_rgw_keystone: "yes"
ceph_osd_store_type: "filestore"
I try to use cephfs nfs backend for manila.
All containers are running and the services of manila is good.
+----+------------------+-------------------+------+---------+-------+----------------------------+
| Id | Binary | Host | Zone | Status | State | Updated_at |
+----+------------------+-------------------+------+---------+-------+----------------------------+
| 1 | manila-data | manila | nova | enabled | up |  
2022-02-16T03:25:34.000000 |
| 2 | manila-scheduler | manila | nova | enabled | up |  
2022-02-16T03:25:35.000000 |
| 3 | manila-share | manila@cephfsnfs1 | nova | enabled | up |  
2022-02-16T03:25:40.000000 |
+----+------------------+-------------------+------+---------+-------+----------------------------+
Here is my share type:
+--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+
| ID                                   | Name               |  
visibility | is_default | required_extra_specs                |  
optional_extra_specs | Description |
+--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+
| 265a6637-0322-4c4a-9185-f30f01b96d12 | default_share_type | public  
    | -          | driver_handles_share_servers : False |             
          | None        |
+--------------------------------------+--------------------+------------+------------+-------------------------------------+----------------------+-------------+
When I create a share, error information appears in manila-share.log file:
2022-02-16 09:30:04.615 20 INFO manila.share.manager  
[req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share  
status
2022-02-16 09:30:04.616 20 DEBUG manila.share.driver  
[req-b57991af-5355-4202-8543-dc7ff914d919 - - - - -] Updating share  
stats. _update_share_stats  
/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/driver.py:1232
2022-02-16 09:30:54.675 20 DEBUG manila.share.drivers.cephfs.driver  
[req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b  
eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -

-] create_share CEPHFSNFS1

name=449a52a4-f19c-4d1b-b437-7dc2443e040c size=1 share_group_id=None  
create_share  
/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py:262
2022-02-16 09:30:54.682 20 INFO ceph_volume_client  
[req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b  
eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -

-] create_volume:

/volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c2022-02-16  
09:30:54.927 20 ERROR manila.share.manager  
[req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b  
eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -

-] Share instance 449a52a4-f19c-4d1b-b437-7dc2443e040c failed on

creation.: cephfs.OSError: error in mkdir  
volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission  
denied [Errno 13]
2022-02-16 09:30:54.929 20 WARNING manila.share.manager  
[req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b  
eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -

-] Share instance information in exception can not be written to

db because it contains {} and it is not a dictionary.:  
cephfs.OSError: error in mkdir  
volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission  
denied [Errno 13]
2022-02-16 09:30:54.955 20 INFO manila.message.api  
[req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b  
eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -

-] Creating message record for request_id =

req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server  
[req-d2c8ef2a-d3ce-483b-9e39-e5a3d857d72b  
eaf0e30a92694889aa46ac5a1d4b7a47 37025e7c9ae447c8975e9ef3a4e5d0ff -

-] Exception during message handling: cephfs.OSError: error in

mkdir volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c:  
Permission denied [Errno 13]
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback  
(most recent call last):
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/usr/lib/python3/dist-packages/ceph_volume_client.py", line 656, in  
_mkdir_p
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
self.fs.stat(subpath)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"cephfs.pyx", line 1257, in cephfs.LibCephFS.stat
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server  
cephfs.ObjectNotFound: error in stat:  
volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: No such file  
or directory [Errno 2]
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server During  
handling of the above exception, another exception occurred:
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server Traceback  
(most recent call last):
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in  
_process_incoming
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server     res =  
self.dispatcher.dispatch(message)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in  
dispatch
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
return self._do_dispatch(endpoint, method, ctxt, args)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in  
_do_dispatch
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
result = func(ctxt, **new_args)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py",  
line 187, in wrapped
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
return f(self, *args, **kwargs)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/manila/utils.py",  
line 568, in wrapper
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
return func(self, *args, **kwargs)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py",  
line 1790, in create_share_instance
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server     exception=e)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py",  
line 220, in __exit__
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
self.force_reraise()
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py",  
line 196, in force_reraise
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
six.reraise(self.type_, self.value, self.tb)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/six.py", line 693,  
in reraise
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server     raise value
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/manager.py",  
line 1753, in create_share_instance
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
context, share_instance, share_server=share_server)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/var/lib/kolla/venv/lib/python3.6/site-packages/manila/share/drivers/cephfs/driver.py", line 272, in  
create_share
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
mode=self._cephfs_volume_mode)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/usr/lib/python3/dist-packages/ceph_volume_client.py", line 677, in  
create_volume
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
self._mkdir_p(path, mode)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"/usr/lib/python3/dist-packages/ceph_volume_client.py", line 658, in  
_mkdir_p
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server      
self.fs.mkdir(subpath, mode)
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server   File  
"cephfs.pyx", line 887, in cephfs.LibCephFS.mkdir
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server  
cephfs.OSError: error in mkdir  
volumes/_nogroup/449a52a4-f19c-4d1b-b437-7dc2443e040c: Permission  
denied [Errno 13]
2022-02-16 09:30:54.971 20 ERROR oslo_messaging.rpc.server