[Manila] Manila user overwriting existing Ceph users

Tom Barron tpb at dyncloud.net
Thu Nov 12 13:30:39 UTC 2020


On 12/11/20 13:32 +0100, Babel Jahson wrote:
>Hi Giulio,
>
>Thank you for your response.
>
>>the cephx user which cinder/glance/nova use has specific permissions to
>>operate on their pools and this is configured in their respective
>>config, not something you have access from the actual openstack guests;
>>are you saying that "access-allow" is overwriting the cephx caps which
>>were set for the cephx user which, for example, cinder is configured to use?
>Yes a cinder user can be overwritten in the Ceph config cluster by the 
>command "access-allow" to a share.
>Basically it goes from something like this :
>[client.cindertest]
>    key = <cindertestkey>
>    caps mon = "profile rbd"
>    caps osd = "profile rbd pool=some-pool, profile rbd pool=some-pool
>
>To something like that :
>[client.cindertest]
>    key = <cindertestkey>
>    caps mds = "allow rw path=/volumes/_nogroup/<some_id>"
>    caps mon = "allow r"
>    caps osd = "allow rw pool=<cephfs_data> namespace=fsvolumens_<some_id>"
>
>Which can be problematic.
>
>>in that case maybe better would be for the manila workflow to add/remove
>>caps to existing users instead of overwriting the caps? is that be what
>>you expected to happen?
>Not really, I mean it's a possibility but is it safe to just add those 
>caps to an existing user ? Won't that interfere with something else ?
>A way to prevent the creation of a user like "cindertest" seems a 
>better solution to me but I maybe wrong.
>It's behavior manila already has. If a user have been created with 
>manila for a share in a project and you ask for that user in another 
>project in openstack it wouldn't let you used it.
>
>Jahson
>
>On 12/11/2020 11:56, Giulio Fidente wrote:
>>On 11/12/20 10:24 AM, Babel Jahson wrote:
>>>Hello everyone,
>>>
>>>I'm currently testing manila with CephFS and I stumbled upon a behavior
>>>where manila is able to overwrite existing Ceph users.
>>>In my testing setup glance, nova, cinder and manila share the same Ceph
>>>cluster. However they have different users.
>>>In this situation when you create a share and allow acces via "manila
>>>access-allow cephshare1 cephx test"
>>>If the user "test" is already used to access some pools on the cluster,
>>>let's say cinder-volume or glance-images it will be overwritten with the
>>>permissions for the share.
>>>Which will break any resources that was using it.
>>>I've recheck the configuration files multiple times to see if I could
>>>set some properties to avoid this but I didn't find any.
>>>By quickly looking at the code here :
>>>https://opendev.org/openstack/manila/src/branch/master/manila/share/drivers/cephfs/driver.py
>>>A check is done but only for the manila user. I'm on Rocky version but
>>>this part doesn't seems to have changed since.
>>>
>>>That lead me to some questions :
>>>- Does manila must have his own dedicated Ceph cluster ?
>>>- Is there any workaroud to this ? Other than putting some gibberish
>>>names for services users ?
>>>- Is it possible to lock some users in the Ceph cluster to prevent this
>>>behavior ?
>>hi Jahnson, I am adding a few folks who can probably help us better but
>>I also wanted to ask a question to understand better the use case
>>
>>the cephx user which cinder/glance/nova use has specific permissions to
>>operate on their pools and this is configured in their respective
>>config, not something you have access from the actual openstack guests;
>>are you saying that "access-allow" is overwriting the cephx caps which
>>were set for the cephx user which, for example, cinder is configured to use?
>>
>>in that case maybe better would be for the manila workflow to add/remove
>>caps to existing users instead of overwriting the caps? is that be what
>>you expected to happen?
>

Babel,

Thanks for this report.  Would you be so kind as to open a Launchpad 
bug against Manila for this issue?

Manila, Nova, Cinder, etc. all use the same Ceph cluster but use 
different pools in that cluster.  It does seem that we need the CephFS 
driver (or perhaps something in the Ceph cluster) to mark the service 
users as such and prevent changes to them by Manila.

-- Tom Barron




More information about the openstack-discuss mailing list