Open Stack

Hi Giulio,

Thank you for your response.

> the cephx user which cinder/glance/nova use has specific permissions to
> operate on their pools and this is configured in their respective
> config, not something you have access from the actual openstack guests;
> are you saying that "access-allow" is overwriting the cephx caps which
> were set for the cephx user which, for example, cinder is configured to use?
Yes a cinder user can be overwritten in the Ceph config cluster by the 
command "access-allow" to a share.
Basically it goes from something like this :
[client.cindertest]
     key = <cindertestkey>
     caps mon = "profile rbd"
     caps osd = "profile rbd pool=some-pool, profile rbd pool=some-pool

To something like that :
[client.cindertest]
     key = <cindertestkey>
     caps mds = "allow rw path=/volumes/_nogroup/<some_id>"
     caps mon = "allow r"
     caps osd = "allow rw pool=<cephfs_data> namespace=fsvolumens_<some_id>"

Which can be problematic.

> in that case maybe better would be for the manila workflow to add/remove
> caps to existing users instead of overwriting the caps? is that be what
> you expected to happen?
Not really, I mean it's a possibility but is it safe to just add those 
caps to an existing user ? Won't that interfere with something else ?
A way to prevent the creation of a user like "cindertest" seems a better 
solution to me but I maybe wrong.
It's behavior manila already has. If a user have been created with 
manila for a share in a project and you ask for that user in another 
project in openstack it wouldn't let you used it.

Jahson

On 12/11/2020 11:56, Giulio Fidente wrote:
> On 11/12/20 10:24 AM, Babel Jahson wrote:
>> Hello everyone,
>>
>> I'm currently testing manila with CephFS and I stumbled upon a behavior
>> where manila is able to overwrite existing Ceph users.
>> In my testing setup glance, nova, cinder and manila share the same Ceph
>> cluster. However they have different users.
>> In this situation when you create a share and allow acces via "manila
>> access-allow cephshare1 cephx test"
>> If the user "test" is already used to access some pools on the cluster,
>> let's say cinder-volume or glance-images it will be overwritten with the
>> permissions for the share.
>> Which will break any resources that was using it.
>> I've recheck the configuration files multiple times to see if I could
>> set some properties to avoid this but I didn't find any.
>> By quickly looking at the code here :
>> https://opendev.org/openstack/manila/src/branch/master/manila/share/drivers/cephfs/driver.py
>> A check is done but only for the manila user. I'm on Rocky version but
>> this part doesn't seems to have changed since.
>>
>> That lead me to some questions :
>> - Does manila must have his own dedicated Ceph cluster ?
>> - Is there any workaroud to this ? Other than putting some gibberish
>> names for services users ?
>> - Is it possible to lock some users in the Ceph cluster to prevent this
>> behavior ?
> hi Jahnson, I am adding a few folks who can probably help us better but
> I also wanted to ask a question to understand better the use case
>
> the cephx user which cinder/glance/nova use has specific permissions to
> operate on their pools and this is configured in their respective
> config, not something you have access from the actual openstack guests;
> are you saying that "access-allow" is overwriting the cephx caps which
> were set for the cephx user which, for example, cinder is configured to use?
>
> in that case maybe better would be for the manila workflow to add/remove
> caps to existing users instead of overwriting the caps? is that be what
> you expected to happen?