Open Stack

Hi Tom,

No problem, I'll create a Launchpadd issue for this.

Jahson

On 12/11/2020 14:30, Tom Barron wrote:
> On 12/11/20 13:32 +0100, Babel Jahson wrote:
>> Hi Giulio,
>>
>> Thank you for your response.
>>
>>> the cephx user which cinder/glance/nova use has specific permissions to
>>> operate on their pools and this is configured in their respective
>>> config, not something you have access from the actual openstack guests;
>>> are you saying that "access-allow" is overwriting the cephx caps which
>>> were set for the cephx user which, for example, cinder is configured 
>>> to use?
>> Yes a cinder user can be overwritten in the Ceph config cluster by 
>> the command "access-allow" to a share.
>> Basically it goes from something like this :
>> [client.cindertest]
>>     key = <cindertestkey>
>>     caps mon = "profile rbd"
>>     caps osd = "profile rbd pool=some-pool, profile rbd pool=some-pool
>>
>> To something like that :
>> [client.cindertest]
>>     key = <cindertestkey>
>>     caps mds = "allow rw path=/volumes/_nogroup/<some_id>"
>>     caps mon = "allow r"
>>     caps osd = "allow rw pool=<cephfs_data> 
>> namespace=fsvolumens_<some_id>"
>>
>> Which can be problematic.
>>
>>> in that case maybe better would be for the manila workflow to 
>>> add/remove
>>> caps to existing users instead of overwriting the caps? is that be what
>>> you expected to happen?
>> Not really, I mean it's a possibility but is it safe to just add 
>> those caps to an existing user ? Won't that interfere with something 
>> else ?
>> A way to prevent the creation of a user like "cindertest" seems a 
>> better solution to me but I maybe wrong.
>> It's behavior manila already has. If a user have been created with 
>> manila for a share in a project and you ask for that user in another 
>> project in openstack it wouldn't let you used it.
>>
>> Jahson
>>
>> On 12/11/2020 11:56, Giulio Fidente wrote:
>>> On 11/12/20 10:24 AM, Babel Jahson wrote:
>>>> Hello everyone,
>>>>
>>>> I'm currently testing manila with CephFS and I stumbled upon a 
>>>> behavior
>>>> where manila is able to overwrite existing Ceph users.
>>>> In my testing setup glance, nova, cinder and manila share the same 
>>>> Ceph
>>>> cluster. However they have different users.
>>>> In this situation when you create a share and allow acces via "manila
>>>> access-allow cephshare1 cephx test"
>>>> If the user "test" is already used to access some pools on the 
>>>> cluster,
>>>> let's say cinder-volume or glance-images it will be overwritten 
>>>> with the
>>>> permissions for the share.
>>>> Which will break any resources that was using it.
>>>> I've recheck the configuration files multiple times to see if I could
>>>> set some properties to avoid this but I didn't find any.
>>>> By quickly looking at the code here :
>>>> https://opendev.org/openstack/manila/src/branch/master/manila/share/drivers/cephfs/driver.py 
>>>>
>>>> A check is done but only for the manila user. I'm on Rocky version but
>>>> this part doesn't seems to have changed since.
>>>>
>>>> That lead me to some questions :
>>>> - Does manila must have his own dedicated Ceph cluster ?
>>>> - Is there any workaroud to this ? Other than putting some gibberish
>>>> names for services users ?
>>>> - Is it possible to lock some users in the Ceph cluster to prevent 
>>>> this
>>>> behavior ?
>>> hi Jahnson, I am adding a few folks who can probably help us better but
>>> I also wanted to ask a question to understand better the use case
>>>
>>> the cephx user which cinder/glance/nova use has specific permissions to
>>> operate on their pools and this is configured in their respective
>>> config, not something you have access from the actual openstack guests;
>>> are you saying that "access-allow" is overwriting the cephx caps which
>>> were set for the cephx user which, for example, cinder is configured 
>>> to use?
>>>
>>> in that case maybe better would be for the manila workflow to 
>>> add/remove
>>> caps to existing users instead of overwriting the caps? is that be what
>>> you expected to happen?
>>
>
> Babel,
>
> Thanks for this report.  Would you be so kind as to open a Launchpad 
> bug against Manila for this issue?
>
> Manila, Nova, Cinder, etc. all use the same Ceph cluster but use 
> different pools in that cluster.  It does seem that we need the CephFS 
> driver (or perhaps something in the Ceph cluster) to mark the service 
> users as such and prevent changes to them by Manila.
>
> -- Tom Barron
>