Hey Christian,

The issue you're encountering with Glance and RBD permissions can indeed be tricky to resolve. Let's break it down together:

1. **Glance and RBD Permissions**:
    - When Glance interacts with RBD (Ceph's block storage), it needs the appropriate permissions to perform operations like deleting images.
    - The error you're seeing, "Operation not permitted," indicates a permission issue.

Yes, but I was simply following the documentation, that's why I started the thread in the first place - to determine what needs to actually go into the documentation.

2. **Children of Glance Images**:
    - Glance images can have dependent children, which are typically Cinder volumes. These volumes reside in a different RBD pool called "volumes."
    - When Glance tries to delete an image, it checks if any dependent children exist. If they do, Glance should handle this gracefully.

Yes. The "funny" thing is, that Ceph (RBD) only replies with the permission error in case there actually are children (in pools the glance user does not have permission for).
But it then does not gracefully return a message about existing children, but replies with a 500 (due to the failure querying for children).

So there is no harm done, but the condition is not handled as intended: Notifying the user about existing children.

3. **Your Current Configuration**:
    - Let's review your current Ceph auth caps configuration for the relevant clients:
        - client.cinder: Has read-only access to the "images" pool and read access to the "volumes" pool.
        - client.glance: Has read access to the "images" pool.
        - client.nova: Has read access to both the "instances" and "images" pools.

My current config is simply "inspired" by the current documentation with the change to managed capabilities called "profiles" such as "rbd" or "rbd-readonly" instead
of raw ACLs such das "rwx". See https://docs.ceph.com/en/latest/rados/operations/user-management/#authorization-capabilities. This also differs in the Cinder / Glance documentation.
Without those the blacklisting of RBD locks does not work, creating other side-effects.

See below for my motivation to discuss this topic and then push some changes towards the documentation.

4. **Missing Permissions**:
    - The issue lies in the Glance configuration. Glance needs read access to the "volumes" pool to handle dependent children correctly.
    - Update the Glance configuration as follows:
        - client.glance:
            - Add the necessary permission for the "volumes" pool:
                ceph auth caps client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=glance-images'
             
5. **Explanation**:
    - The added permission allows Glance to read the children (dependent volumes) from the "volumes" pool.
    - It's more restrictive than allowing full access (allow *), which aligns with your desire for tighter security.

Thanks again. My intention is not (just) to make my setup work, but to discuss and document the proper permissions and to then have them reflected correctly in the installation guides.
It makes no sense for everyone having to figure this out individually.

Regards

Christian