On Thu, 14 Dec 2023 at 10:28, Eugen Block <eblock@nde.ag> wrote:

Interesting, I have a kolla-ansible one-node cluster with Antelope and 
there I see what you describe as well. So the behavior did indeed 
change. I guess the docs should be updated and contain read-only rbd 
profile for glance.

This sounds like regression to me.

Indeed this is a regression and it was a wild ride following the various strings along  ...

a) Commit https://github.com/openstack/glance_store/commit/3d221ec529862d43ab303644e74ee9ad6ce8cd40 introduced the method "_snapshot_has_external_reference" to the yoga
release to fix [1]. The commit message also briefly states:

NOTE: To check this dependency glance osd needs 'read' access to
cinder and nova side RBD pool.

but there is zero mention of this requirement in the release notes for Yoga [2].

b) The mentioned method was removed again with [4] and this change was backported to the 2023.1 release.
There again was no mention of the change to operators via the release notes, who could now remove
the read access for volumes from the Glance user again.

c) For none of the changes a and b there was any update to the actual documentation on how to configure the glance user ceph caps.

d) Adding to c, devstack very much is out of sync to what would currently be considered "correct" in regards to caps [7].
Too liberal caps / ACLs  are not helpful when testing for regressions.

e) The "_snapshot_has_external_reference" method is currently just dangling and unused [5].

f) @Jonathan Overriding some managed code should really just be a temporary fix (it was for Stein if I read this correctly).
Could those openstack_keys in [6], once we figured out what the caps really should be, be converted into a PR against upstream of ceph-ansible [8] to fix things at the root?

g) I am still wondering what the caps to allow reading "rbd_children" prefixed rados objects is or was used for?
Especially with the managed profiles such as "rbd" or "rbd-readonly", things should be pretty well covered.

My proposal still is .. to
  * determine the correct caps (least privileges, caps via profiles where possible, ...)

  * fix the documentation and code devstack as "upstreams" first
  * write an upgrade bullet point to release notes for Caracal for operators to check and align their caps from what they might have become over the various releases
  * distribute this as a reference to the deployment tools and also the Ceph docs

Regards

Christian

[1] https://bugs.launchpad.net/glance-store/+bug/1954883
[2] https://docs.openstack.org/releasenotes/glance/yoga.html#
[3] https://review.opendev.org/q/topic:%22bug/1954883%22
[4] https://review.opendev.org/q/I34dcd90a09d43127ff2e8b477750c70f3cc01113
[5] https://opendev.org/openstack/glance_store/src/commit/054bd5ddf5d4d255076bd5f44296f2521e899394/glance_store/_drivers/rbd.py#L455
[6] https://opendev.org/openstack/openstack-ansible/commit/0f92985608c0f6ff941ea0445ae25eab20e94fb4
[7] https://opendev.org/openstack/devstack-plugin-ceph/src/commit/4c22c3d0905589d676bf4865ca5cf57994eb426d/devstack/lib/ceph#L712
[8] https://github.com/ceph/ceph-ansible/blob/b6102975549d8f870b0c20a01edda59d6ceac422/group_vars/all.yml.sample#L642