Thanks for looping keystone into this, Ben.
This sounds very similar, and related, to the default roles work keystone implemented in Rocky [0]. TL;DR, keystone ensures the ``member`` and ``reader`` roles are available during ``keystone-manage bootstrap``, which is typically done during installation and sometimes on upgrade. Pre-Rocky, only the ``admin`` role existed in keystone.
The idea is to make it easier for other services, like cinder, to rely on these roles by default. As opposed to cinder changing a policy default to something like ``role:reader`` and another service using something like ``role:auditor``, but they both mean the same thing. We want to avoid the case where operators would need to make sure both roles exist in keystone and users have both if they're expected to perform read-only operations from each service.
The work keystone did in Rocky made it possible for cinder to start making those assumptions about the existence of roles called ``member`` and ``reader``.
As far as testing goes, I remember there being a cinder session in Denver about policy changes, regression, and making sure we can protect against it happening, especially late in the release. Ideally, it would be fantastic to have protection tests in each service that assert the default policies work as expected for protecting APIs. Unfortunately, testing this in its entirety makes it an integration test that requires keystone. Luckily, there are some ways for services to expand protection testing without having that dependency, which falls within the parameters of unit tests. I proposed a couple of changes to cinder that show how to do this [1], and it requires modeling an API request similar to what keystonemiddleware/oslo.context do and ensuring the actual defaults are tested by removing overridden test policies [2] that leave APIs unprotected. We attempted to document a generalized approach to this sort of testing in oslo.policy [3].