Hey all, I want to follow up on this thread because there's been some discussion and questions (some of which are in reviews) as services work through the proposed changes [0]. TL;DR - OpenStack services implementing secure RBAC should update default policies with the `reader` role in a consistent manner, where it is not meant to protect sensitive information. In the process of reviewing changes for various resources, some folks raised concerns about the `reader` role definition. One of the intended use-cases for implementing a `reader` role was to use it for auditing, as noted in the keystone definitions for each role and persona [1]. Another key point of that document, and the underlying design of secure RBAC, is that the default roles have role implications built between them (e.g., reader implies member, and member implies admin). This detail serves two important functions. First, it reduces duplication in check strings because keystone expands role implications in token response bodies. For example, someone with the `admin` role on a project will have `member` and `reader` roles in their token body when they authenticate for a token or validate a token. This reduces the complexity of our check strings by writing the policy to the highest level of authorization required to access an API or resource. Users with anything above that level will work through the role implications feature. Second, it reduces the need for extra role assignments. If you grant someone the `admin` role on a project you don't need to also give them `reader` and `member` role assignments. This is true regardless of how services implement check strings. Ultimately, the hierarchical role structure in keystone and role expansion in token responses give us shorter check strings and less role assignments. But, one thing we're aware of now is that we need to be careful how we expose certain information to users via the `reader` role, since it is the least-privileged role in the hierarchy. For example, one concern was exposing license key information in images to anyone with the `reader` role on the system. Some deployments, depending on their security posture or auditing targets, might not allow sensitive information to be implicitly exposed. Instead, they may require deployments to explicitly grant access to sensitive information [2]. So what do we do moving forward? I think it's clear that there are APIs and resources in OpenStack that fall into a special category where we shouldn't expose certain information to the lowest level of the role hierarchy, regardless of the scope. But, the role implication functionality served a purpose initially to deliver a least-privileged role used only for read operations within a given scope. I think breaking that implication now is confusing considering we implemented the implication in Rocky [3], but I think future work for an elevated read-only role is a good path forward. Eventually, keystone can consider implementing support for a new default role, which implies `reader`, making all the work we do today still useful. At that time, we can update relevant policies to expose sensitive information with the elevated read-only role. I suspect this will be a much smaller set of APIs and policies. I think this approach strikes a balance between what we have today, and a way to move forward that still protects sensitive data. I proposed an update to the documentation in keystone to clarify this point [4]. It also doesn't assume all audits are the same. Instead, it phrases the ability to use `reader` roles for auditing in a way that leaves that up to the deployer and auditor. I think that's an important detail since different deployments have different security requirements. Instead of assuming everyone can use `reader` for auditing, we can give them a list of APIs they can interact with as a `reader` (or have them generate those policies themselves, especially if they have custom policy) and let them determine if that access is sufficient for their audit. If it isn't, deployers aren't in a worse position today, but it emphasizes the importance of expanding the default roles to include another tier for elevated read-only permissions. Given where we are in the release cycle for Wallaby, I don't expect keystone to implement a new default role this late in the release [5]. Perhaps Xena is a better target, but I'll talk with Kristi about it next week during the keystone meeting. I hope this helps clarify some of the confusion around the secure RBAC patches. If you have additional comments or questions about this topic, let me know. We can obviously iterate here, or use the policy pop up time slot which is in a couple of days [6]. Thanks, Lance [0] https://review.opendev.org/q/topic:secure-rbac [1] https://docs.openstack.org/keystone/latest/admin/service-api-protection.html [2] FedRAMP control AC -06 (01) is an example of this - *The organization explicitly authorizes access to [Assignment: organization-defined security functions (deployed in hardware, software, and firmware) and security-relevant information].* [3] https://docs.openstack.org/releasenotes/keystone/rocky.html#new-features [4] https://review.opendev.org/c/openstack/keystone/+/771509 [5] https://releases.openstack.org/wallaby/schedule.html [6] https://etherpad.opendev.org/p/default-policy-meeting-agenda On Thu, Dec 10, 2020 at 7:15 PM Ghanshyam Mann <gmann@ghanshyammann.com> wrote:

...

Hey everyone,

I wanted to take an opportunity to clarify some work we have been doing upstream, specifically modifying the default policies across projects.

These changes are the next phase of an initiative that’s been underway since Queens to fix some long-standing security concerns in OpenStack [0]. For context, we have been gradually improving policy enforcement for years. We started by improving policy formats, registering default policies into code [1], providing better documentation for policy writers, implementing necessary identity concepts in keystone [2], developing support for those concepts in libraries [3][4][5][6][7][8], and consuming all of those changes to provide secure default policies in a way operators can consume and roll out to their users [9][10].

All of this work is in line with some high-level documentation we started writing about three years ago [11][12][13].

There are a handful of services that have implemented the goals that define secure RBAC by default, but a community-wide goal is still out-of-reach. To help with that, the community formed a pop-up team with a focused objective and disbanding criteria [14].

The work we currently have in progress [15] is an attempt to start applying what we have learned from existing implementations to other

---- On Wed, 09 Dec 2020 14:04:57 -0600 Lance Bragstad < lbragstad@gmail.com> wrote ---- projects. The hope is that we can complete the work for even more projects in Wallaby. Most deployers looking for this functionality won't be able to use it effectively until all services in their deployment support it.

Thanks, Lance for pushing this work forwards. I completely agree and that is what we get feedback in forum sessions also that we should implement this in all the services first before we ask operators to move their cloud to the new RBAC.

We discussed these in today's policy-popup meeting also and encourage every project to help in those patches to add tests and review. This will help to finish the work on priority and we can provide better RBAC experience to the deployer.

-gmann

...

I hope this helps clarify or explain the patches being proposed.

As always, I'm happy to elaborate on specific concerns if folks have

them.

...

Thanks,

Lance

[0] https://bugs.launchpad.net/keystone/+bug/968696/ [1]

https://governance.openstack.org/tc/goals/selected/queens/policy-in-code.htm...

...

[2] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/s... [3] https://review.opendev.org/c/openstack/keystoneauth/+/529665 [4] https://review.opendev.org/c/openstack/python-keystoneclient/+/524415 [5] https://review.opendev.org/c/openstack/oslo.context/+/530509 [6] https://review.opendev.org/c/openstack/keystonemiddleware/+/564072 [7] https://review.opendev.org/c/openstack/oslo.policy/+/578995 [8] https://review.opendev.org/q/topic:%22system-scope%22+(status:open%20OR%20st...) [9] https://review.opendev.org/q/status:merged+topic:bp/policy-defaults-refresh+... [10] https://review.opendev.org/q/topic:%22implement-default-roles%22+(status:ope...) [11] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/... [12] https://docs.openstack.org/keystone/latest/admin/service-api-protection.html [13] https://docs.openstack.org/keystone/latest/contributor/services.html#authori... [14] https://governance.openstack.org/tc/reference/popup-teams.html#secure-defaul... [15] https://review.opendev.org/q/topic:%2522secure-rbac%2522+status:open

On 19/01, Lance Bragstad wrote:

Hey all,

I want to follow up on this thread because there's been some discussion and questions (some of which are in reviews) as services work through the proposed changes [0].

TL;DR - OpenStack services implementing secure RBAC should update default policies with the `reader` role in a consistent manner, where it is not meant to protect sensitive information.

Hi Lance, Thank you very much for this great summary of all the different discussions and decisions. I have just one question regarding your TL;DR, shouldn't it be "where it is meant to protect sensitive information"? As I understood it the reader should be updated so it doesn't expose sensitive information (thus protecting it) because it's the least privileged role. Cheers, Gorka.

In the process of reviewing changes for various resources, some folks raised concerns about the `reader` role definition.

One of the intended use-cases for implementing a `reader` role was to use it for auditing, as noted in the keystone definitions for each role and persona [1]. Another key point of that document, and the underlying design of secure RBAC, is that the default roles have role implications built between them (e.g., reader implies member, and member implies admin). This detail serves two important functions.

First, it reduces duplication in check strings because keystone expands role implications in token response bodies. For example, someone with the `admin` role on a project will have `member` and `reader` roles in their token body when they authenticate for a token or validate a token. This reduces the complexity of our check strings by writing the policy to the highest level of authorization required to access an API or resource. Users with anything above that level will work through the role implications feature.

Second, it reduces the need for extra role assignments. If you grant someone the `admin` role on a project you don't need to also give them `reader` and `member` role assignments. This is true regardless of how services implement check strings.

Ultimately, the hierarchical role structure in keystone and role expansion in token responses give us shorter check strings and less role assignments. But, one thing we're aware of now is that we need to be careful how we expose certain information to users via the `reader` role, since it is the least-privileged role in the hierarchy. For example, one concern was exposing license key information in images to anyone with the `reader` role on the system. Some deployments, depending on their security posture or auditing targets, might not allow sensitive information to be implicitly exposed. Instead, they may require deployments to explicitly grant access to sensitive information [2].

So what do we do moving forward?

I think it's clear that there are APIs and resources in OpenStack that fall into a special category where we shouldn't expose certain information to the lowest level of the role hierarchy, regardless of the scope. But, the role implication functionality served a purpose initially to deliver a least-privileged role used only for read operations within a given scope. I think breaking that implication now is confusing considering we implemented the implication in Rocky [3], but I think future work for an elevated read-only role is a good path forward. Eventually, keystone can consider implementing support for a new default role, which implies `reader`, making all the work we do today still useful. At that time, we can update relevant policies to expose sensitive information with the elevated read-only role. I suspect this will be a much smaller set of APIs and policies. I think this approach strikes a balance between what we have today, and a way to move forward that still protects sensitive data.

I proposed an update to the documentation in keystone to clarify this point [4]. It also doesn't assume all audits are the same. Instead, it phrases the ability to use `reader` roles for auditing in a way that leaves that up to the deployer and auditor. I think that's an important detail since different deployments have different security requirements. Instead of assuming everyone can use `reader` for auditing, we can give them a list of APIs they can interact with as a `reader` (or have them generate those policies themselves, especially if they have custom policy) and let them determine if that access is sufficient for their audit. If it isn't, deployers aren't in a worse position today, but it emphasizes the importance of expanding the default roles to include another tier for elevated read-only permissions. Given where we are in the release cycle for Wallaby, I don't expect keystone to implement a new default role this late in the release [5]. Perhaps Xena is a better target, but I'll talk with Kristi about it next week during the keystone meeting.

I hope this helps clarify some of the confusion around the secure RBAC patches. If you have additional comments or questions about this topic, let me know. We can obviously iterate here, or use the policy pop up time slot which is in a couple of days [6].

Thanks,

Lance

[0] https://review.opendev.org/q/topic:secure-rbac [1] https://docs.openstack.org/keystone/latest/admin/service-api-protection.html [2] FedRAMP control AC -06 (01) is an example of this - *The organization explicitly authorizes access to [Assignment: organization-defined security functions (deployed in hardware, software, and firmware) and security-relevant information].* [3] https://docs.openstack.org/releasenotes/keystone/rocky.html#new-features [4] https://review.opendev.org/c/openstack/keystone/+/771509 [5] https://releases.openstack.org/wallaby/schedule.html [6] https://etherpad.opendev.org/p/default-policy-meeting-agenda

On Thu, Dec 10, 2020 at 7:15 PM Ghanshyam Mann <gmann@ghanshyammann.com> wrote:

...

...

Hey everyone,

I wanted to take an opportunity to clarify some work we have been doing upstream, specifically modifying the default policies across projects.

These changes are the next phase of an initiative that’s been underway since Queens to fix some long-standing security concerns in OpenStack [0]. For context, we have been gradually improving policy enforcement for years. We started by improving policy formats, registering default policies into code [1], providing better documentation for policy writers, implementing necessary identity concepts in keystone [2], developing support for those concepts in libraries [3][4][5][6][7][8], and consuming all of those changes to provide secure default policies in a way operators can consume and roll out to their users [9][10].

All of this work is in line with some high-level documentation we started writing about three years ago [11][12][13].

There are a handful of services that have implemented the goals that define secure RBAC by default, but a community-wide goal is still out-of-reach. To help with that, the community formed a pop-up team with a focused objective and disbanding criteria [14].

The work we currently have in progress [15] is an attempt to start applying what we have learned from existing implementations to other

---- On Wed, 09 Dec 2020 14:04:57 -0600 Lance Bragstad < lbragstad@gmail.com> wrote ---- projects. The hope is that we can complete the work for even more projects in Wallaby. Most deployers looking for this functionality won't be able to use it effectively until all services in their deployment support it.

Thanks, Lance for pushing this work forwards. I completely agree and that is what we get feedback in forum sessions also that we should implement this in all the services first before we ask operators to move their cloud to the new RBAC.

We discussed these in today's policy-popup meeting also and encourage every project to help in those patches to add tests and review. This will help to finish the work on priority and we can provide better RBAC experience to the deployer.

-gmann

...

I hope this helps clarify or explain the patches being proposed.

As always, I'm happy to elaborate on specific concerns if folks have

them.

...

Thanks,

Lance

[0] https://bugs.launchpad.net/keystone/+bug/968696/ [1]

https://governance.openstack.org/tc/goals/selected/queens/policy-in-code.htm...

...

[2] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/s... [3] https://review.opendev.org/c/openstack/keystoneauth/+/529665 [4] https://review.opendev.org/c/openstack/python-keystoneclient/+/524415 [5] https://review.opendev.org/c/openstack/oslo.context/+/530509 [6] https://review.opendev.org/c/openstack/keystonemiddleware/+/564072 [7] https://review.opendev.org/c/openstack/oslo.policy/+/578995 [8] https://review.opendev.org/q/topic:%22system-scope%22+(status:open%20OR%20st...) [9] https://review.opendev.org/q/status:merged+topic:bp/policy-defaults-refresh+... [10] https://review.opendev.org/q/topic:%22implement-default-roles%22+(status:ope...) [11] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/... [12] https://docs.openstack.org/keystone/latest/admin/service-api-protection.html [13] https://docs.openstack.org/keystone/latest/contributor/services.html#authori... [14] https://governance.openstack.org/tc/reference/popup-teams.html#secure-defaul... [15] https://review.opendev.org/q/topic:%2522secure-rbac%2522+status:open

On 26/01, Pete Zaitcev wrote:

On Tue, 26 Jan 2021 11:22:45 +0100 Gorka Eguileor <geguileo@redhat.com> wrote:

...

On 19/01, Lance Bragstad wrote:

...

...

TL;DR - OpenStack services implementing secure RBAC should update default policies with the `reader` role in a consistent manner, where it is not meant to protect sensitive information.

...

I have just one question regarding your TL;DR, shouldn't it be "where it is meant to protect sensitive information"?

As I understood it the reader should be updated so it doesn't expose sensitive information (thus protecting it) because it's the least privileged role.

I think Lance means that the reader role can see what member sees (in any scope - system, domain, project). If a member sees some sensitive information, then reader is not a place to make access reduction. In fact it's harmful, because reader is not emulating member sufficiently for being useful for anything. At least that's how I read Lance's intent.

Hi Pete, What I understood was that the reader wouldn't be able to see any sensitive information, not even from a user. Erno brought up a specific case with Glance and how a user may be able to see their licenses but a reader shouldn't. So either I'm misunderstanding something or Erno's concerns won't be addressed with our current approach. Cheers, Gorka.

...

...

I think it's clear that there are APIs and resources in OpenStack that fall into a special category where we shouldn't expose certain information to the lowest level of the role hierarchy, regardless of the scope. But, the role implication functionality served a purpose initially to deliver a least-privileged role used only for read operations within a given scope. I think breaking that implication now is confusing considering we implemented the implication in Rocky [3], but I think future work for an elevated read-only role is a good path forward.

That's not going to do a whole lot to simplify those check strings. Perfect example of kicking the can down the road and committing to more technical baggage.

-- Pete

On Tue, Jan 26, 2021 at 4:23 AM Gorka Eguileor <geguileo@redhat.com> wrote:

On 19/01, Lance Bragstad wrote:

...

Hey all,

I want to follow up on this thread because there's been some discussion and questions (some of which are in reviews) as services work through the proposed changes [0].

TL;DR - OpenStack services implementing secure RBAC should update default policies with the `reader` role in a consistent manner, where it is not meant to protect sensitive information.

Hi Lance,

Thank you very much for this great summary of all the different discussions and decisions.

I have just one question regarding your TL;DR, shouldn't it be "where it is meant to protect sensitive information"?

As I understood it the reader should be updated so it doesn't expose sensitive information (thus protecting it) because it's the least privileged role.

Cheers, Gorka.

I think you're understanding things correctly, and re-reading my original TL;DR I can see how the wording is confusing. Does the following help? *OpenStack services implementing secure RBAC should update default policies with the `reader` role in a consistent manner, where users with the `reader` role are not allowed to view sensitive information.* Nice catch, Gorka.

...

In the process of reviewing changes for various resources, some folks raised concerns about the `reader` role definition.

One of the intended use-cases for implementing a `reader` role was to use it for auditing, as noted in the keystone definitions for each role and persona [1]. Another key point of that document, and the underlying design of secure RBAC, is that the default roles have role implications built between them (e.g., reader implies member, and member implies admin). This detail serves two important functions.

First, it reduces duplication in check strings because keystone expands role implications in token response bodies. For example, someone with the `admin` role on a project will have `member` and `reader` roles in their token body when they authenticate for a token or validate a token. This reduces the complexity of our check strings by writing the policy to the highest level of authorization required to access an API or resource. Users with anything above that level will work through the role implications feature.

Second, it reduces the need for extra role assignments. If you grant someone the `admin` role on a project you don't need to also give them `reader` and `member` role assignments. This is true regardless of how services implement check strings.

Ultimately, the hierarchical role structure in keystone and role expansion in token responses give us shorter check strings and less role assignments. But, one thing we're aware of now is that we need to be careful how we expose certain information to users via the `reader` role, since it is the least-privileged role in the hierarchy. For example, one concern was exposing license key information in images to anyone with the `reader` role on the system. Some deployments, depending on their security posture or auditing targets, might not allow sensitive information to be implicitly exposed. Instead, they may require deployments to explicitly grant access to sensitive information [2].

So what do we do moving forward?

I think it's clear that there are APIs and resources in OpenStack that fall into a special category where we shouldn't expose certain information to the lowest level of the role hierarchy, regardless of the scope. But, the role implication functionality served a purpose initially to deliver a least-privileged role used only for read operations within a given scope. I think breaking that implication now is confusing considering we implemented the implication in Rocky [3], but I think future work for an elevated read-only role is a good path forward. Eventually, keystone can consider implementing support for a new default role, which implies `reader`, making all the work we do today still useful. At that time, we can update relevant policies to expose sensitive information with the elevated read-only role. I suspect this will be a much smaller set of APIs and policies. I think this approach strikes a balance between what we have today, and a way to move forward that still protects sensitive data.

I proposed an update to the documentation in keystone to clarify this point [4]. It also doesn't assume all audits are the same. Instead, it phrases the ability to use `reader` roles for auditing in a way that leaves that up to the deployer and auditor. I think that's an important detail since different deployments have different security requirements. Instead of assuming everyone can use `reader` for auditing, we can give them a list of APIs they can interact with as a `reader` (or have them generate those policies themselves, especially if they have custom policy) and let them determine if that access is sufficient for their audit. If it isn't, deployers aren't in a worse position today, but it emphasizes the importance of expanding the default roles to include another tier for elevated read-only permissions. Given where we are in the release cycle for Wallaby, I don't expect keystone to implement a new default role this late in the release [5]. Perhaps Xena is a better target, but I'll talk with Kristi about it next week during the keystone meeting.

I hope this helps clarify some of the confusion around the secure RBAC patches. If you have additional comments or questions about this topic, let me know. We can obviously iterate here, or use the policy pop up time slot which is in a couple of days [6].

Thanks,

Lance

[0] https://review.opendev.org/q/topic:secure-rbac [1]

...

[2] FedRAMP control AC -06 (01) is an example of this - *The organization explicitly authorizes access to [Assignment: organization-defined security functions (deployed in hardware, software, and firmware) and security-relevant information].* [3] https://docs.openstack.org/releasenotes/keystone/rocky.html#new-features [4] https://review.opendev.org/c/openstack/keystone/+/771509 [5] https://releases.openstack.org/wallaby/schedule.html [6] https://etherpad.opendev.org/p/default-policy-meeting-agenda

On Thu, Dec 10, 2020 at 7:15 PM Ghanshyam Mann <gmann@ghanshyammann.com> wrote:

...

---- On Wed, 09 Dec 2020 14:04:57 -0600 Lance Bragstad < lbragstad@gmail.com> wrote ----

...

Hey everyone,

I wanted to take an opportunity to clarify some work we have been doing upstream, specifically modifying the default policies across projects.

These changes are the next phase of an initiative that’s been underway since Queens to fix some long-standing security concerns in OpenStack [0]. For context, we have been gradually improving policy enforcement for years. We started by improving policy formats, registering default policies into code [1], providing better documentation for policy writers, implementing necessary identity concepts in keystone [2], developing support for

...

...

concepts in libraries [3][4][5][6][7][8], and consuming all of those changes to provide secure default policies in a way operators can consume and roll out to their users [9][10].

...

All of this work is in line with some high-level documentation we

started writing about three years ago [11][12][13].

...

There are a handful of services that have implemented the goals that

define secure RBAC by default, but a community-wide goal is still out-of-reach. To help with that, the community formed a pop-up team with a focused objective and disbanding criteria [14].

...

The work we currently have in progress [15] is an attempt to start

applying what we have learned from existing implementations to other projects. The hope is that we can complete the work for even more

...

...

in Wallaby. Most deployers looking for this functionality won't be able to use it effectively until all services in their deployment support it.

Thanks, Lance for pushing this work forwards. I completely agree and

https://docs.openstack.org/keystone/latest/admin/service-api-protection.html those projects that

...

...

is what we get feedback in forum sessions also that we should implement this in all the services first before we ask operators to move their cloud to the new RBAC.

We discussed these in today's policy-popup meeting also and encourage every project to help in those patches to add tests and review. This will help to finish the work on priority and we can provide better RBAC experience to the deployer.

-gmann

...

I hope this helps clarify or explain the patches being proposed.

As always, I'm happy to elaborate on specific concerns if folks have

them.

...

Thanks,

Lance

[0] https://bugs.launchpad.net/keystone/+bug/968696/ [1]

https://governance.openstack.org/tc/goals/selected/queens/policy-in-code.htm...

...

...

[2]

https://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/s...

...

...

[3] https://review.opendev.org/c/openstack/keystoneauth/+/529665 [4] https://review.opendev.org/c/openstack/python-keystoneclient/+/524415 [5] https://review.opendev.org/c/openstack/oslo.context/+/530509 [6] https://review.opendev.org/c/openstack/keystonemiddleware/+/564072 [7] https://review.opendev.org/c/openstack/oslo.policy/+/578995 [8]

https://review.opendev.org/q/topic:%22system-scope%22+(status:open%20OR%20st...)

...

...

[9]

https://review.opendev.org/q/status:merged+topic:bp/policy-defaults-refresh+...

...

...

[10]

https://review.opendev.org/q/topic:%22implement-default-roles%22+(status:ope...)

...

...

[11]

https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/...

...

...

[12]

https://docs.openstack.org/keystone/latest/admin/service-api-protection.html

...

...

[13]

https://docs.openstack.org/keystone/latest/contributor/services.html#authori...

...

...

[14]

https://governance.openstack.org/tc/reference/popup-teams.html#secure-defaul...

...

...

[15] https://review.opendev.org/q/topic:%2522secure-rbac%2522+status:open