<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 10, 2022 at 10:13 AM Ghanshyam Mann <<a href="mailto:gmann@ghanshyammann.com">gmann@ghanshyammann.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> ---- On Wed, 09 Mar 2022 17:14:02 -0600 Takashi Kajinami <<a href="mailto:tkajinam@redhat.com" target="_blank">tkajinam@redhat.com</a>> wrote ----<br>
> <br>
> <br>
> On Thu, Mar 10, 2022 at 2:59 AM Ghanshyam Mann <<a href="mailto:gmann@ghanshyammann.com" target="_blank">gmann@ghanshyammann.com</a>> wrote:<br>
> ---- On Wed, 09 Mar 2022 06:43:03 -0600 Takashi Kajinami <<a href="mailto:tkajinam@redhat.com" target="_blank">tkajinam@redhat.com</a>> wrote ----<br>
> > Hi Rico.<br>
> > Thank you for your insights.Please find my replies inline.<br>
> > <br>
> > On Wed, Mar 9, 2022 at 6:22 PM Rico Lin <<a href="mailto:ricolin@ricolky.com" target="_blank">ricolin@ricolky.com</a>> wrote:<br>
> > <br>
> > <br>
> > On Wed, Mar 9, 2022 at 1:57 PM Takashi Kajinami <<a href="mailto:tkajinam@redhat.com" target="_blank">tkajinam@redhat.com</a>> wrote:<br>
> > ><br>
> > > Hello,<br>
> > ><br>
> > ><br>
> > > I've been working on updating policy rules in heat according to the latest<br>
> > > guideline for SRBAC implementation.<br>
> > <br>
> > Thanks so much for working on it, definitely +1 on enhancing security.<br>
> > ><br>
> > > [1] <a href="https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change" rel="noreferrer" target="_blank">https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change</a><br>
> > ><br>
> > > However this change reveals one potential issue with system resources.<br>
> > ><br>
> > > In heat there are some resource types like OS::Nova::Flavor or OS::Keystone::User<br>
> > > which is allowed for only system users. (I'll call these resources as "system resources" here)<br>
> > We create those resources based on checking `role:admin`, so project admin is accepted.This means a user under that project with role:admin, can operate those resources.I'm not sure what system user means here, so correct me if I'm wrong.<br>
> > The core problem is that backend services like nova enforces different policies from just role:admin whenSRBAC is enforced. I reported that a while ago as <a href="https://storyboard.openstack.org/#!/story/2009787" rel="noreferrer" target="_blank">https://storyboard.openstack.org/#!/story/2009787</a> .<br>
> > <br>
> > The main problem here is that heat currently supports resources which require different scopes of credentials.For example, once all deprecated policy rules are removed, nova allows project users to create instances(OS::Nova::Server)[1],while nova allows only system admins to create flavors(OS::Nova::Flavor)[2]. Currently Heat leverages the user credentialused to create a stack when Heat requests backend services to create its resources, and it's "impossible" to create a stackwith these two resources by a single credential (as each token can be scoped by project or system.<br>
> > [1] <a href="https://github.com/openstack/nova/blob/d1b036fdb124d1ef48f5cf024d4677bfe5b481d7/nova/policies/servers.py#L210[2]" rel="noreferrer" target="_blank">https://github.com/openstack/nova/blob/d1b036fdb124d1ef48f5cf024d4677bfe5b481d7/nova/policies/servers.py#L210[2]</a> <a href="https://github.com/openstack/nova/blob/d1b036fdb124d1ef48f5cf024d4677bfe5b481d7/nova/policies/flavor_manage.py#L28-L36" rel="noreferrer" target="_blank">https://github.com/openstack/nova/blob/d1b036fdb124d1ef48f5cf024d4677bfe5b481d7/nova/policies/flavor_manage.py#L28-L36</a><br>
> > If we keep allowing system users to create stacks, then it would allow users to create system resourceslike flavors. But this never solves the problem with the "mixed" pattern which I described above.Hope this will explain these three options more clearly.<br>
> > <br>
> > ><br>
> > > Because heat uses user credentials to create resources in backend services, if we require<br>
> > > project scope for stack/resource management then we are no longer able to create these system<br>
> > > resources as part of stack. (*1)><br>
> > > There are some options I can think of at this moment.<br>
> > ><br>
> > > 1. Deprecate and remove all resources which require system scope<br>
> > > Pros<br>
> > > - This is most easy solution to be implemented<br>
> > > Cons<br>
> > > - This one has the biggest user impact<br>
> > <br>
> > Don't :)<br>
> <br>
> Yeah, not sure how removing will work.<br>
> <br>
> > > 2. Use service credential for resource creation<br>
> > > Pros:<br>
> > > - This might have smallest user impact<br>
> > > Cons:<br>
> > > - This requires a relatively big change in heat's architecture, which I've not yet evaluated.<br>
> > > - We need to determine the project scope role to be allowed to do system operation.<br>
> > > <br>
> > I think this is something we can discuss more.<br>
> > > 3. Implement additional project-role in each service to allow creation via stack creation.<br>
> > > Pros:<br>
> > > - This would avoid user impact.<br>
> > > Cons:<br>
> > > - requires work in multiple components.<br>
> > > - We need to determine the project scope role to be allowed to do system operation<br>
> > <br>
> > <br>
> > Indeed current stack operation require `(role:admin and system_scope:all) OR (role:member and project_id:%(project_id)s)`.And if we remove the system part and keep `(role:member and project_id:%(project_id)s)`, will mean only a user in that project with a member role can operate stack, and if the user got also admin role, that user can manage User/Flavor resources through stack too.The main problem here is that nova will no longer allow project admins to create flavors,and it will require system admin when SRBAC is enforced(which means all deprecated rulesare disabled)<br>
> > <br>
> > Also, note thatFor the case of Nova::Flavor, both policies from Nova and Heat resource will be enforced here, so it only accepts the intersection part of rules.<br>
> > So if we go with removing the system admin, that means it's time to also remove some deprecated global scope operation.<br>
> > IMO, to accept Project Admin or member make more reasonable (like you proposed in option 3)<br>
> <br>
> This is a similar case that we discussed in the service role ML thread[1] or in the spec[2].<br>
> <br>
> For such cases only, where services need to interact with another service we will have the 'service' as<br>
> a special role. In this case, the heat having the project scoped token will use the 'service' role token for<br>
> Nova Flavor APIs calls. Now whether that 'service' role token needs to be system scoped or not is<br>
> open question Mark brought up in that discussion. IMO, we should use the scoped token even in case<br>
> of 'service' role and do not skip scope checks for 'service' role in oslo policy as it can be a valid role used<br>
> by any users at production. So with the new SRBAC, heat needs to use the system scoped 'service' role token.<br>
> <br>
> But as 'service' role is not yet implemented by keystone[2], we can use the legacy admin token which is currently<br>
> used in heat or other service-to-service interaction. <br>
> <br>
> Thanks gmann for inputs.<br>
> HoweverI think we still need a different role here.<br>
> The role I've mentioned here is not the role assigned to the Heat service user but the project-roleassigned to users to allow them create stacks with system resources like nova flavor or keystone user.This role is assigned to actual cloud users so service would not be appropriate.<br>
> <br>
> I thought of using the admin role initially but this means that we provide a way to "bypass" policy checkand expose system operations to project users.<br>
> <br>
> As a side note, current heat does not use its own credential but user credentials used for stack API calls. <br>
> We can introduce this mechanism and let heat manage system resources by its service user but that isthe completely new feature we need to implement.<br>
<br>
Thanks for the detailed information. For service to service interaction, I think heat needs to start using the configured<br>
service user token instead of using actual user token.<br>
<br>
For example nova use for neutron and similarly for other service interaction<br>
- <a href="https://zuul.opendev.org/t/openstack/build/654ccb626d0e41a69308db1ed3328824/log/controller/logs/etc/nova/nova_conf.txt#40-51" rel="noreferrer" target="_blank">https://zuul.opendev.org/t/openstack/build/654ccb626d0e41a69308db1ed3328824/log/controller/logs/etc/nova/nova_conf.txt#40-51</a><br></blockquote><div><br></div><div>I'll look into that option but It might take some time because I've not yet evaluated the impact of that change.</div><div>My current idea is to use service credentials for system resources only so that we can keep using the user credential</div><div>for project resources. <br></div><div><br></div><div>As heat allows only project users to manage stacks, we need to determine the project role to allow creating</div><div>stacks with such system resources. Does anyone have any suggestions about this ?</div><div><br></div><div>Do we agree we can allow project-admin to create stacks with flavors or keystone users as the previous policy rules</div><div>allow or should we implement a new role to restrict management of system resources ?<br></div><div><br></div></div><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
-gmann<br>
<br>
> [1] <a href="http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026777.html" rel="noreferrer" target="_blank">http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026777.html</a><br>
> [2] <a href="https://review.opendev.org/c/openstack/keystone-specs/+/818616" rel="noreferrer" target="_blank">https://review.opendev.org/c/openstack/keystone-specs/+/818616</a><br>
> <br>
> -gmann<br>
> <br>
> > ><br>
> > > Thank you,<br>
> > > Takashi<br>
> > ><br>
> > <br>
> <br>
> <br>
<br>
</blockquote></div></div>