Open Stack

On 06/04/2015 05:49 PM, Morgan Fainberg wrote:
> Hi Everyone!
>
> I've been reading through this thread and have had some conversations 
> along the side and wanted to jump in to distill out what I think are 
> the key points we are trying to address here. I'm going to outline 
> about 4 items that seem to make sense to me regarding the evolution of 
> policy. I also want to say that the "notification" that something has 
> changed from the defaults in a way that may cause odd behavior to the 
> side (the warning Sean was outlining); we can keep the desire to have 
> those types of warnings for operators down the line (nothing that is 
> being proposed here or what I'm going to outline will make it more or 
> less difficult to add the functionality later on). This is not to say 
> we wouldn't provide validation of an override, but a subjective "this 
> is a problematic policy configuration" doesn't need to be directly 
> part of this conversation today (it can happen once we know what the 
> model of policy looks like going forward).
>
> 1. The first thing that I'm hearing from the conversation (this is 
> based upon Sean's proposal) is that we already trust the individual 
> projects to know the enforcement profile for their resources. It seems 
> like the project should be authoritative on what that enforcement 
> should look like. Handing off the enforcement definition to Keystone 
> is the wrong direction. I really like the concept of defining within 
> Nova the default policy that nova works with (we do this already 
> today, and I don't want to require the nova team to come to Keystone 
> to make changes to the policy down the line). The Projects are trusted 
> to know what the enforcement points are and trusted to distribute a 
> basic profile of enforcement.
>
> To the end that the enforcement definition is handled by the 
> individual projects, making it something that is more than a "blob of 
> text" also makes a lot of sense. A code-like model that is easier to 
> understand for the developers that are implementing enforcement would 
> be useful. The key pieces are that this code-like-construct must be 
> able to be serialized out into the "common" format.
The policy file format is JSON, which is the standard for all of the 
APIs in OpenStack thus far.

> Second, this code-construct is just the basic level of "truth", the 
> idea is that the dynamic policy will provide the overrides - and 
> *everything* can be overridden.  The code-like construct will also aid 
> in profiling/testing the base/defaults (and then the dynamic policy 
> overrides) without having to standup the entire stack. We can enable 
> base functionality testing / validation and then the more integrated 
> testing with the full stack (in different environments). This will 
> enable more accurate and better base policy development by the teams 
> we already trust to build the enforcement for a given project (e.g. Nova).
We already have this.  It is the default policy.json that each project 
keeps up to date.

Please don't suggest putting annotations on the coder and running a 
preprocessor.  That way leads to madness.  I see no reason to have a 
team write  policy in Python and then serialize to JSON.

> --

The real current problem we have is this:  On a given API, we don't know 
where to look for the project ID and (almost all) policy needs to be 
enforced on the project scope.  What is required is for the base 
repository to have a document of how to set up the scoping for the call 
(token.proiject.id must match fetched_object.tenant_id), and we could 
mark that as "dangerous to change".  What I would not want is to have 
the project hardcode the Role required.  Perhaps the API indicates one 
of two levels:  Admin vs Member, on an API, indicating the expected 
consumer of the API.  However, the current Policy file format represents 
this sufficiently.  We just need the Nova team to stay on top of this 
for Nova, and the other teams for their projects.

What I would love to be able to get from Nova is "this api will end up 
calling these apis in Glance, Cinder, and neutron."  So we can properly 
delegate.


>
> 2. We will need a way to handle the bi-directional notifications for 
> policy changes. This would encompass when a project is restarted and 
> has a new code-policy construct and how that gets to Keystone. We also 
> need to ensure that changes to the overrides are pushed down to the 
> projects. This is not an easy canned solution today, but I am sure we 
> can solve it. Likely this is tied to Keystone Middleware or something 
> similar (I believe code is already in the works to this end).

I appreciate that Nova is growing, but I would not expect huge amounts 
of new policy code with each updtate...at this point, new policy would 
be "deny until there is a new rule to handle it" and "upload these new 
rules".  Upgrading code is already part of an organizations deployment 
stategy, and trying to build in an additional two way notification 
system is more than we should be doing here.

If Nova feels that each micro version needs its own delta to the policy 
file, it can produce them, but I would be wary of just blanket 
overriding what the operators have in production.

> --
>
> 3. The HA-Proxy mode of deployment with projects that can handle 
> no-downtime upgrades mean that we need to add in versioning into the 
> policy structures. The policy files for a "Kilo" vintage of Nova may 
> (likely) will be incompatible with "Liberty" nova. This means we 
> cannot assume that policy can be centralized easily even for a 
> specific grouping of api-services running as a single endpoint. This 
> becomes an even more important mechanism as we move towards more and 
> more services with microversioned APIs. It means it is totally 
> reasonable to upgrade 1 or 2 nova APIs behind an HA Proxy since the 
> new APIs will handle the old microversion of the API.
This does not correspond with what I have seen from the policy files 
thus far.  If microversions are going to require significantly different 
policy, they need to fetch rules from the policy file. APIs themselves 
do not change that much.  When Keystone went from V2 to V3, it underwent 
a radical rewriting of the strucutre of the data, but those are handled 
by different API calls, and thus different policy calls.
>
> This leads to needing policy to likewise be versioned. This also means 
> that only the service can be authoritative with the base-policy 
> construct. This means whatever tool we use for handling the overrides 
> on the Keystone side will need to be aware of policy versions as well. 
> Having Keystone side being exclusively authoratative for the entire 
> policy makes development, testing, and understanding of policy harder. 
> This is another case of the project itself should be in control of the 
> base policy definition.
Again, this does not correspond with what I have seen in the current 
policy files.  We've been locked to static policy for as long as we've 
had policy in OpenStack.  We finally have the opportunity to move 
forward.  A default rule of "deny until we have a viable policy"  makes 
far more sense than trying to automatically merge in new policies when 
code updates.

Since the policy files replace existing rules, it would make more sense 
for Nova to put a new micro-version policy file with just the new rules 
into its repository.   We could make one option on the database  be that 
when an update comes in, reject it if it attempts to overwrite any 
existing rules, and use that for loading deltas. That way, we can append 
new policy.json files to existing ones without breaking an existing 
deployment.

Yes, it means that you can't redefine how policy works on a given API, 
but you should not do that anyway. That breaks people's custom policy, 
and that is not kind to operators.

> --
>
> 4. As a note that came out of the conversation I had with Sean, we 
> should look at no longer making the policy definition for an API keyed 
> on the intern-method name of a project. While "instance_create" is 
> relatively descriptive, there are many other API calls that you cannot 
> really know what changing the policy will do without trying it. Sean 
> and I were discussing moving towards supplying a representation of the 
> URI path instead of "image_create". This is something that consumers 
> and deployers of OpenStack will be more familiar with. It also 
> eliminates some of the mapping needed to know what the URI of "image 
> create" is when utilized in the Horizon context (asking Keystone "what 
> can be done with authorization XXX").

We would all like that.Keep in mind, however that there is nothing that 
requires policy only be enforced once on an API call, or that policy 
cannot be called and enforced deep within the code.  A project may 
decide that it wants to enforce policy at the Database layer, and make 
sure that all requests that attempt to read a specific record pass a 
policy check.  We should not prevent them from doing so.

> --
>
> All of the conversation in this thread and in IRC has been good. But I 
> think we're at the point where we need to start setting a clear 
> direction on the policy bits. All the recommendations have had merit 
> and bring to the table interesting perspectives.
I'd like to actually start making progress on the Dynamic policy 
implementation.  The current approach is broken and a security risk.I 
think the current approach covers Sean's concerns without writing policy 
in Python code.

>
> Lets start defining the next steps (this is not going to be easy) so 
> we can start really working on the foundation we need to make all of 
> this a reality.
>
> --Morgan
>
>
>
> On Thu, Jun 4, 2015 at 12:00 PM, Adam Young <ayoung at redhat.com 
> <mailto:ayoung at redhat.com>> wrote:
>
>     On 06/04/2015 01:16 PM, Sean Dague wrote:
>
>         It gets overwritten by the central store.
>
>         And you are wrong, that gives me what I want, because we can
>         emit a
>         WARNING in the logs if the patch is something crazy. The
>         operators will
>         see it, and be able to fix it later.
>
>         I'm not trying to prevent people from changing their policy in
>         crazy
>         ways. I'm trying to build in some safety net where we can
>         detect it's
>         kind of a bad idea and emit that information a place that
>         Operators can
>         see and sort out later, instead of pulling their hair out.
>
>         But you can only do that if you have encoded what's the
>         default, plus
>         annotations about ways that changing the default are unwise.
>
>     When would you expect this warning to be emitted, and to whom?  I
>     think you have the right idea, but I suggest that the appropriate
>     time to give that warning would be back when the policy is
>     written, which would be under the scope of the Database-Driven
>     policy management.  I would think that, if a user changes a
>     policy, it would go to a staged state, not deployed immediately,
>     and at that point, we'd want a check to run.  That check would be
>     what told the author they did something unexpected.  Waiting until
>     the policy hits the server is probably too late.
>
>
>
>
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150604/2ed7e277/attachment.html>