Open Stack

I completely agree that making Congress successful will rely crucially on addressing performance and scalability issues.  Some thoughts...

1. We're definitely intending to cache data locally to avoid repeated API calls.  In fact, a prototype cache is already in place.  We haven't yet hooked up API calls (other than to AD).  We envision some data sources pushing us data (updates) and some data sources requiring us to periodically pull.  

2. My main concern for scalability/performance is for proactive enforcement, where at least conceptually Congress is on the critical path for API calls.

One thought is that we could splice out, say, the network portion of the Congress policy and push it down into neutron, assuming neutron could enforce that policy.  This would at least eliminate cross-component communication.  It would require a policy engine on each of the OS components, but (a) there already is one on many components and (b) if there isn't, we can rely on reactive enforcement.

The downside with policy-caching on other OS components are the usual problems with staleness and data replication, e.g. maybe we'd end up copying all of nova's VM data into neutron so that neutron could enforce its policy.  But because we have reactive enforcement to rely on, we could always approximate the policy that we push down (conservatively) to catch the common mistakes and leave the remainder to reactive enforcement.  For example, we might be able to auto-generate the current policy.json files for each component from Congress's policy.

Keeping Congress out of the critical path for every API call is one of the reasons it was designed to do reactive enforcement as well as proactive enforcement.

3. Another option is to build high-performance optimizations for certain fragments of the policy language.  Then the cloud architect can decide whether she wants to utilize a more expressive language whose performance is worse or a less expressive language whose performance is better.

Tim




----- Original Message -----
| From: "Flavio Percoco" <flavio at redhat.com>
| To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
| Sent: Thursday, November 14, 2013 6:05:46 AM
| Subject: Re: [openstack-dev] Congress: an open policy framework
| 
| On 14/11/13 04:40 -0800, Morgan Fainberg wrote:
| >On Wed, Nov 13, 2013 at 10:40 AM, Tim Hinrichs
| ><thinrichs at vmware.com> wrote:
| >> We're just getting started with Congress and understanding how it
| >> will integrate with the OS ecosystem, but here's our current
| >> thinking about how Congress relates to Oslo's policy engine and
| >> to Keystone.  Comments and suggestions are welcome.
| >>
| >>
| >> Congress and Oslo
| >> --------------------
| >> Three dimensions for comparison: policy language, data sources,
| >> and policy engine.
| >>
| >> We've always planned to make Congress compatible with existing
| >> policy languages like the one in oslo.  The plan is to build a
| >> front-end for a number of policy languages/formats, e.g.
| >> oslo-policy language, XACML, JSON, YAML, SQL, etc.  The idea
| >> being that the syntax/language you use is irrelevant as long as
| >> it can be mapped into Congress's native policy language.  As of
| >> now, Congress is using Datalog, which is a variant of SQL and is
| >> at least as expressive as all of the policy languages we've run
| >> across in the cloud domain, including the oslo-policy language.
| >>
| >> In terms of the data sources you can reference in the policy,
| >> Congress is designed to enable policies that reference arbitrary
| >> data sources in the cloud.  For example, we could write a Nova
| >> authorization policy that permits a new VM to be created if that
| >> VM is connected to a network owned by a tenant (info stored in
| >> Neutron) where the VM owner (info in the request) is in the same
| >> group as the network owner (info stored in Keystone/LDAP).
| >>  Oslo's handles some of these data sources with its terminal
| >> rules, but it's not involved in data integration to the same
| >> extent Congress is.
| >>
| >> In terms of policy engines, Congress is intended to enforce
| >> policies in 2 different ways: proactively (stopping policy
| >> violations before they occur) and reactively (acting to eliminate
| >> a violation after it occurs).  Ideally we wouldn't need reactive
| >> enforcement, but there will always be cases where proactive
| >> enforcement is not possible (e.g. a DOS attack brings app
| >> latencies out of compliance).  The oslo-engine does proactive
| >> enforcement only--stopping API calls before they violate the
| >> policy.
| >>
| >
| >Does this mean all policy decisions need to ask this new service?
| >There are many policy checks that occur across even a given action
| >(in
| >some cases).  Could this have a significant performance implication
| >on
| >larger scale cloud deployments?  I like the idea of having reactive
| >(DOS prevention) policy enforcement as well as external (arbitrary)
| >data to help make policy decisions, I don't want to see Congress be
| >limited in deployment because large scale clouds getting
| >bottle-necked
| >trying to communicate with it.
| 
| This is exactly what worries me about Congress. I mentioned in my
| last
| email that some kind of 'local' cache managed by the Confress library
| is a must to avoid the performance penalty.
| 
| >There might be some value  to seeing some work being done to provide
| >more information to Keystone, but I think this will become more
| >apparent as Congress develops.
| 
| +1
| 
| Cheers,
| FF
| 
| --
| @flaper87
| Flavio Percoco
| 
| _______________________________________________
| OpenStack-dev mailing list
| OpenStack-dev at lists.openstack.org
| http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
|