Open Stack

On 19/07/17 01:23, Colleen Murphy wrote:
> On Tue, Jul 18, 2017 at 1:39 AM, Zane Bitter <zbitter at redhat.com
> <mailto:zbitter at redhat.com>> wrote:
>
>     So the application credentials spec has merged - huge thanks to
>     Monty and the Keystone team for getting this done:
>
>     https://review.openstack.org/#/c/450415/
>     <https://review.openstack.org/#/c/450415/>
>     http://specs.openstack.org/openstack/keystone-specs/specs/keystone/pike/application-credentials.html
>     <http://specs.openstack.org/openstack/keystone-specs/specs/keystone/pike/application-credentials.html>
>
>     However, it appears that there was a disconnect in how two groups
>     of folks were reading the spec that only became apparent towards
>     the end of the process. Specifically, at this exact moment:
>
>     http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-keystone.2017-06-09.log.html#t2017-06-09T17:43:59
>     <http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-keystone.2017-06-09.log.html#t2017-06-09T17:43:59>
>
>     To summarise, Keystone folks are uncomfortable with the idea of
>     application credentials that share the lifecycle of the project
>     (rather than the user that created them), because a consumer could
>     surreptitiously create an application credential and continue to
>     use that to access the OpenStack APIs even after their User
>     account is deleted. The agreed solution was to delete the
>     application credentials when the User that created them is
>     deleted, thus tying the lifecycle to that of the User.
>
>     This means that teams using this feature will need to audit all of
>     their applications for credential usage and rotate any credentials
>     created by a soon-to-be-former team member *before* removing said
>     team member's User account, or risk breakage. Basically we're
>     relying on users to do the Right Thing (bad), but when they don't
>     we're defaulting to breaking [some of] their apps over leaving
>     them insecure (all things being equal, good).
>
>     Unfortunately, if we do regard this as a serious problem, I don't
>     think this solution is sufficient. Assuming that application
>     credentials are stored on VMs in the project for use by the
>     applications running on them, then anyone with access to those
>     servers can obtain the credentials and continue to use them even
>     if their own account is deleted. The solution to this is to rotate
>     *all* application keys when a user is deleted. So really we're
>     relying on users to do the Right Thing (bad), but when they don't
>     we're defaulting to breaking [some of] their apps *and*
>     [potentially] leaving them insecure (worst possible combination).
>
>     (We're also being inconsistent, because according to the spec if
>     you revoke a role from a User then any application credentials
>     they've created that rely on that role continue to work. It's only
>     if you delete the User that they're revoked.)
>
>
>     As far as I can see, there are only two solutions to the
>     fundamental problem:
>
>     1) Fine-grained user-defined access control. We can minimise the
>     set of things that the application credentials are authorised to
>     do. That's out of scope for this spec, but something we're already
>     planning as a future enhancement.
>     2) Automated regular rotation of credentials. We can make sure
>     that whatever a departing team member does manage to hang onto
>     quickly becomes useless.
>
>     By way of comparison, AWS does both. There's fine-grained defined
>     access control in the form of IAM Roles, and these Roles can be
>     associated with EC2 servers. The servers have an account with
>     rotating keys provided through the metadata server. I can't find
>     the exact period of rotation documented, but it's on the order of
>     magnitude of 1 hour.
>
>     There's plenty not to like about this design. Specifically, it's
>     2017 not 2007 and the idea that there's no point offering to
>     segment permissions at a finer grained level than that of a VM no
>     longer holds water IMHO, thanks to SELinux and containers. It'd be
>     nice to be able to provide multiple sets of credentials to
>     different services running on a VM, and it's probably essential to
>     our survival that we find a way to provide individual credentials
>     to containers. Nevertheless, what they have does solve the problem.
>
>     Note that there's pretty much no sane way for the user to automate
>     credential rotation themselves, because it's turtles all the way
>     down. e.g. it's easy in principle to set up a Heat template with a
>     Mistral workflow that will rotate the credentials for you, but
>     they'll do so using trusts that are, in turn, tied back to the
>     consumer who created the stack. (It suddenly occurs to me that
>     this is a problem that all services using trusts are going to need
>     to solve.) Somewhere it all has to be tied back to something that
>     survives the entire lifecycle of the project.
>
>     Would Keystone folks be happy to allow persistent credentials once
>     we have a way to hand out only the minimum required privileges?
>
>
> I agree that in the haste of getting this approved before the spec
> freeze deadline we took this in the wrong direction. I think that this
> spec was fine before the addition of "Will be deleted when the
> associated User is deleted" constraint.
>
> As I understand it, the worry coming from the team is that a user who
> sneakily copies the secret keys to an offsite location before their
> user is deleted will still have access to the resources the credential
> had access to. This worry is irrational for two reasons:
>
> 1) Once the application credential is created, it is going to be
> shared with the whole team. Therefore if ANY member of the team is
> deleted, whether or not they were the one to create the credential,
> they could still have copied this key and still gain access to those
> resources after their user was terminated. Tying the lifecycle of the
> credential to its creating user does nothing to prevent other
> teammates from regaining this access.
>
> 2) The ability to rotate the secret, whether it's automated or not, is
> already an infinitely better situation than where we currently are,
> which is that a user must live forever and share their personal
> passwords with the whole team in order to keep applications running.
> We can encourage immediate credential rotation upon user termination
> and provide helpful tooling and perhaps Heat integration, but even
> without all of that it's still better than forcing users to share
> their passwords with one another and being unable to delete users who
> have left the organization.
>
> I'm supportive of revising the spec to undo this constraint.
>
While I understand worries behind these security problems, I'd like to
add they they aren't really our problem. They can't be solved with code
on our end. These are procedural problems, and this problem persists in
any organisation, in any form of shared user passwords, in any form of
automation. If an application credential is created for a service
used/maintained by a team, there is nothing stopping anyone from the
team from storing that key elsewhere regardless of who created it,
because once created, that key is in the 'shared' team domain where
anyone can see it, exactly as Colleen points out. If it wasn't shared
with more than one person, then it's useless since it means the service
using the key itself would entirely have to be maintained/controlled by
that one person and no one else would have access. Teams don't work like
that.

The problem is then entirely procedural within a team. Do they rotate
all keys when one person leaves? Anything less is the same problem. All
we can do is make rotation less of a pain, but it will still be painful
no matter what, and depending on the situation the team makes the choice
of how to handle rotation if at all.

The sole reason for project level ownership of these application
credentials is so that a user leaving/being deleted isn't a scramble to
replace keys, and a team has the option/time to do it if they care about
the possibility of that person having known the keys (again, not our
problem, not a security flaw in code). Anything else, pretty much makes
this feature useless for teams. :(

Having both options (owned by project vs user) is useful, but the
'security issues' are kind of implied by using project owned app creds.
It's a very useful feature with some 'use at your own risk' attached.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170720/4b237e43/attachment.html>