Open Stack

So the application credentials spec has merged - huge thanks to Monty 
and the Keystone team for getting this done:

https://review.openstack.org/#/c/450415/
http://specs.openstack.org/openstack/keystone-specs/specs/keystone/pike/application-credentials.html

However, it appears that there was a disconnect in how two groups of 
folks were reading the spec that only became apparent towards the end of 
the process. Specifically, at this exact moment:

http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-keystone.2017-06-09.log.html#t2017-06-09T17:43:59

To summarise, Keystone folks are uncomfortable with the idea of 
application credentials that share the lifecycle of the project (rather 
than the user that created them), because a consumer could 
surreptitiously create an application credential and continue to use 
that to access the OpenStack APIs even after their User account is 
deleted. The agreed solution was to delete the application credentials 
when the User that created them is deleted, thus tying the lifecycle to 
that of the User.

This means that teams using this feature will need to audit all of their 
applications for credential usage and rotate any credentials created by 
a soon-to-be-former team member *before* removing said team member's 
User account, or risk breakage. Basically we're relying on users to do 
the Right Thing (bad), but when they don't we're defaulting to breaking 
[some of] their apps over leaving them insecure (all things being equal, 
good).

Unfortunately, if we do regard this as a serious problem, I don't think 
this solution is sufficient. Assuming that application credentials are 
stored on VMs in the project for use by the applications running on 
them, then anyone with access to those servers can obtain the 
credentials and continue to use them even if their own account is 
deleted. The solution to this is to rotate *all* application keys when a 
user is deleted. So really we're relying on users to do the Right Thing 
(bad), but when they don't we're defaulting to breaking [some of] their 
apps *and* [potentially] leaving them insecure (worst possible combination).

(We're also being inconsistent, because according to the spec if you 
revoke a role from a User then any application credentials they've 
created that rely on that role continue to work. It's only if you delete 
the User that they're revoked.)


As far as I can see, there are only two solutions to the fundamental 
problem:

1) Fine-grained user-defined access control. We can minimise the set of 
things that the application credentials are authorised to do. That's out 
of scope for this spec, but something we're already planning as a future 
enhancement.
2) Automated regular rotation of credentials. We can make sure that 
whatever a departing team member does manage to hang onto quickly 
becomes useless.

By way of comparison, AWS does both. There's fine-grained defined access 
control in the form of IAM Roles, and these Roles can be associated with 
EC2 servers. The servers have an account with rotating keys provided 
through the metadata server. I can't find the exact period of rotation 
documented, but it's on the order of magnitude of 1 hour.

There's plenty not to like about this design. Specifically, it's 2017 
not 2007 and the idea that there's no point offering to segment 
permissions at a finer grained level than that of a VM no longer holds 
water IMHO, thanks to SELinux and containers. It'd be nice to be able to 
provide multiple sets of credentials to different services running on a 
VM, and it's probably essential to our survival that we find a way to 
provide individual credentials to containers. Nevertheless, what they 
have does solve the problem.

Note that there's pretty much no sane way for the user to automate 
credential rotation themselves, because it's turtles all the way down. 
e.g. it's easy in principle to set up a Heat template with a Mistral 
workflow that will rotate the credentials for you, but they'll do so 
using trusts that are, in turn, tied back to the consumer who created 
the stack. (It suddenly occurs to me that this is a problem that all 
services using trusts are going to need to solve.) Somewhere it all has 
to be tied back to something that survives the entire lifecycle of the 
project.

Would Keystone folks be happy to allow persistent credentials once we 
have a way to hand out only the minimum required privileges?

If not I think we're back to https://review.openstack.org/#/c/222293/

cheers,
Zane.