[openstack-dev] [keystone][nova] Persistent application credentials

Monty Taylor mordred at inaugust.com
Thu Jul 20 03:19:19 UTC 2017

On 07/19/2017 12:11 AM, Zane Bitter wrote:
> On 17/07/17 23:12, Lance Bragstad wrote:
>>     Would Keystone folks be happy to allow persistent credentials once
>>     we have a way to hand out only the minimum required privileges?
>> If I'm understanding correctly, this would make application 
>> credentials dependent on several cycles of policy work. Right?
> My thought here was that if this were the case (i.e. persistent 
> credentials are OK provided the user can lock down the privileges) then 
> you could make a case that the current spec is on the right track. For 
> now we implement the application credentials as non-persistent, people 
> who know about it use at their own risk, and for people who don't 
> there's no exposure. Later on we add the authorisation stuff and relax 
> the non-persistence requirement.
> On further reflection, I'm not convinced by this - if we care about 
> protecting people who don't intentionally use/know about the feature 
> now, then we should probably still care once the tools are in place for 
> the people who are using it intentionally to lock it down tightly.
> So I'm increasingly convinced that we need to do one of two things. Either:
> * Agree with Colleen (elsewhere in the thread) that persistent 
> application credentials are still better than the status quo and 
> reinstate the project-scoped lifecycle in accordance with original 
> intent of the spec; or
> * Agree that the concerns raised by Morgan & Adam will always apply, and 
> look for a solution that gives us automatic key rotation - which might 
> be quite different in shape (I can elaborate if necessary).
> (That said, I chatted about this briefly with Monty yesterday and he 
> said that his recollection was that there is a long-term solution that 
> will keep everyone happy. He'll try to remember what it is once he's 
> finished on the version discovery stuff he's currently working on.)

Part of this comes down to the fact that there are actually multiple 
scenarios and persistent credentials actually only applies to a scenario 
that typically requires a human with elevated credentials.

SO - I think we can get a long way forward by divvying up some 
responsibilities clearly.

What I mean is:

* The simple consume case ("typical public cloud") is User-per-Project 
with User lifecycle tied to Project lifecycle. In this case the idea of 
a 'persistent' credential is meaningless, because there is no 'other' 
User with access to the Project If the User in this scenario creates a 
Credential it doesn't actually matter what the Credential lifecycle is 
because the act of Account is ultimately about disabling or deleting 
access to the Project in question. We can and should help the folks who 
are running clouds in this model with $something (we need to talk 
details) so that if they are running in this model they don't 
accidentally or by default leave a door open when they think they've 
disabled someone's User as part of shutting off their Account. But in 
this scenario OpenStack adding project-persistent credentials is not a 
big deal - it doesn't provide value. (While a User in that scenario, who 
typically does not have the Role to create a new User being able to 
manage Application Credentials is a HUGE win)

* The other scenario is where there is more than one Human who has a 
User that have been granted Roles on a Project. This is the one where 
project-lifecycle credentials are meaningful and valuable, but it's also 
one that involves some Human with elevated admin-style privileges having 
been involved at some point because that is required to assign Users 
Roles in the Project in the first place.

I believe if we divide application credentials into two kinds:

1) Application Credentials with lifecycle tied to User
2) Application Credentials with lifecycle tied to Project

Then I think it's ok for the ability to do (2) to require a specific 
Role in policy. If we do that, then whatever Human it is that is mapping 
multiple Users into a single Project can decide whether any of those 
Users should be granted the ability to make Project-lifecycle 
Application Credentials. Such a Human is already a Human who has a User 
with elevated permissions, as you have to be to assign Roles to Users 

In any case, as I mentioned in the other mail, I think there are a bunch 
of details here that are going to require us being in the room - and 
everyone realizing that everyones use cases and everyones concerns are 
important. If we dig in, I'm sure we can come out on the other side with 
happiness and joy.

> I'm trying to avoid taking a side here because everyone is right. 

> Currently anybody who want to do anything remotely 'cloudy' (i.e. have 
> the application talk to OpenStack APIs) has to either share their 
> personal password with the application (and by extension their whole 

Or - create an account in the Team's name and by storing the password 
for that account realizing that everyone on the team has access to the 
password so that a user-leaving-team event necessitates a "quick, let's 
go re-key everything" event.

> team) or to do the thing that is the polar opposite of cloud: file a 
> ticket with IT to get a service user account added <bangs head against 
> desk> and share that password instead.
Yup. Pretty much same thing.

  And this really is a disaster for
> OpenStack. On the other hand, allowing the creation of persistent 
> application credentials in the absence of regular automatic rotation 
> does create risk for those folks who are not aggressively auditing them 
> (perhaps because they have no legitimate use for them) and the result is 
> likely to be lots of clouds disabling them by policy, keeping their 
> users in the dark age of IT-ticket-filing <head...desk> and frustrating 
> our interoperability goals.


> It is possible in theory to satisfy both via the 'instance users' 
> concept, but the Nova team's response to this has consistently been 
> "prove to us that this has to be in Nova". Well, here's your answer.

Instance users do not solve this. Instance users can be built with this- 
but instance users are themselves not sufficient. Instance users are 
only sufficient in single-cloud ecosystems where it is possible to grant 
permissions on all the resources in the single-cloud ecosystem to an 
instance. We are not a single-cloud ecosystem.

Nodepool runs in Rackspace's DFW region. It has accounts across nine 
different clouds. If this were only solved with Instance users we'd have 
to boot a VM in each cloud so that we could call the publicly-accessible 
REST APIs of the clouds to boot VMs in each cloud.

If I have to boot a VM so that I can boot a VM I swear to god I'm going 
to rage-quit tech and raise goats.

Our current system that does not have a per-cloud VM-launching-VM 
handles ~20k VM lifecycles per day load and it doesn't have to deal with 
cross-WAN clustering concerns. Putting a password in a config file is 
better than adding a fleet of VM-launching-VMs - we don't lose Infra 
Root members THAT often.

Application Credentials, on the other hand, would allow us improve the 
situation without imposing an architecture on us that is a copy of an 
architecture from some clouds that have a vested interest in keeping 
people thinking about problems in a uni-cloud manner.

Long-winded way of saying that I think if we get this right on the App 
Creds side, we can actually solve all the use cases, including but not 
limited to the Instance User use case.


More information about the OpenStack-dev mailing list