Open Stack

Hello,

As you know, keystone introduced non-persistent tokens in kilo -- Fernet 
tokens. These tokens use Fernet keys, that are rotated from time to time. A 
great description of key rotation and replication can be found on [0] and [1] 
(thanks, lbragstad). In HA setup there are multiple nodes with Keystone and 
that requires key replication. How do we do that with new Fernet tokens?

Please keep in mind that the solution should be HA -- there should not be any 
"master" server, pushing keys to slave servers, because master server might go 
down.

I can see some ways to do that.

1. Mount some distributed network file system to /etc/keystone/fernet-keys/ 
(the directory, where keys are) and leave syncronization and dealing with race 
conditions to it. This solution will not require any changes to existing code.

Are there any mature filesystems for that?

2. Use a queue of staged keys. It would mean that a new staging key will be 
generated if there are no other staging keys in queue. Example:

Suppose we have keystone setup on 2 servers.

I. In the beginning they have keys 0, 1, 2.

II. Rotation happens on keystone-1. 0 becomes 3, 1 is removed. Before 
generating 0, check that there are no keys in the queue. There are no keys in 
the queue, generate it and push to keystone-2's queue.

III. Rotations happens on keystone-2. 0 becomes 3, 1 is removed. Before 
generating 0, check that there are no keys in the queue. There is a key from 
keystone-1, use it as new 0.

Thanks to Alexander Makarov for the idea.

How do we store this queue? Should we use some backend, rely on creation time 
or something else?

This way requires changes to keystone code.

3. Store keys in backend completely and use well-known sync mechanisms. This 
would require some changes to keystone code too.

-- 
Best regards,
Boris Bobrov