[openstack-dev] [keystone][fernet] Fernet tokens sync
bbobrov at mirantis.com
Fri Mar 27 14:14:28 UTC 2015
As you know, keystone introduced non-persistent tokens in kilo -- Fernet
tokens. These tokens use Fernet keys, that are rotated from time to time. A
great description of key rotation and replication can be found on  and 
(thanks, lbragstad). In HA setup there are multiple nodes with Keystone and
that requires key replication. How do we do that with new Fernet tokens?
Please keep in mind that the solution should be HA -- there should not be any
"master" server, pushing keys to slave servers, because master server might go
I can see some ways to do that.
1. Mount some distributed network file system to /etc/keystone/fernet-keys/
(the directory, where keys are) and leave syncronization and dealing with race
conditions to it. This solution will not require any changes to existing code.
Are there any mature filesystems for that?
2. Use a queue of staged keys. It would mean that a new staging key will be
generated if there are no other staging keys in queue. Example:
Suppose we have keystone setup on 2 servers.
I. In the beginning they have keys 0, 1, 2.
II. Rotation happens on keystone-1. 0 becomes 3, 1 is removed. Before
generating 0, check that there are no keys in the queue. There are no keys in
the queue, generate it and push to keystone-2's queue.
III. Rotations happens on keystone-2. 0 becomes 3, 1 is removed. Before
generating 0, check that there are no keys in the queue. There is a key from
keystone-1, use it as new 0.
Thanks to Alexander Makarov for the idea.
How do we store this queue? Should we use some backend, rely on creation time
or something else?
This way requires changes to keystone code.
3. Store keys in backend completely and use well-known sync mechanisms. This
would require some changes to keystone code too.
More information about the OpenStack-dev