Open Stack

On Sat, Aug 1, 2015 at 3:41 PM, Clint Byrum <clint at fewbar.com> wrote:
> This too is overly complex and will cause failures. If you replace key 0,
> you will stop validating tokens that were encrypted with the old key 0.

No. Key 0 is replaced after rotation.

Also, come on, does http://paste.openstack.org/show/406674/ look overly 
complex? (it should be launched from Fuel master node).

> You simply need to run rotate on one, and then rsync that key repository
> to all of the others. You _must not_ run rotate again until you rsync to
> all of the others, since the key 0 from one rotation becomes the primary
> token encrypting key going forward, so you need it to get pushed out to
> all nodes as 0 first.

I agree. What step in my logic misses that part?

On Saturday 01 August 2015 15:50:13 Matt Fischer wrote:
> Agree that you guys are way over thinking this. You don't need to rotate
> keys at exactly the same time, we do it in within a one or two hours
> typically based on how our regions are setup. We do it with puppet, 
puppet
> runs on one keystone node at a time and drops the keys into place.

There is a constraint: sometimes you cannot connect from one keystone 
node to another. For example, in a cloud deployed by Fuel you cannot ssh 
from one controller to another afaik.

> The
> actual rotation and generation we handle with a script that then 
proposes
> the new key structure as a review which is then approved and deployed 
via
> the normal process. For this process I always drop keys 0, 1, 2 into 
place,
> I'm not bumping the numbers like the normal rotations do.

I dislike this solution because there is more than 1 point of configiration. If 
your cloud administrator decides to use not 3 keys, but 5, he will have to 
change not only the option in keystone.conf, but also in your script. Yes, 
keystone will still work, but there will be some inconsistency.

I also dislike it because keys should be generated only be a single tool. If it 
would turn out that keys used for fernet tokens are too weak and 
developers decide to change key length from 32 bytes to 64, it will have to 
be fixed outside of that tool too. Which is not good. Now this tool is 
keystone-manage

> We had also considered ansible which would be perfect for this, but that
> makes our ability to setup throw away environments with a single click a
> bit more complicated. If you don't have that requirement, a simple 
ansible
> script is what you should do.
> 
> On Sat, Aug 1, 2015 at 3:41 PM, Clint Byrum <clint at fewbar.com> wrote:
> > Excerpts from Boris Bobrov's message of 2015-08-01 14:18:21 -0700:
> > > On Saturday 01 August 2015 16:27:17 bdobrelia at mirantis.com 
wrote:
> > > > I suggest to use pacemaker multistate clone resource to rotate 
and
> > > 
> > > rsync
> > > 
> > > > fernet tokens from local directories across cluster nodes. The
> > > > resource
> > > > prototype is described here
> > > 
> > > > https://etherpad.openstack.org/p/fernet_tokens_pacemaker> 
Pros:
> > > Pacemaker
> > > 
> > > > will care about CAP/split-brain stuff for us, we just design rotate
> > > > and
> > > > rsync logic. Also no shared FS/DB involved but only Corosync CIB - 
to
> > > 
> > > store
> > > 
> > > > few internal resource state related params, not tokens. Cons: 
Keystone
> > > > nodes hosting fernet tokens directories must be members of 
pacemaker
> > > > cluster. Also custom OCF script should be created to implement 
this.
> > > > __
> > > > Regards,
> > > > Bogdan Dobrelya.
> > > > IRC: bogdando
> > > 
> > > Looks complex.
> > > 
> > > I suggest this kind of bash or python script, running on Fuel master
> > 
> > node:
> > > 0. Check that all controllers are online;
> > > 1. Go to one of the controllers, rotate keys there;
> > > 2. Fetch key 0 from there;
> > > 3. For each other controller rotate keys there and put the 0-key 
instead
> > 
> > of
> > 
> > > their new 0-key.
> > > 4. If any of the nodes fail to get new keys (because they went offline
> > 
> > or for
> > 
> > > some other reason) revert the rotate (move the key with the 
biggest
> > > index
> > > back to 0).
> > > 
> > > The script can be launched by cron or by button in Fuel.
> > > 
> > > I don't see anything critically bad if one rotation/sync event fails.
> > 
> > This too is overly complex and will cause failures. If you replace key 0,
> > you will stop validating tokens that were encrypted with the old key 0.
> > 
> > You simply need to run rotate on one, and then rsync that key 
repository
> > to all of the others. You _must not_ run rotate again until you rsync to
> > all of the others, since the key 0 from one rotation becomes the 
primary
> > token encrypting key going forward, so you need it to get pushed out 
to
> > all nodes as 0 first.
> > 
> > Don't over think it. Just read http://lbragstad.com/?p=133 and it will
> > remain simple.
> > 
> > 
__________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-
request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
С наилучшими пожеланиями,
Boris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150802/d71d6507/attachment.html>