[openstack-dev] [nova] key_pair update on rebuild (a whole lot of conversations)

Sean Dague sean at dague.net
Tue Oct 3 20:16:48 UTC 2017

There is currently a spec up for being able to specify a new key_pair
name during the rebuild operation in Nova -

For those not completely familiar with Nova operations, rebuild triggers
the "reset this vm to initial state" by throwing out all the disks, and
rebuilding them from the initial glance images. It does however keep the
IP address and device models when you do that. So it's useful for
ephemeral but repeating workloads, where you'd rather not have the
network information change out from under you.

The spec is a little vague about when this becomes really useful,
because this will not save you from "I lost my private key, and I have
important data on that disk". Because the disk is destroyed. That's the
point of rebuild. We once added this preserve_ephemeral flag to rebuild
for trippleo on ironic, but it's so nasty we've scoped it to only work
with ironic backends. Ephemeral should mean ephemeral.

Rebuild bypasses the scheduler. A rebuilt server stays on the same host
as it was before, which means the operation has a good chance of being
faster than a DELETE + CREATE, as the image cache on that host should
already have the base image for you instance.

A bunch of data was collected today in a lot of different IRC channels
(#openstack-nova, #openstack-infra, #openstack-operators).

= OpenStack Operators =

mnaser said that for their customers this would be useful. Keys get lost
often, but keeping the IP is actually valuable. They would also like this.

penick said that for their existing environment, they have a workflow
where this would be useful. But they are moving away from using nova for
key distribution because in Nova keys are user owned, which actually
works poorly given that everything else is project owned. So they are
building something to do key distribution after boot in the guest not
using nova's metadata.

Lots of people said they didn't use nova's keypair interfaces, they just
did it all in config management after the fact.

= Also on reboot? =

Because the reason people said they wanted it was: "I lost my private
key", the question at PTG was "does that mean you want it on reboot?"

But as we dive through the constraints of that, people that build "pet"
VMs typically delete or disable cloud-init (or similar systems) after
first boot. Without that kind of agent, this isn't going to work anyway.

So also on reboot seems very fragile and unuseful.

= Infra =

We asked the infra team if this is useful to them, the answer was no.
What would be useful them is if keypairs could be updated. They use a
symbolic name for a keypair but want to do regular key rotation. Right
now they do this by deleting then recreating keypairs, but that does
mean there is a window where there is no keypair with that name, so
server creates fail.

It is agreed that something supporting key rotation in the future would
be handy, that's not in this scope.

= Barbican =

In the tradition of making a simple fix a generic one, it does look like
there is a longer term part of this where Nova should really be able to
specify a Barbican resource url for a key so that things like rotation
could be dealt with in a system that specializes in that. It also would
address the very weird oddity of user vs. project scoping.

That's a bigger more nebulous thing. Other folks would need to be
engaged on that one.

= Where I think we are? =

I think with all this data we're at the following:

Q: Should we add this to rebuild
A: Yes, probably - after some enhancement to the spec *

* - we really should have much better use cases about the situations it
is expected to be used in. We spend a lot of time 2 and 3 years out
trying to figure out how anyone would ever use a feature, and adding
another one without this doesn't seem good

Q: should this also be on reboot?
A: NO - it would be too fragile

I also think figuring out a way to get Nova out of the key storage
business (which it really shouldn't be in) would be good. So if anyone
wants to tackle Nova using Barbican for keys, that would be ++. Rebuild
doesn't wait on that, but Barbican urls for keys seems like a much
better world to be in.


Sean Dague

More information about the OpenStack-dev mailing list