Open Stack

Hey Pino,

mriedem pointed me to the vendordata code [1] which shows some fields are
passed (such as project ID) and that SSL is supported. So that's good.

The docs on vendordata suck. But I think it'll do what you're looking for.
Michael Still wrote up a helpful post titled "Nova vendordata deployment,
an excessively detailed guide"[2] and he's written a vendordata service
example[3] which even shows keystone integration.

At Oath, we have a system that provides a unique x509 certificate for each
host, including the ability to sign host SSH keys against an HSM. In our
case what we do is have Nova call the service, which generates and returns
a signed (and time limited) host bootstrap document, which is injected into
the instance. When the instance boots it calls our identity service and
provides its bootstrap document as a bearer certificate. The identity
service trusts this one-time document to attest the instance, and will then
provide an x509 certificate as well as sign the hosts SSH keys. After the
initial bootstrap the host will rotate its keys frequently, by providing
its last certificate in exchange for a new one. The service tracks all host
document and certificate IDs which have been exchanged until their expiry,
so that a cert cannot be re-used.

This infrastructure relies on Athenz [4] as the AuthNG system for all
principals (users, services, roles, domains, etc) as well as an internal
signatory service which signs x509 certificates and SSH host keys using an
HSM infrastructure.

Instead, you could write a vendordata service which, when called, would
generate an ssh host keypair, sign it, and return those files as encoded
data, which can be expanded into files in the correct locations on first
boot. I strongly suggest using not only using keystone auth, but that you
ensure all calls from vendordata to the microservice are encrypted with TLS
mutual auth.

-James


1:
https://github.com/openstack/nova/blob/master/nova/api/metadata/vendordata_dynamic.py#L77
2: https://www.stillhq.com/openstack/000022.html
3: https://github.com/mikalstill/vendordata
4: https://athenz.io


On Fri, Sep 29, 2017 at 5:17 PM, Fox, Kevin M <Kevin.Fox at pnnl.gov> wrote:

> https://review.openstack.org/#/c/222293/
> ------------------------------
> *From:* Giuseppe de Candia [giuseppe.decandia at gmail.com]
> *Sent:* Friday, September 29, 2017 1:05 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] Supporting SSH host certificates
>
> Ihar, thanks for pointing that out - I'll definitely take a close look.
>
> Jon, I'm not very familiar with Barbican, but I did assume the full
> implementation would use Barbican to store private keys. However, in terms
> of actually getting a private key (or SSH host cert) into a VM instance,
> Barbican doesn't help. The instance needs permission to access secrets
> stored in Barbican. The main question of my e-mail is: how do you inject a
> credential in an automated but secure way? I'd love to hear ideas - in the
> meantime I'll study Ihar's link.
>
> thanks,
> Pino
>
>
>
> On Fri, Sep 29, 2017 at 2:49 PM, Ihar Hrachyshka <ihrachys at redhat.com>
> wrote:
>
>> What you describe (at least the use case) seems to resemble
>> https://review.openstack.org/#/c/456394/ This work never moved
>> anywhere since the spec was posted though. You may want to revive the
>> discussion in scope of the spec.
>>
>> Ihar
>>
>> On Fri, Sep 29, 2017 at 12:21 PM, Giuseppe de Candia
>> <giuseppe.decandia at gmail.com> wrote:
>> > Hi Folks,
>> >
>> >
>> >
>> > My intent in this e-mail is to solicit advice for how to inject SSH host
>> > certificates into VM instances, with minimal or no burden on users.
>> >
>> >
>> >
>> > Background (skip if you're already familiar with SSH certificates):
>> without
>> > host certificates, when clients ssh to a host for the first time (or
>> after
>> > the host has been re-installed), they have to hope that there's no man
>> in
>> > the middle and that the public key being presented actually belongs to
>> the
>> > host they're trying to reach. The host's public key is stored in the
>> > client's known_hosts file. SSH host certicates eliminate the
>> possibility of
>> > Man-in-the-Middle attack: a Certificate Authority public key is
>> distributed
>> > to clients (and written to their known_hosts file with a special syntax
>> and
>> > options); the host public key is signed by the CA, generating an SSH
>> > certificate that contains the hostname and validity period (among other
>> > things). When negotiating the ssh connection, the host presents its SSH
>> host
>> > certificate and the client verifies that it was signed by the CA.
>> >
>> >
>> >
>> > How to support SSH host certificates in OpenStack?
>> >
>> >
>> >
>> > First, let's consider doing it by hand, instance by instance. The only
>> > solution I can think of is to VNC to the instance, copy the public key
>> to my
>> > CA server, sign it, and then write the certificate back into the host
>> (again
>> > via VNC). I cannot ssh without risking a MITM attack. What about using
>> Nova
>> > user-data? User-data is exposed via the metadata service. Metadata is
>> > queried via http (reply transmitted in the clear, susceptible to
>> snooping),
>> > and any compute node can query for any instance's meta-data/user-data.
>> >
>> >
>> >
>> > At this point I have to admit I'm ignorant of details of cloud-init. I
>> know
>> > cloud-init allows specifying SSH private keys (both for users and for
>> SSH
>> > service). I have not yet studied how such information is securely
>> injected
>> > into an instance. I assume it should only be made available via
>> ConfigDrive
>> > rather than metadata-service (again, that service transmits in the
>> clear).
>> >
>> >
>> >
>> > What about providing SSH host certificates as a service in OpenStack?
>> Let's
>> > keep out of scope issues around choosing and storing the CA keys, but
>> the CA
>> > key is per project. What design supports setting up the SSH host
>> certificate
>> > automatically for every VM instance?
>> >
>> >
>> >
>> > I have looked at Vendor Data and I don't see a way to use that, mainly
>> > because 1) it doesn't take parameters, so you can't pass the public key
>> out;
>> > and 2) it's queried over http, not https.
>> >
>> >
>> >
>> > Just as a feasibility argument, one solution would be to modify Nova
>> compute
>> > instance boot code. Nova compute can securely query a CA service asking
>> for
>> > a triplet (private key, public key, SSH certificate) for the specific
>> > hostname. It can then inject the triplet using ConfigDrive. I believe
>> this
>> > securely gets the private key into the instance.
>> >
>> >
>> >
>> > I cannot figure out how to get the equivalent functionality without
>> > modifying Nova compute and the boot process. Every solution I can think
>> of
>> > risks either exposing the private key or vulnerability to a MITM attack
>> > during the signing process.
>> >
>> >
>> >
>> > Your help is appreciated.
>> >
>> >
>> >
>> > --Pino
>> >
>> >
>> > ____________________________________________________________
>> ______________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: OpenStack-dev-request at lists.op
>> enstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171005/c9aaed1f/attachment.html>