[openstack-dev] Supporting SSH host certificates

Blair Bethwaite blair.bethwaite at gmail.com
Fri Oct 6 00:20:23 UTC 2017


A related bug that hasn't seen any love for some time:
https://bugs.launchpad.net/nova/+bug/1613199

On 6 October 2017 at 07:47, James Penick <jpenick at gmail.com> wrote:

> Hey Pino,
>
> mriedem pointed me to the vendordata code [1] which shows some fields are
> passed (such as project ID) and that SSL is supported. So that's good.
>
> The docs on vendordata suck. But I think it'll do what you're looking for.
> Michael Still wrote up a helpful post titled "Nova vendordata deployment,
> an excessively detailed guide"[2] and he's written a vendordata service
> example[3] which even shows keystone integration.
>
> At Oath, we have a system that provides a unique x509 certificate for each
> host, including the ability to sign host SSH keys against an HSM. In our
> case what we do is have Nova call the service, which generates and returns
> a signed (and time limited) host bootstrap document, which is injected into
> the instance. When the instance boots it calls our identity service and
> provides its bootstrap document as a bearer certificate. The identity
> service trusts this one-time document to attest the instance, and will then
> provide an x509 certificate as well as sign the hosts SSH keys. After the
> initial bootstrap the host will rotate its keys frequently, by providing
> its last certificate in exchange for a new one. The service tracks all host
> document and certificate IDs which have been exchanged until their expiry,
> so that a cert cannot be re-used.
>
> This infrastructure relies on Athenz [4] as the AuthNG system for all
> principals (users, services, roles, domains, etc) as well as an internal
> signatory service which signs x509 certificates and SSH host keys using an
> HSM infrastructure.
>
> Instead, you could write a vendordata service which, when called, would
> generate an ssh host keypair, sign it, and return those files as encoded
> data, which can be expanded into files in the correct locations on first
> boot. I strongly suggest using not only using keystone auth, but that you
> ensure all calls from vendordata to the microservice are encrypted with TLS
> mutual auth.
>
> -James
>
>
> 1: https://github.com/openstack/nova/blob/master/nova/api/
> metadata/vendordata_dynamic.py#L77
> 2: https://www.stillhq.com/openstack/000022.html
> 3: https://github.com/mikalstill/vendordata
> 4: https://athenz.io
>
>
> On Fri, Sep 29, 2017 at 5:17 PM, Fox, Kevin M <Kevin.Fox at pnnl.gov> wrote:
>
>> https://review.openstack.org/#/c/222293/
>> ------------------------------
>> *From:* Giuseppe de Candia [giuseppe.decandia at gmail.com]
>> *Sent:* Friday, September 29, 2017 1:05 PM
>> *To:* OpenStack Development Mailing List (not for usage questions)
>> *Subject:* Re: [openstack-dev] Supporting SSH host certificates
>>
>> Ihar, thanks for pointing that out - I'll definitely take a close look.
>>
>> Jon, I'm not very familiar with Barbican, but I did assume the full
>> implementation would use Barbican to store private keys. However, in terms
>> of actually getting a private key (or SSH host cert) into a VM instance,
>> Barbican doesn't help. The instance needs permission to access secrets
>> stored in Barbican. The main question of my e-mail is: how do you inject a
>> credential in an automated but secure way? I'd love to hear ideas - in the
>> meantime I'll study Ihar's link.
>>
>> thanks,
>> Pino
>>
>>
>>
>> On Fri, Sep 29, 2017 at 2:49 PM, Ihar Hrachyshka <ihrachys at redhat.com>
>> wrote:
>>
>>> What you describe (at least the use case) seems to resemble
>>> https://review.openstack.org/#/c/456394/ This work never moved
>>> anywhere since the spec was posted though. You may want to revive the
>>> discussion in scope of the spec.
>>>
>>> Ihar
>>>
>>> On Fri, Sep 29, 2017 at 12:21 PM, Giuseppe de Candia
>>> <giuseppe.decandia at gmail.com> wrote:
>>> > Hi Folks,
>>> >
>>> >
>>> >
>>> > My intent in this e-mail is to solicit advice for how to inject SSH
>>> host
>>> > certificates into VM instances, with minimal or no burden on users.
>>> >
>>> >
>>> >
>>> > Background (skip if you're already familiar with SSH certificates):
>>> without
>>> > host certificates, when clients ssh to a host for the first time (or
>>> after
>>> > the host has been re-installed), they have to hope that there's no man
>>> in
>>> > the middle and that the public key being presented actually belongs to
>>> the
>>> > host they're trying to reach. The host's public key is stored in the
>>> > client's known_hosts file. SSH host certicates eliminate the
>>> possibility of
>>> > Man-in-the-Middle attack: a Certificate Authority public key is
>>> distributed
>>> > to clients (and written to their known_hosts file with a special
>>> syntax and
>>> > options); the host public key is signed by the CA, generating an SSH
>>> > certificate that contains the hostname and validity period (among other
>>> > things). When negotiating the ssh connection, the host presents its
>>> SSH host
>>> > certificate and the client verifies that it was signed by the CA.
>>> >
>>> >
>>> >
>>> > How to support SSH host certificates in OpenStack?
>>> >
>>> >
>>> >
>>> > First, let's consider doing it by hand, instance by instance. The only
>>> > solution I can think of is to VNC to the instance, copy the public key
>>> to my
>>> > CA server, sign it, and then write the certificate back into the host
>>> (again
>>> > via VNC). I cannot ssh without risking a MITM attack. What about using
>>> Nova
>>> > user-data? User-data is exposed via the metadata service. Metadata is
>>> > queried via http (reply transmitted in the clear, susceptible to
>>> snooping),
>>> > and any compute node can query for any instance's meta-data/user-data.
>>> >
>>> >
>>> >
>>> > At this point I have to admit I'm ignorant of details of cloud-init. I
>>> know
>>> > cloud-init allows specifying SSH private keys (both for users and for
>>> SSH
>>> > service). I have not yet studied how such information is securely
>>> injected
>>> > into an instance. I assume it should only be made available via
>>> ConfigDrive
>>> > rather than metadata-service (again, that service transmits in the
>>> clear).
>>> >
>>> >
>>> >
>>> > What about providing SSH host certificates as a service in OpenStack?
>>> Let's
>>> > keep out of scope issues around choosing and storing the CA keys, but
>>> the CA
>>> > key is per project. What design supports setting up the SSH host
>>> certificate
>>> > automatically for every VM instance?
>>> >
>>> >
>>> >
>>> > I have looked at Vendor Data and I don't see a way to use that, mainly
>>> > because 1) it doesn't take parameters, so you can't pass the public
>>> key out;
>>> > and 2) it's queried over http, not https.
>>> >
>>> >
>>> >
>>> > Just as a feasibility argument, one solution would be to modify Nova
>>> compute
>>> > instance boot code. Nova compute can securely query a CA service
>>> asking for
>>> > a triplet (private key, public key, SSH certificate) for the specific
>>> > hostname. It can then inject the triplet using ConfigDrive. I believe
>>> this
>>> > securely gets the private key into the instance.
>>> >
>>> >
>>> >
>>> > I cannot figure out how to get the equivalent functionality without
>>> > modifying Nova compute and the boot process. Every solution I can
>>> think of
>>> > risks either exposing the private key or vulnerability to a MITM attack
>>> > during the signing process.
>>> >
>>> >
>>> >
>>> > Your help is appreciated.
>>> >
>>> >
>>> >
>>> > --Pino
>>> >
>>> >
>>> > ____________________________________________________________
>>> ______________
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe: OpenStack-dev-request at lists.op
>>> enstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>> ____________________________________________________________
>>> ______________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.op
>>> enstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Cheers,
~Blairo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171006/050944e0/attachment-0001.html>


More information about the OpenStack-dev mailing list