Open Stack

> -----Original Message-----
> From: Daniel P. Berrange [mailto:berrange at redhat.com]
> Sent: 24 October 2012 13:48
> To: Clark, Robert Graham
> Cc: OpenStack Development Mailing List
> Subject: Re: [openstack-dev] Secure RPC
>
> On Wed, Oct 24, 2012 at 12:11:50PM +0000, Clark, Robert Graham wrote:
> > > -----Original Message-----
> > How many of the Nova API messages really need to be encrypted? If it's
> > few then I'd suggest there are probably better approaches than
> > encrypting everything.
>
> Not many messages need to have encryption - minimally just those which are
> transporting encryption keys, but once you add support for encryption to the
> RPC layer the decision about whether to encrypt everything or only some
> messages comes down to two core factors
>
>  - Whether it is a unicast or broadcast message
>  - Whether the computational overhead is acceptable.
>
> Given that modern CPUs include AES support in hardware, the computational
> overheads may not matter significantly. TBD based on real world testing of
> course.
>
> For broadcast messages you have the problem of what key to encrypt with.
> So it is probably simplest to just not encrypt broadcast messages.
>
> >           A big issue in nova is that various components have
> > different attack surfaces, which makes them more or less likely to be
> > compromised than other components. Having each component sign messages
> > before transmission would deal with the most immediate threat: An
> > attacker on your network can control _everything_ by watching the wire
> > and injecting their own packets.
>
> For the purposes of disk encryption, I've been working on the basis that in 
> Nova
> the compute nodes should be considered the least trusted part of the system
> and their access / abilities must be tightly controlled / limited as 
> compared to
> the api / schedular nodes.
>
> > The next step (that's really needed before encryption makes much
> > sense) is to shoehorn some sort of RBAC into Nova so that when a
> > component receives a signed message (signing proves it's from "X") it
> > can check that "X" is in the right role to send that message. Once
> > this is done we start having some pretty robust mechanisms for
> > controlling data inside of Nova.
>
> Yes, you absolutely need to have a way to identify what role a sender is
> permitted to be in. For example, to be able to prevent a compromised compute
> node from sending a 'boot VM' message - only the schedular can be allowed to
> send that particular message. And so on.
>
> > Of the two things I've mentioned so far, signing and RBAC, signing
> > probably seems to be the easiest but you need to decide how you're
> > going to sign and what crypto primitives we're going to use (this is
> > relevant to the encryption discussion too). At the design summit
> > someone suggested sending a X.509 public certificate along with a
> > signed call - this is kind of crazy, there's no way we can send around
> > 1.5kb of extra data for each message, you could send the public key
> > lets assume we're using 2048bit RSA - that still means we're adding
> > 256bytes to every message. In either case you'd need a way to check
> > the public component was valid and not revoked. I think there are
> > probably some smart things that can be done by just sending an X.509
> > fingerprint or a keyID and having centrally distributed,
> > cryptographically verifiable local lookup caches - (this is also a
> > neat way of sidestepping any CRL verification issues : if the cert
> > isn't in your trust cache, it's not trusted).
>
> I would be interested to know the kind of data volumes / patterns seen for 
> the
> RPC service in the real world, before ruling out the use of x509 certs. A 
> supposed
> 1.5kb overhead per call needs to be put in context of the overall traffic &
> capacity before we judge that it is unacceptable overhead. Just sending an 
> x509
> fingerprint sounds interesting idea, but I'm not sure what security 
> implications
> that would have. One of the things I try to remember is that I'm not Bruce
> Schneier, so it is best to aim for tried-and-tested design patterns, rather 
> than try
> to invent some special/clear new security system ;-P
>
> Personally I'm not so worried about the network traffic volumes.
> The more important concerns to me are around the overall resillience &
> computational scalability of the system. In particular making sure you don't 
> have
> to do something crazy like check keystone (or another centralized auth 
> service)
> to validate every single RPC call received.
> Also trying to avoid putting any significant administrative setup or ongoing
> burden on deploying openstack; retaining the flexility to quickly 
> re-configure the
> location of nova services, and the speed with which you can react to an 
> hostile
> attack and isolate compromised services.
>
> > At the summit there seemed to be lots of confusion over how this would
> > actually work and a whole bunch of frankly quite worrying ideas were
> > being thrown around. This is part of the reason that the OpenStack
> > Security Group was formed, I can't wait to see your design document, I
> > hope that it blows away all of the concerns I've listed above but if
> > it doesn't perhaps you would be open to working with the OSSG to lock
> > it down or even joining the OSSG if you're "security inclined"
> > generally. The OSSG is a group of stackers who come from good security
> > backgrounds and want to help developers get security right first time
> > around by providing review, advice and consultation
> > https://launchpad.net/~openstack-ossg
>
> Yes, I'm already in the OSSG team.
>
> One of the reasons I've not raised any discussions thus far is that I've not
> finished writing up my ideas in a form that's presentable for discussion. I 
> only
> mention it today since the topic was brought up recently at the Summit 
> (which I
> was unable to attend) and I want to make sure all interested parties know 
> about
> each other's ideas / work to avoid duplication of effort.
>
> Regards,
> Daniel
> --

Thanks Daniel, I think we're largely aligned. In my organisation we will pay 
the overhead for crypt more often than not but have found in some areas, even 
when using AES acceleration (which btw, I don't think you can do in OpenStack 
without switching out the M2Crypto calls for pyOpenSSL) that we struggle on 
speed. That said as we're just talking about API calls here the cost may be 
acceptable.

It seems that Eric is suggesting something far more extensible than I had 
thought. We could have done with significantly more time to run through this 
at the summit. Unfortunately I wasn't able to attend Eric's conference talk 
but I'm looking forward to seeing the video.

None of us can compare to Bruce I'm sure (all praise Schneier!) and I'm 
regularly guilty of running off down crypto rabbit holes - allow me to bring 
this back up to the high level momentarily. Is it likely that signing and 
encryption would both be predicated on a-symmetric cryptography? If that's the 
case then the primitives in use will likely be the same and perhaps we should 
come to a point where signing/encryption are both options that pivot on the 
same extension? Or perhaps look a little more closely at the work Eric has 
already done on extending RPC so that we can come up with something flexible 
enough to support both.

As before, looking forward to seeing your design, feel free to mail direct to 
me before it's polished if you'd like some early feedback.

Cheers
-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6190 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121024/f0991612/attachment-0001.bin>