<div dir="ltr">German:<div><br></div><div>I'm hearing from a lot of different sources / organizations on this list that re-encryption at the load balancer is a must-have feature. And I was already part of previous discussions on SSL functionality. ( <a href="https://wiki.openstack.org/wiki/Neutron/LBaaS/SSL">https://wiki.openstack.org/wiki/Neutron/LBaaS/SSL</a> )</div>
<div><br></div><div>Also, even if the load balancer does support re-encryption on the back-end, this doesn't prevent users from using a VPN-based solution either.</div><div><br></div><div>Stephen</div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Mon, Apr 21, 2014 at 11:51 AM, Eichberger, German <span dir="ltr"><<a href="mailto:german.eichberger@hp.com" target="_blank">german.eichberger@hp.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Despite there are some good use cases for the re-encryption I think it’s out of scope for a Load Balancer. We can defer that functionality to the VPN – as long
as we have a mechanism to insert a LoadBalancer as a VPN node we should get all kind of encryption infrastructure “for free”.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I like the Unix philosophy of little programs doing one task very well and can be chained. So in our case we might want to chain a firewall to a load balancer
to a VPN to get the functionality we want.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thoughts?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">German<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Stephen Balukoff [mailto:<a href="mailto:sbalukoff@bluebox.net" target="_blank">sbalukoff@bluebox.net</a>]
<br>
<b>Sent:</b> Friday, April 18, 2014 9:07 PM</span></p><div class=""><br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
</div><b>Subject:</b> Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question<u></u><u></u><p></p>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi y'all!<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Carlos: When I say 'client cert' I'm talking about the certificate / key combination the load balancer will be using to initiate the SSL connection to the back-end server. The implication here is that if the back-end server doesn't like
the client cert, it will reject the connection (as being not from a trusted source). By 'CA cert' I'm talking about the certificate (sans key) that the load balancer will be using the authenticate the back-end server. If the back-end server's "server certificate"
isn't signed by the CA, then the load balancer should reject the connection.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Of course, the use of a client cert or CA cert on the load balancer should be optional: As Clint pointed out, for some users, just using SSL without doing any particular authentication (on either the part of the load balancer or back-end)
is going to be good enough.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Anyway, the case for supporting re-encryption on the load-balancers has been solidly made, and the API proposal we're making will reflect this capability. Next question:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">When specific client certs / CAs are used for re-encryption, should these be associated with the pool or member?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I could see an argument for either case:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><b>Pool</b> (ie. one client cert / CA cert will be used for all members in a pool):<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">* Consistency of back-end nodes within a pool is probably both extremely common, and a best practice. It's likely all will be accessed the same way.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">* Less flexible than certs associated with members, but also less complicated config.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">* For CA certs, assumes user knows how to manage their own PKI using a CA.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><b>Member</b> (ie. load balancer will potentially use a different client cert / CA cert for each member individually):<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">* Customers will sometimes run with inconsistent back-end nodes (eg. "local" nodes in a pool treated differently than "remote" nodes in a pool).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">* More flexible than certs associated with members, more complicated configuration.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">* If back-end certs are all individually self-signed (ie. no single CA used for all nodes), then certs must be associated with members.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">What are people seeing "in the wild"? Are your users using inconsistently-signed or per-node self-signed certs in a single pool?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Stephen<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Fri, Apr 18, 2014 at 5:56 PM, Carlos Garza <<a href="mailto:carlos.garza@rackspace.com" target="_blank">carlos.garza@rackspace.com</a>> wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Apr 18, 2014, at 12:36 PM, Stephen Balukoff <<a href="mailto:sbalukoff@bluebox.net" target="_blank">sbalukoff@bluebox.net</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<div>
<p class="MsoNormal">Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.)
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the
proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"> If you choose to support re-encryption on your service then you are free to charge for the extra CPU cycles. I'm convinced re-encryption and SslTermination is general needs to be mandatory but I think the API should allow them to be
specified. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server,
it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"> Yes but considering you have no problem advocating pure ssl termination for your customers(Decryption on the front end and plain text) on the back end I'm actually surprised this disturbs you. I would recommend users use Straight SSL
passthrough or re-enecryption but I wouldn't force this on them should they choose naked encryption with no checking.
<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a "pool" or to a "member.")<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal">When you say client Cert are you referring to the end users X509 Certificate (To be rejected by the backend server)or are you referring to the back end servers X509Certificate which the loadbalancer would reject if it discovered the back
end server had a bad signature or mismatched key? I am speaking of the case where the user wants re-encryption but wants to be able to install CA certificates that sign backend servers Keys via PKIX path building. I would even like to offer the customer the
ability to skip hostname validation since not every one wants to expose DNS entries for IPs that are not publicly routable anyways. Unless your suggesting that we should force this on the user which likewise forces us to host a name server that maps hosts
to the X509s subject CN fields. Users should be free to validate back end hostnames, just the subject name and key or no validation at all. It should be up to them.<u></u><u></u></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It's a bit of a rabbit hole, eh.<u></u><u></u></p>
</div>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">Stephen<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Fri, Apr 18, 2014 at 10:21 AM, Eichberger, German <<a href="mailto:german.eichberger@hp.com" target="_blank">german.eichberger@hp.com</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Stephen,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward
field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme
for HP Cloud’s swift storage.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">German</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Stephen Balukoff [mailto:<a href="mailto:sbalukoff@bluebox.net" target="_blank">sbalukoff@bluebox.net</a>]
<br>
<b>Sent:</b> Friday, April 18, 2014 8:22 AM</span><u></u><u></u></p>
<div>
<p class="MsoNormal"><br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><b>Subject:</b> [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">Howdy, folks!<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer
makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they
didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Stephen<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal">--
<br>
Stephen Balukoff <br>
Blue Box Group, LLC <br>
<a href="tel:%28800%29613-4305%20x807" target="_blank">(800)613-4305 x807</a> <u></u>
<u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <br>
Stephen Balukoff <br>
Blue Box Group, LLC <br>
<a href="tel:%28800%29613-4305%20x807" target="_blank">(800)613-4305 x807</a> <u></u>
<u></u></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <br>
Stephen Balukoff <br>
Blue Box Group, LLC <br>
<a href="tel:%28800%29613-4305%20x807" value="+18006134305" target="_blank">(800)613-4305 x807</a> <u></u><u></u></p>
</div>
</div></div></div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><span></span>Stephen Balukoff
<br>Blue Box Group, LLC
<br>(800)613-4305 x807
</div>