<div dir="ltr">Thanks for the quick responses, y'all! I'm working on feedback on the existing development direction with L7 and SSL support, as well as a few of the other things touched on briefly. I agree that slogging through massive responses is going to make this discussion more difficult to follow, so I'll try splitting my responses up in to sub-categorical subject areas that seem to make sense. (These will be forthcoming in the next day or so as I have time.)<div>
<br></div><div>Thanks,</div><div>Stephen</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Feb 11, 2014 at 7:12 AM, Eugene Nikanorov <span dir="ltr"><<a href="mailto:enikanorov@mirantis.com" target="_blank">enikanorov@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Stephen,<div><br></div><div>Thanks a lot for your interest and such detailed post!</div><div><br></div>
<div>I'll try to answer some of your questions/proposals. I also think it makes sense to may be split this into several threads on each feature or model change.</div>
<div>So...</div><div class=""><div><i><font face="arial, helvetica, sans-serif">> <span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">and while a new ‘loadbalancer’ entity has been proposed, I’ve not yet seen an updated data model diagram which plugs this in anywhere: </span><span style="line-height:14.720000267028809px;text-decoration:underline;background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><a href="https://wiki.openstack.org/w/images/e/e1/LBaaS_Core_Resource_Model_Proposal.png" style="line-height:14.720000267028809px;text-decoration:none" target="_blank">https://wiki.openstack.org/w/images/e/e1/LBaaS_Core_Resource_Model_Proposal.png</a></span></font></i></div>
</div><div><font face="arial, helvetica, sans-serif">The proposed model is described here, although without a picture: <a href="https://wiki.openstack.org/wiki/Neutron/LBaaS/LoadbalancerInstance" target="_blank">https://wiki.openstack.org/wiki/Neutron/LBaaS/LoadbalancerInstance</a></font></div>
<div class="">
<div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><span style="font-family:Arial;line-height:17.25px;white-space:pre-wrap"><i>> These features are certainly among the ‘essential’ features our production customers use in other load balancer products, all of which seem like good ideas to eventually add to OpenStack’s load balancer functionality:</i></span><font face="arial, helvetica, sans-serif"><br>
</font></div><div><span style="font-family:Arial;line-height:17.25px;white-space:pre-wrap"><i><br></i></span></div></div><div><font face="Arial"><span style="line-height:17.25px;white-space:pre-wrap">As Sam mentioned, L7 rules and SSL is actively developed right now, autoscaling is something that generally out of scope of Neutron project and more in the scope of Heat project.</span></font></div>
<div><span style="background-color:transparent;white-space:pre-wrap;line-height:1.15"><font face="arial, helvetica, sans-serif">Automated provisioning of load balancer devices as was said is what 'service vm framework' is trying to address.</font></span></div>
<div class="">
<div><font face="Arial"><span style="line-height:17.25px;white-space:pre-wrap"><br></span></font></div><div><font face="Arial"><span style="line-height:17.25px;white-space:pre-wrap">> </span></font><span style="font-family:Arial;font-style:italic;line-height:17.25px;white-space:pre-wrap">Multiple load balanced services per floating IP</span></div>
</div><div>That is what blueprint 'multiple vips per pool' is trying to address.</div><div><br></div><div>I'll study your proposal more carefully before diving into more detailed discussion.</div><div>Feel free to reach me on irc or via email!</div>
<div><br></div><div>Thanks,</div><div>Eugene!</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Tue, Feb 11, 2014 at 4:45 AM, Stephen Balukoff <span dir="ltr"><<a href="mailto:sbalukoff@bluebox.net" target="_blank">sbalukoff@bluebox.net</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr"><span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="line-height:1.15;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Howdy folks!</span><br>
</p><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Let me start by apologizing for the length of this e-mail. Over the past several months I’ve been trying to get up to date on the current status of LBaaS in the OpenStack community, and from what I can tell, it looks like Neutron LBaaS is the project that is both the one most open to new contributors, as well as the one seeing the most active development at this time. I have a proposal I want to discuss, and I understand that this mailing list is probably the best place to attempt a discussion of Neutron LBaaS, where it’s at, and where it’s going. If I should be posting this elsewhere, please excuse my newbishness and point me in the right direction!</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:bold">Background</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">The company I work for has developed its own cloud operating system over the past several years. Most recently, like many organizations, we’ve been making a big shift toward OpenStack and are attempting to both mold our current offerings to work with this platform, and contribute improvements to OpenStack as best we’re able. I am on the team that built two successive versions of the load balancer product (in addition to having experience with F5 BIG-IPs) which works in our cloud operating system environment. It’s basically a software load balancer appliance solution utilizing stunnel and haproxy at its core, but with control, automation, and API built in-house with the goal of meeting the 90% use case of our customer base (which tend to be web applications of some kind).</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:bold">Neutron LBaaS today</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Looking at the status of Neutron LBaaS right now, as well as the direction and rate of progress on feature improvements (as discussed in the weekly IRC meetings), it seems like Neutron LBaaS accomplishes several features of one particular competitor’s load balancer product, but still falls short of offering a product able to meet 90% of consumer’s needs, let alone a significant number of the basic features almost every commercial load balancer appliance is able to do at this point. (See the bulleted list below.) I know there’s been talk of adding functionality to the Neutron LBaaS solution so that drivers for individual commercial products can extend the API as they see fit. While this might be ultimately necessary to expose certain extremely useful features of these products, I think there’s a more fundamental problem with the Neutron LBaaS right now: Specifically, I think the model we’re using here is too simplistic to effectively support more advanced load balancer features without significant (and annoying) hack-ish workarounds for deficiencies in the model.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">I should mention that I’m going off this model, as it appears to correspond with Neutron LBaaS as it exists in Havana (and I think Icehouse-- since Icehouse is unlikely to get the SSL offloading and L7 features currently under development), and while a new ‘loadbalancer’ entity has been proposed, I’ve not yet seen an updated data model diagram which plugs this in anywhere: </span><a href="https://wiki.openstack.org/w/images/e/e1/LBaaS_Core_Resource_Model_Proposal.png" style="text-decoration:none" target="_blank"><span style="font-size:15px;font-family:Arial;background-color:transparent;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">https://wiki.openstack.org/w/images/e/e1/LBaaS_Core_Resource_Model_Proposal.png</span></a><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Given the above model, it looks like significant changes will need to happen to support the following features in a “non-hackish” kind of way. These features are certainly among the ‘essential’ features our production customers use in other load balancer products, all of which seem like good ideas to eventually add to OpenStack’s load balancer functionality:</span></p>
<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Multiple pools per VIP (ie. “layer 7” support)</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Multiple load balanced services per floating IP</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">SSL offloading (with SNI support)</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">High Availability</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Automated provisioning of load balancer devices</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Automated scaling of load-balancer services</span></p></li></ul><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:bold">New Model Proposal</span></p><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">So, having already solved this problem twice for our legacy cloud operating system, I’d like to propose a model change which will ease in the addition of the above features when they eventually get added. These models are based closely on our Blocks Load Balancer version 2 (BLBv2) product, which, again, is really just a software load balancer appliance based on stunnel + haproxy (with our own glue added). I’ll attach these models to this e-mail (and will happily provide the .dot files I used to generate these graphs to anyone who wants them -- I like diagrams whose format plays nicely with revision control systems like git).</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">The first model closely resembles what we do today with BLBv2. Significant differences being that we like the use of cascading attributes (which simplifies provisioning of new pools, pool members, etc. as they inherit attributes from less-specific contexts), and we never saw the need to separate out “stats” and “health monitoring” to their own data model representations because these tend to be closely bound to front-ends or back-ends (respectively). Also note that I realize it’s possible for certain attributes (eg. member IP) to be gleaned from Nova. This might help from a security perspective (ie. a tenant can only add nodes that are part of their cluster, rather than any IP arbitrarily). But I also realize that sometimes it’s advantageous to add “members” to a pool that aren’t actually part of a given OpenStack cluster. Also, I don’t know how tightly we want Neutron LBaaS to be coupled with Nova in this case. I think this is worth a discussion, in any case.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">The second model more closely resembles an enhancement of the current Neutron LBaaS model, including the (IMO unnecessary) data models representing the healthmonitor and stats entities, and no cascading attributes.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">In both models, I’ve split up the current Neutron LBaaS concept of a “VIP” in the current model into “instance” and “listener.” “Instance” in this case is essentially similar to a floating IP (and in an HA configuration, actually would be a floating IP, though not in the typical “Neutron floating IP” sense). It’s an IP address assigned to a tenant (internal or external) on which they can set up multiple listening services. The “listener” is one such listening service (in the case of our appliance, a single haproxy or stunnel instance bound to a given TCP port).</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:bold">Benefits of a new model</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">If we were to adopt either of these data models, this would enable us to eventually support the following feature sets, in the following ways (for example):</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">SSL offloading (with SNI support)</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">This should mostly be self-evident from the model, but in any case, a TLS certificate should be associated with a listener. The tls_certificate_hostname table is just a list of all the hostnames for which a given SSL certificate is valid (CN plus all x509 subject alternative names). Technically, this can be easily derived from the certificate itself on the fly, but in our code we found we were referring to this list in enough places it was simpler to enumerate it whenever a certificate was imported.<br>
<br>SNI support is accomplished by associating multiple certificates to a given listener and marking one as the ‘default’. (Current implementations of stunnel do SNI just fine, including multiple wildcard certificates.) If you’re a load balancer appliance vendor that can’t do SNI, then just using the default cert in this model and ignoring any others probably makes sense.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Multiple pools per VIP</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">This is accomplished, in the haproxy world, through the use of ACLs (hence the ACL join table). Note that I’ve not seen any discussion as of yet regarding what kinds of layer-7 functionality Neutron LBaaS ought to support-- that is, what is a minimally viable feature set here that both customers are asking for and that vendors can support. In our customer base, we have found that the following kinds of acls are the ones most often used:</span></p>
<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">ACL based on server hostname (eg. “<a href="http://api.example.com" target="_blank">api.example.com</a> goes to one pool, </span><a href="http://www.example" style="text-decoration:none" target="_blank"><span style="font-size:15px;background-color:transparent;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">www.example</span></a><span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">.com goes to another)</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">ACL based on URL path prefix (eg. URLs starting with “</span><a href="http://www.example.com/api" style="text-decoration:none" target="_blank"><span style="font-size:15px;background-color:transparent;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">/api</span></a><span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">” go to one pool, all others go to another)</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">ACL based on client IP or network range</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">ACL based on cookie</span></p></li></ul><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">In any case, more discussion needs to happen in general about this as we add L7 support to Neutron LBaaS, whether or not y’all like my model change proposal.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Multiple load balanced services per floating IP</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Obvious use case is HTTP and HTTPS listeners on the same IP. Hopefully this is clear how this is now accomplished by splitting up “VIP” into “instance” and “listener” entities.</span></p>
<br><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">High Availability</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">This is where the “cluster” concept of the model comes into play. A cluster in this case indicates one or more “load balancer” objects which carry similar configuration for balancing all the instances and listeners defined and assigned to it. If the cluster_model is “active-standby” then the presumption here is that you’ve got two load balancer entities (which could be software appliances or vendor appliances) which are configured to act as a highly available pair of load balancers.<br>
<br>Note that non-HA topologies also work fine with this model. In this case, the cluster would have only one load balancer associated with it.</span></p><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">(Full disclosure: In our environment, at the present time we operate our BLBv2 boxes exclusively in an active-standby HA configuration, where the cluster IP addresses (both IPv4 and IPv6) are floating IPs that the two nodes keep alive using corosync and pacemaker. I think we can accomplish the same thing with a simpler tool like ucarp. We also exclusively use layer-3 routing to route “instance” addresses to the right cluster’s ha_ip_address. I think this is an as of yet unsolved “routed mode” version of load balancing that I’ve seen discussed around Neutron LBaaS.)</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Also note that although we don’t do this with BLBv2 right now, it should be possible to set up an n-node active-active load balancer cluster by having a flow-based router that lives “above” the load balancers in a given network topology, and is configured to split flows between the load balancers for incoming TCP connections. In theory, this makes for being able to scale load balancer capacity “infinitely” horizontally, so long as your flow-based router can keep up, and as long as you’ve got network and compute capacity in your pools to keep up. This is extremely important for SSL offloading (as I’ll talk about later). And given the discussions of distributed routing going on within the Neutron project as a whole, sounds like the flow-based router might eventually also scale well horizontally.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Automated provisioning of load balancer devices</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Since load balancers are an entity in this model, it should be possible for a given driver to spawn a new load balancer, populate it with the right configuration, then associate it with a given cluster.<br>
<br>It should also be possible for a cloud administrator to pre-provision a given cluster with a given vendor’s load balancer appliances, which can then be used by tenants for deploying instances and listeners.</span></p>
<br><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Automated scaling of load-balancer services</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">I talked about horizontal scaling of load balancers above under “High Availability,” but, at least in the case of a software appliance, vertical scaling should also be possible in an active-standby cluster_model by killing the standby node, spawning a new, larger one, pushing instance/listener configuration to it, flip-flopping the active/standby role between the two nodes, and then killing and respawning the remaining node in the same way. Scale-downs should be doable the same way.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:bold">Backward compatibility with old model workflows</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">I realize that, especially if the old model has seen wide-scale adoption, it’s probably going to be necessary to have a tenant workflow which is backward compatible with the old model. So, let me illustrate in pseudo code, one way of supporting this with the new model I’ve proposed. I’m going off the tenant workflow described here:</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><ol style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:15px;margin-left:47px;font-family:Arial">
<p dir="ltr" style="line-height:1.2988635843450373;margin-top:3pt;margin-bottom:1pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Create the load balancer pool</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:15px;margin-left:47px;font-family:Arial">
<p dir="ltr" style="line-height:1.2988635843450373;margin-top:3pt;margin-bottom:1pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Create a health monitor for the pool and associate it with the pool</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:15px;margin-left:47px;font-family:Arial"><p dir="ltr" style="line-height:1.2988635843450373;margin-top:3pt;margin-bottom:1pt">
<span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Add members (back-end nodes) to the pool</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:15px;margin-left:47px;font-family:Arial">
<p dir="ltr" style="line-height:1.2988635843450373;margin-top:3pt;margin-bottom:1pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Create a VIP associated with the pool.</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:15px;margin-left:47px;font-family:Arial">
<p dir="ltr" style="line-height:1.2988635843450373;margin-top:3pt;margin-bottom:1pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">(Optional) Create a floating IP and point it at the VIP for the pool.</span></p>
</li></ol><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">So, using the proposed model:</span></p><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">1. Client issues request to create load balancer pool</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"><span style="white-space:pre-wrap"> </span>* Agent creates pool object, sets legacy_network attribute to network_id requested in client API command</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial">2. Create a health monitor for the pool and associate it with the pool</span><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"><span style="white-space:pre-wrap"> </span>* Agent creates healthmonitor object and associates it with the pool. For v1 model, agent sets appropriate monitor cascading attributes on pool object.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.2988635843450373;margin-top:3pt;margin-bottom:1pt">
<span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">3. </span><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial">Add members (back-end nodes) to the pool</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"><span style="white-space:pre-wrap"> </span>* Agent creates member objects and associates them with the pool.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial">4. Create a VIP associated with the pool.</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial"><span style="white-space:pre-wrap"> </span></span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">* Agent checks to see if cluster object already exists that tenant has access to that also has access to the legacy_network set in the pool.</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>* If not, create cluster object, then spawn a virtual load balancer node. Once the node is up, associate it with the cluster object.</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"><span style="white-space:pre-wrap"> </span>* Agent creates instance object, assigns an IP to it corresponding to the cluster’s network, associates it with the cluster</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"><span style="white-space:pre-wrap"> </span>* Agent creates listener object, associates it with the instance object</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"><span style="white-space:pre-wrap"> </span>* Agent creates ‘default’ acl object, associates it with both the listener and the pool</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"><span style="white-space:pre-wrap"> </span>* Agent pushes listener configuration to all load balancers in the cluster</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial">5. (Optional) Create a floating IP and point it at the VIP for the pool.</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial"><span style="white-space:pre-wrap"> </span></span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">* Client creates this in the usual way, as this operation is already strictly out of the scope of the Neutron LBaaS functionality.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">Beyond the above, I think it’s possible to create functional equivalents to all the existing neutron load balancer API commands that work with the new model I’m proposing.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:bold">Advanced workflow with proposed model</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">There’s actually quite a bit of flexibility here with how a given client can go about provisioning load balancing services for a given cluster. However, I would suggest the following workflow, if we’re not going to use the legacy workflow:</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">1. OpenStack admin or Tenant creates a new cluster object, setting the cluster_model. OpenStack admin can specify whether cluster is shared.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">2. Depending on permissions, OpenStack admin or Tenant creates load balancer object(s) and associates these with the cluster. In practice, only OpenStack admins are likely to add proprietary vendor appliances. Tenants may likely only be able to add virtual appliances (which are run as VMs in the compute environment, typically).</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">3. Client creates instance, specifying the cluster_id it should belong to.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">4. Client creates listener, specifying the instance_id it should belong to. (Note at this point, for non-HTTPS services with haproxy at least, we know enough that the service can already be taken live.) If client just wants service to be a simple redirect, specify that in the listener attributes and then do only the next step for HTTPS services:</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">5. For HTTPS services, client adds tls_certificate(s), specifying listener_id they should belong to.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">6. Client creates pool. For v1 model, client can set monitoring attributes at this point (or associate these with the instance, listener, etc. Again, there’s a lot of flexibility here.)</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">7. Client creates healthmonitor, specifying pool_id it should be associated with.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">8. Client creates 0 or more member(s), specifying pool_id they should be associated with.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">9. Client associates pool to listener(s), specifying ACL string (or default) that should be used for L7 pool routing. First pool added to a listener is implicitly default.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">10. Repeat from step 6 for additional back-end pools.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">11. Repeat from step 4 for additional services listening on the same IP.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">12. Repeat from step 3 for additional services that should listen on different IPs yet run on the same load balancer cluster.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">A note on altering the above objects:</span></p>
<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Adding or removing a load balancer from a cluster will usually require an update to all existing (or remaining) load balancers within the cluster, and specifically for all instances and listeners in the cluster. It may be wise not to allow the cluster_model to be changed after the cluster is created, as well. (Because switching from a HA configuration to a single-node configuration on the fly requires a level of agility that is problematic to reliably achieve.)</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Of course any changes to any of the instances, listeners, tls_certificates, pools, members, etc. all require any linked objects to have updates pushed to the load balancers as well.</span></p>
</li></ul><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial;font-weight:bold">The case for software virtual appliance</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">You may notice that the model seems to inherently favor load balancer topologies that use actual vendor appliances or software virtual appliances, rather than the default haproxy method that ships readily with Neutron LBaaS in its current form. (That is, the default driver where a given “VIP” in this case ends up being an haproxy process that runs directly on the Neutron server in a given openstack topology.) While the models I’m proposing should still work just fine with this method, I do tend to bias toward having the load balancing role live somewhere other than core routing functionality because it’s both more secure and more scalable. Specifically:</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial">Security</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">In our experience, when a given customer’s web application cluster gets attacked by malicious entities on the internet, the load balancers in the cluster are the most commonly attacked components within the cluster. As such, it makes sense to not double-up its role in the cluster topology with any other role, if possible. One doesn’t want a compromised load balancer to, for example, be able to alter the network topology of the tenant network or disable the firewall protecting more vital components.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">Also, having load balancing live somewhere separate from routing functionality lends more flexible ways to add other security features to the cluster (like, say, a web application firewall.)</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial;font-weight:bold"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial">Scalability</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">SSL offloading is the most CPU intensive task that most load balancers will do. And while the openSSL library is fairly multi-threaded, there are enough single-threaded “critical code” components within what it is doing (eg. SSL cache access) that in our experience it might as well be limited to a single core. Therefore, the upper limit on how many new connections per second a given SSL termination process (eg. stunnel) can handle is ultimately going to be limited by the clock speed of the processor. Even with the hardware acceleration on modern Intel procs. As the strategy within CPU manufacturing has been pushed by physical limitations to go with more cores rather than faster clocks, and as the requirements for key length are going to ever be increasing, this means that it’s going to become more and more expensive to do SSL termination on a single node. Even with better parallelization of the SSL functionality, at a certain point the linux scheduler becomes the limiting factor as it churns through thousands of context switches per second.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">To give some context to this: In our experience, a non-SSL haproxy process can handle several tens of thousands of new connections per second with the right kernel tuning. But on current-generation processors, a single stunnel process can handle only about 1200-1400 new connections per second when using 1024-bit RSA keys (depreciated), and 200-300 new connections per second using 2048-bit RSA keys (current standard), before becoming CPU-bound. When 4096-bit keys become the standard, we can expect another 5x decrease in performance. Turning keepalive on can help a lot, depending on the web application we’re fronting. (That SSL handshake upon starting a new connection is by far the most expensive part of this process.) But ultimately, it should be clear the only real scaling strategy right now is to go horizontal and have many more processes terminating client connections.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">So given the above, it seems to me the only real scaling strategy here must be able to run a load balancing cluster across many physical machines.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-style:italic;font-size:15px;white-space:pre-wrap;font-family:Arial">Other Benefits of Appliance model</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">There are a couple other benefits that bear mentioning that the appliance model has over the run-haproxy-on-the-nework-node model:</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Clients are often not agnostic about what load balancer product they use for various reasons. Even though this largely goes against the whole philosophy of “cloud-like” behavior, I can see a need for clients to be able to specify specific load balancers a given VIP gets deployed to.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">In a distributed routing scenario, appliances fit into the topology better than attempting to do, say, session management across several network nodes.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">The software virtual appliance model closely parallels the vendor hardware appliance model-- meaning that if we were to adopt a software virtual appliance model and (eventually) scrap the haproxy-on-the-network-node model, we’ll have fewer code paths to test and maintain.</span></p>
</li></ul><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial;font-weight:bold">Stuff missing from my proposed models</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">The following are not yet well defined in the model I’ve given, though most of these should be able to be added with relatively minor model changes:</span></p>
<ul style="margin-top:0pt;margin-bottom:0pt"><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Stats model that automatically sends data to other OpenStack components (like ceilometer)</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Logging offload (though, again, this should be pretty trivial to add)</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Other load balancing strategies/topologies than the ones I’ve mentioned in this e-mail.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Discussion around the secure transmission of SSL private keys needs to continue. In our environment, we use an SSL encrypted REST service listening on the load balancer’s ip address which does both client and server certificate verification, thereby handling both auth and encryption. I understand discussions around this are already underway for how to handle this in the current Neutron LBaaS environment. I see no reason why whatever we figure out here wouldn’t directly translate to the models I’ve proposed.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:15px;vertical-align:baseline;white-space:pre-wrap">Tenant network ID is conspicuously absent from my model at present, though it’s somewhat implied with the IP address information that gets set in the various objects. I’m guessing tenant network would apply at the “cluster” level in my models, but need to understand more about how and where it’s used in the current Neutron LBaaS data model before I can be certain of that. Also, I would guess some discussion needs to happen around the case of providers who use a vendor's appliance which may need to have access to multiple tenant networks.</span></p>
</li></ul><br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial;font-weight:bold">Next Steps for Me</span><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">I am fully aware that in the OpenStack community, code talks and vaporware walks. My intent in writing this excessively long e-mail was to broach the subject of a significant model change that I believe can really help us not to paint ourselves into a corner when it comes to features that are undoubtedly going to be desirable if not essential for OpenStack’s load balancing functionality to have.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">So, while we discuss this, in the mean time my intention is to concentrate on creating the completely open-source software virtual appliance that I think is glaringly absent from Neutron LBaaS right now, working through the driver model presently in Neutron LBaaS. As far as I’m aware, nobody else is working on this right now, so I don’t think I’d be duplicating effort here. (If you are working on this-- let’s talk!) This problem will have to be solved for the model I’ve proposed anyway.</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">Also, since we’ve essentially solved the software appliance problem twice already, I don’t imagine it will take all that long to solve again, in the grand scheme of things. (I would simply open-source the latest incarnation of our “Blocks Load Balancer”, but there are a few components therein that are very specific to our legacy cloud OS that I’ve not gotten definite clearance on releasing. Besides, all of our glue is written in a combination of perl, ruby, and shell -- and I understand with OpenStack it’s python or the highway.)</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">So there we have it! I would very much appreciate your feedback on any of the above!</span></p>
<br><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"></span><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">Thanks,</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial">Stephen</span></p><span><font color="#888888"><div>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;font-family:Arial"><br>
</span></div></font></span></span><span><font color="#888888"><div><br></div>-- <br><span></span>Stephen Balukoff
<br>Blue Box Group, LLC
<br><a href="tel:%28800%29613-4305%20x807" value="+18006134305" target="_blank">(800)613-4305 x807</a>
</font></span></div>
<br></div></div>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><span></span>Stephen Balukoff
<br>Blue Box Group, LLC
<br>(800)613-4305 x807
</div>