<div dir="ltr">Hi Stephen,<div><br></div><div>Thanks for the great document. As I promised, I'll try to make a few action items out if it.</div><div>First of all, I'd like to say that the API you have proposed is very close to what is proposed in the blueprint <a href="https://review.openstack.org/#/c/89903/">https://review.openstack.org/#/c/89903/</a> with several differences I'd like to address here and make them action items.</div>
<div><br></div><div>So, first of all, I think that API described in the doc seems to account for all cases we had in mind, i didn't check on case-by-cases basis, it's just a first glance impression looking at REST resource set and their attributes.</div>
<div><br></div><div>General idea of the whole API/obj model improvement is that we create a baseline for all advanced features/usecases that we have in the roadmap. Which means that those features then can be added incrementally. Incrementally means that resources or attributes might be added, but workflow remains and backward compatibility is preserved. That was not the case with multiple listeners and L7.</div>
<div><br></div><div>So, a couple on general comments:</div><div><br></div><div>1. Whole discussion about API/obj model improvement had the goal of allowing multiple pools and multiple listeners.</div><div>For that purpose loadbalancer instance might be an extra. The good thing about 'loadbalancer' is that it can be introduced in the API in incremental way. </div>
<div>So, VIP+listeners itself is already quite flexible construct (where VIP is a root object playing loadbalancer role) that addresses our immediate needs.</div><div>So I'd like to extract loadbalancer API and corresponding use cases in another blueprint. </div>
<div>You know that loadbalancer concept has raised very heated discussions so it makes sense to continue discussing it separately, keeping in mind that introducing loadbalancer is not very complex and it may be done on top of the VIPs/listeners API</div>
<div><br></div><div>2. SSL-related objects. SSL is rather big deal, both from API and object model, it was a separate blueprint in Icehouse and i think it makes sense to work separately on it.</div><div>What I mean is that ssl don't affect core API (VIPs/listeners/pools) other than adding some attributes to listeners.</div>
<div><br></div><div>3. L7 is also a separate work, it will not be accounted in 'API improvement' blueprint. You can sync with Samuel for this as we already have pretty detailed blueprints on that.</div><div><br></div>
<div>4. Attribute differences in REST resources.</div><div>This falls into two categories: </div><div>- existing attributes that should belong to one or another resource, </div><div>- attributes that should be added (e.g. they didn't exist in current API)</div>
<div>The first class is better to be addressed in the blueprint review. The second class could be a small action items/blueprints or even bugs. </div><div><font face="arial, helvetica, sans-serif">Example:</font></div><div>
<font face="arial, helvetica, sans-serif"> 1) <span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">"custom_503" - that attribute deserves it's own miniblueprint, I'd keep it out of scope of 'API improvement' work.</span></font></div>
<div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap"> 2) ipv6_subnet_id/addressv6 - that IMO also deserves it's own miniblueprint (whole thing about ipv6 support)</span></font></div>
<div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap"><br></span></font></div><div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">So, I'd like to make the following action items out of the document:</span></font></div>
<div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap"><br></span></font></div><div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">1. Extract 'core API' - VIPs/Listeners/Pools/Members/Healthmonitors.</span></font></div>
<div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">This action item is actually the blueprint that I've filed and that's what I'm going to implement</span></font></div>
<div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap"><br></span></font></div><div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">2. Work on defining single-call API that goes along with single object core API (above)</span></font></div>
<div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">Your document already does a very good job on this front.</span></font></div><div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap"><br>
</span></font></div><div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">3. Extract 'Loadbalancer' portion of API into additional Doc/blueprint. I deserves it's own discussion and use cases.</span></font></div>
<div><font face="arial, helvetica, sans-serif"><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap">I think separating it will </span></font><span style="color:rgb(0,0,0);font-family:arial,helvetica,sans-serif;white-space:pre-wrap">also </span><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap;font-family:arial,helvetica,sans-serif">help to reduce discussion contention.</span></div>
<div><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap;font-family:arial,helvetica,sans-serif"><br></span></div><div><span style="background-color:transparent;color:rgb(0,0,0);white-space:pre-wrap;font-family:arial,helvetica,sans-serif">4. Work with </span><a href="https://review.openstack.org/#/c/89903/">https://review.openstack.org/#/c/89903/</a> to define proper attribute placement of existing attributes</div>
<div><br></div><div>5. Define a set of attributes that are missing in proposed API and make a list of work items based on that.</div><div>(I assume that there also could be some, that may make sense to include in proposal)</div>
<div><br></div><div>I think following this list will actually help us to make iterative progress and also to work on items in parallel.</div><div><br></div><div>Thanks again for the great document!</div><div><br></div><div>
Eugene.</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 24, 2014 at 4:07 AM, Stephen Balukoff <span dir="ltr"><<a href="mailto:sbalukoff@bluebox.net" target="_blank">sbalukoff@bluebox.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Brandon!<div><br></div><div>Thanks for the questions. Responses inline:</div><div class="gmail_extra">
<br><br><div class="gmail_quote"><div class="">On Wed, Apr 23, 2014 at 2:51 PM, Brandon Logan <span dir="ltr"><<a href="mailto:brandon.logan@rackspace.com" target="_blank">brandon.logan@rackspace.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hey Stephen!<br>
Thanks for the proposal and spending time on it (I know it is a
large time investment). This is actually very similar in structure
to something I had started on except a load balancer object was the
root and it had a one-to-many relationship to VIPs and each VIP had
a one-to-many relationship to listeners. We decided to scrap that
because it became a bit complicated and the concept of sharing VIPs
across load balancers (single port and protocol this time),
accomplished the same thing but with a more streamlined API. The
multiple VIPs having multiple listeners was the main complexity and
your proposal does not have that either.<br>
<br>
Anyway, some comments and questions on your proposal are listed
below. Most are minor quibbles, questions and suggestions that can
probably be fleshed out later when we decide on one proposal and I
am going to use your object names as terminology.<br>
<br>
1. If a VIP can have IPv4 and IPv6 IPs at the same time is that
really a single VIP? Why not call that a load balancer? I'm always
going to advocate for calling the root object a load balancer, and I
think even in this proposal calling the VIP a load balancer makes
sense. Renaming your model's load balancer to something else should
be trivial. <br></div></blockquote><div><br></div></div><div>A couple things about this: The more I think about it, the more I like the idea of not using the term 'loadbalancer' for any of the primitives in the model. In my original version of this we had called the thing presently known as a load balancer a "cluster" but then that's also a pretty over-used term. The problem with the term "loadbalancer" is that people seem to jump to conclusions too readily as to what it means (which slows down discussions all over the place-- both talking to users and talking to Neutron core developers).I was *really* tempted to call it what we've started to call it internally ("Efkalb" reminiscent of the "squonky" discussion we had earlier)... but then I figured it probably made more sense to stick with the glossary this group has already agreed upon for now. I'm still for changing the name and making the policy decision that no single primitive will be called a "loadbalancer." My thought is that this whole project we're working on is "the load balancer," but when we're talking about the nitty-gritty details, we should use more specific terms. :/</div>
<div><br></div><div>Having said this, I think VIP still makes good sense for the name of the object that is the root of the user API. One of the key characteristics of a "virtual IP" as the world knows it is that it can "float" from machine to machine as necessary. The fact that this object can have both an IPv4 and an IPv6 address just means these addresses should "float" together.</div>
<div><br></div><div>We had considered creating an object that just has an 'ip_address' field and then have an 'address_type' field or something to distinguish it between IPv4 and IPv6. But we decided to go with what's there because of practical considerations:</div>
<div><br></div><div>The *only* case we've seen in use by our customers where they want services exposed on both IPv4 and IPv6 is where they're delivering the exact same functionality or content, just over the different IP protocol. This means that in a load balancer configuration, the listeners, pools, rules, SSL certificates, etc. are all the same for both the IPv4 and IPv6 service. To allow for an IPv4 and IPv6 VIP to share all the same child primitives, we'd have to make some kind of other primitive which acts as a join between VIPs and Listeners (as you alluded to above was in your original model), which seemed like a lot more complication than was necessary, considering the only case where we'd likely see it in use.</div>
<div><br></div><div>Also, in a lot of actual implementations, it's also more efficient to have a single process listen on multiple IPs if they're going to have the same back-ends, rules, SSL certs, etc.</div><div>
<br></div><div>So we cheated a bit, and said a 'VIP' can have both a single IPv4 and a single IPv6 address.</div><div><br></div><div>It's also worth saying that in designing this object model, we took pains to try to minimize the explosion of too many primitive objects because this leads to a lot of complication that most people never need anyway. And, it's also hard to get this group to agree to drastic changes because backward compatibility, and the need to support the models we do agree on for a very long time factor into the cost of maintaining things.</div>
<div class="">
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
2. How would a user be able to add another IPv4 or IPv6 IP to the
same VIP?<br></div></blockquote><div><br></div></div><div>A single VIP can have only a single IPv4 address and a single IPv6 address. If you need another IPv4 address, create another VIP. :)</div><div class=""><div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
3. Pool does not have a subnet attribute, how do you define what
subnet the pool members should be on?<br></div></blockquote><div><br></div></div><div>So, this is one where I'm a bit torn: It's clear from some of the re-encryption discussion that sometimes members of the same pool will be in different subnets. (The example given was that some members are local, and some are hosted with an entirely different provider.) Given this, it doesn't make sense to have the assumption that all members of the same pool will be in the same subnet.</div>
<div><br></div><div>What's really important, in any case, is that the appliance or service which actually does the load balancing is able to communicate directly or indirectly with each of the members. We solved this at our company by making them talk IPv6 to each member when the appliance wasn't hosted on the same back-end network as the members. Since IPv6 addresses are globally routable, this becomes a cinch to manage. However, in the world of Neutron networking, where tenants can potentially create an arbitrary number of virtual networks and have them have overlapping rfc1918 IP space... things get a lot more complicated.</div>
<div><br></div><div>So maybe it makes sense to have a 'neutron_subnet' as an optional attribute to the member objects? (Again, it only becomes important if the device(s) doing the load balancing need to have something like a neutron network port connected to the subnets the members are using.)<br>
</div><div><br></div><div><div>Keep in mind, also, that I don't know of any load balancing solution at present that's prepared to deal with overlapping IP space in a single instance like this either. (For example, if I have member A on neutron_subnet B with IP address 10.0.0.5 in the pool, and member X on neutron_subnet Y with IP address 10.0.0.5 in the same pool... is there any solution available today that can actually deal with this?)</div>
</div><div><br></div><div>Again, I'm still a bit torn on this and would love to hear others' ideas... Everything I've come up with so far looks like it's so dependent on the implementation details for the cloud operator that it's hard to know what's appropriate to make part of the model.</div>
<div class="">
<div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
4. In the single create call, how would a user reuse an object that
is defined inside that request body since they will not have an
actual id.<br></div></blockquote><div><br></div></div><div>I understand there are actually ways to do specify this in JSON, but I also understand these are nasty, difficult to understand, and easy to screw up. If you know you're going to want to re-use a specific primitive a whole bunch, I would recommend creating that primitive as a separate operation prior to the "single-call," then just referencing its ID in your single call.</div>
<div><br></div><div>Yes, technically this isn't single-call anymore, but it's probably easier than the alternative.</div><div><br></div><div>Alternatively, you can just repeat the same object over and over again (ie. don't reuse). The LBaaS code shouldn't care if two primitives are identical in every way except ID.</div>
<div class="">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
5. I would like to see expanded details of child objects when
getting the details of an object (i.e. GET /pools shows details of a
health monitor)<br></div></blockquote><div><br></div></div><div>We considered this as well-- and it would be pretty easy to add. Didn't do it in the proposal just because we wanted to get the proposal out the door. :) But again, this sort of thing is pretty trivial to do.</div>
<div class="">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
6. Why is there a protocol on the pool object and the listener
object? Is this for translating from secure protocols to insecure
protocols (i.e. HTTPS to HTTP).<br></div></blockquote><div><br></div></div><div>Yep. It's important that the protocol of the listener be compatible with the protocol of the pool (and the protocol of the health check). It doesn't make sense to have an HTTPS listener that points to an IMAP pool. (Not that we're proposing supporting IMAP at this time... but... yeah.)</div>
<div><br></div><div>Note that I didn't say they had to be equivalent, just compatible. As you point out, there are valid use cases for having an HTTPS listener and an HTTP pool.</div><div class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
7. When returning lists of objects (i.e. GET /vips, GET /pools) I'd
like to see the name returned as well.<br></div></blockquote><div><br></div></div><div>Yep, makes sense. See answer to #5 above. But I agree that this probably makes sense.</div><div class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
8. Can all primitives be shared among other parent objects belonging
to the same tenant?<br></div></blockquote><div><br></div></div><div>No. We didn't see great enough need to allow members to be shared among pools, nor L7Rules among L7Policies. (In fact, I was really tempted to roll all the attributes of the Health monitor into the Pool primitive since I don't see much use in sharing health monitors either.) If we allowed this kind of sharing, we'd need to add more "join" primitives to allow for the n:m relationship between pools and members, for example.</div>
<div><br></div><div>As a rule of thumb that I try to follow: If you have a 'join' primitive that has no attributes other than the IDs of the primitives you're joining... take a closer look at your object model. There's probably a way to do it without the join. :)</div>
<div class="">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
9. can pool members be shared between pools on the same tenant? <br>
-if so, what happens if two pools are sharing the same pool
member, one pool has a health monitor, the other does not. The pool
member's status will get updated to "DOWN" for both pools.<br>
-if not, why not just make them children resources of /pools
(i.e. /pools/{pool_id}/members).<br></div></blockquote><div><br></div></div><div>The answer is: No. And you're right! We could easily make members a pure leaf primitive connected to the pool primitive. This would mean that when the pool is destroyed, the members are implicitly destroyed. The same is true for L7Rules and L7Policies.</div>
<div><br></div><div>Can anyone think of an instance where a user might want to destroy a pool, but leave its member primitives intact (presumably so they could be attached to another pool at a later time)?</div><div><br>
</div>
<div>Absent any counterexamples, I'm all for making Members a leaf primitive of Pools, and L7Rules a leaf primitive of L7Policies.</div><div class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<br>
Again, thanks for spending the time on this. It has a lot of good
ideas and things we did not think about. We've been requested to do
a POC of our proposal, will you and your team be able to do the
same?<br></div></blockquote><div><br></div></div><div>No problem, eh! I'm glad you've apparently found it useful. :)</div><div><br></div><div>And... I guess it depends on what form the POC has to take. Blue Box is a much, much smaller organization than Rackspace, so we don't have nearly the amount of man-power for pursuing POCs and whatnot as I would like. (It's a normal course of events for prototypes we build to end up being put into production for a given customer shortly thereafter. We do a lot of custom things for our customers, and occasionally we get to build something more general purpose, like the load balancer software appliances we made...)</div>
<div><br></div><div>Assuming we're still making progress in the discussion on things like this API revision, or how exactly we're going to solve the operator concerns around HA functionality... I'm hoping to spend more time porting our software appliance to a form that can be deployed on-demand for OpenStack. (We're open-sourcing the software appliance for this purpose-- so far, though, I've only had time to work on refining its API documentation... which is of course not the same one we've been discussing here. XD)</div>
<div><br></div><div>What did you have in mind, as far as a POC is concerned?</div><div><br></div><div>Thanks,</div><div>Stephen</div><div><br></div></div><div class=""><div><br></div>-- <br><span></span>Stephen Balukoff
<br>Blue Box Group, LLC
<br>(800)613-4305 x807
</div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>