Open Stack

Hi Phillip,


Adding my thoughts below. I’ll first answer the questions you raised with
what I think should be done, and then give my explanations to reason
through with those views.



1. Do we want to add logic in the plugin to call the FLIP association API?


 >> We should implement the logic in the new v2 extension and the plugin
layer as a single API call. We would need to add to the existing v2 API to
be able to do this. The best place to add this option of passing the FLIP
info/request to the VIP is in the VIP create and update API calls via new
parameters.


2. If we have logic in the plugin should we have configuration that
identifies whether to use/return the FLIP instead of the port VIP?


 >> Yes and no, in that we should return the complete result of the VIP
create/update/list/show API calls, in which we show the VIP internal IP,
but we also show the FLIP either as empty or having a FLIP uuid. External
users will anyway use only the FLIP, else they wouldn’t be able to reach
the LB and the VIP IP, but the APIs need to show both fields.


3. Would we rather have logic for FLIP association in the drivers?


 >> This is the hardest part to decide. To understand this, we need to look
at two important drivers of LBaaS design:


 I)  The Neutron core plugin we’re using.

II) The different types of LB devices - physical, virtual standalone, and
virtual controlled by a management plane. This leads to different kinds of
LBaaS drivers and different kinds of interaction or the lack of it between
them and the core neutron plugin.


The reason we need to take into account both these is that port
provisioning as well as NATing for the FLIP to internal VIP IP will be
configured differently by the different network management/backend planes
that the plugins use, and the way drivers configure LBs can be highly
impacted by this.


For example, we can have an NSX infrastructure that will implement the FLIP
to internal IP conversion in the logical router module which sits pretty
much outside of Openstack’s realm, using openflow. Or we can use lighter
solutions directly on the hypervisor that still employ open flow entries
without actually having a dedicated logical routing module. Neither will
matter much if we are in a position to have them deploy our networking for
us, i.e., in the cases of us using virtual LBs sitting on compute nodes.
But if we have a physical LB, the neutron plugins cannot do much of the
network provisioning work for us, typically because physical LBs usually
sit outside of the cloud, and are among the earliest points of contact from
the external world.


This already nudges us to consider putting the FLIP provisioning
functionality in the driver. However, consider again more closely the major
ways in which LBaaS drivers talk to LB solutions today depending on II) :


 a) LBaaS drivers that talk to a virtual LB device on a compute node,
directly.

b) LBaaS drivers that talk to a physical LB device (or a virtual LB sitting
outside the cloud) directly.

c) LBaaS drivers that talk to a management plane like F5’s BigIQ, or
Netscaler’s NCC, or as in our case, Octavia, that try to provide tenant
based provisioning of virtual LBs.

d) The HAProxy reference namespace driver.


d) is really a PoC use case, and we can forget it. Let’s consider a), b)
and c).


If we use a) or b), we must assume that the required routing for the
virtual LB has been setup correctly, either already through nova or
manually. So we can afford to do our FLIP plumbing in the neutron plugin
layer, but, driven by the driver - how? - typically, after the VIP is
successfully created on the LB, and just before the driver updates the
VIP’s status as ACTIVE, it can create the FLIP. Of course, if the FLIP
provisioning fails for any reason, the VIP still stands. It’ll be empty in
the result, and the API will error out saying “VIP created but FLIP
creation failed”. It must be manually deleted by another delete VIP call.
We can’t afford to provision a FLIP before a VIP is active, for external
traffic shouldn’t be taken while the VIP isn’t up yet. If the lines are
getting hazy right now because of this callback model, let’s just focus on
the point that we’re initiating FLIP creation in the driver layer while the
code sits in the plugin layer because it will need to update the database.
But in absolute terms, we’re doing it in the driver.


It is use case c) that is interesting. In this case, we should do all
neutron based provisioning neither in the driver, nor in the plugin in
neutron, rather, we should do this in Octavia, and in the Octavia
controller to be specific. This is very important to note, because if
customers are using this deployment (which today has the potential to be
way greater in the near future than any other model simply because of the
sheer existing customer base), we can’t be creating the FLIP in the plugin
layer and have the controller reattempt it. Indeed, the controllers can
change their code to not attempt this, but like just noted, it isn’t
logically orderly to create the FLIP before creating the VIP successfully.
So let’s leave this op to the driver.


I am guilty of not following the proceedings on the Octavia controller
design due to internal work deadlines, but if I remember right, the design
specified a controller being built to spawn and configure virtual HAProxy
LB instances. If Octavia is not taking that route today, I think that the
best place to do this provisioning would be as described above for a) and
b).



4) If logic is in the drivers would we still return the port VIP to the
user then later overwrite it with the FLIP?


 >> As described in 2), we always return the VIP port along with the FLIP
info in API calls. Users like PaaS that sit on top of IaaS (Openstack) will
only use the FLIP when they commission endpoints for their users, who will
only be concerned with the FLIP.


5) Or would we have configuration to not return the port VIP initially, but
an additional query would show the associated FLIP.


 >> Not inclined to doing this :) It’s just another field associated with
the VIP - an API call for it would be overkill.


6) Is there an internal service call for this, and if so would we use it
instead of API calls?


 >> Really depends on the core plugin you’re using - each plugin does this
differently, but all you need to do is to simply call
core_plugin.create_floatingip() and it should do everything for you - it
should correctly create the FLIP entry in the db, then go to its backend (a
controller or ovs or the L3 agent etc) which will correctly provision the
necessary interfaces and/or program the required flow rules or iptables
rules or equivalent and return success or failure.



All said and done, the fact is that in LBaaS, as with many other
components, we can always override the default plugin and driver layers and
write our own and do whatever we want in them :) This allows vendors to
rewrite implementations if they need to. Hopefully we’ll get the design
well enough that they don’t have to do that in more than a few modules.


Hope this helps! Please do let me know if you have any questions or need
more pointers with the extensions/neutron/etc.



Cheers!

Regards,

Vijay




On Mon, Oct 13, 2014 at 1:18 PM, Phillip Toohill <
phillip.toohill at rackspace.com> wrote:

> Diagrams in jpeg format..
>
> On 10/12/14 10:06 PM, "Phillip Toohill" <phillip.toohill at RACKSPACE.COM>
> wrote:
>
> >Hello all,
> >
> >Heres some additional diagrams and docs. Not incredibly detailed, but
> >should get the point across.
> >
> >Feel free to edit if needed.
> >
> >Once we come to some kind of agreement and understanding I can rewrite
> >these more to be thorough and get them in a more official place. Also, I
> >understand theres other use cases not shown in the initial docs, so this
> >is a good time to collaborate to make this more thought out.
> >
> >Please feel free to ping me with any questions,
> >
> >Thank you
> >
> >
> >Google DOCS link for FLIP folder:
> >
> https://drive.google.com/folderview?id=0B2r4apUP7uPwU1FWUjJBN0NMbWM&usp=sh
> >a
> >ring
> >
> >-diagrams are draw.io based and can be opened from within Drive by
> >selecting the appropriate application.
> >
> >On 10/7/14 2:25 PM, "Brandon Logan" <brandon.logan at RACKSPACE.COM> wrote:
> >
> >>I'll add some more info to this as well:
> >>
> >>Neutron LBaaS creates the neutron port for the VIP in the plugin layer
> >>before drivers ever have any control.  In the case of an async driver,
> >>it will then call the driver's create method, and then return to the
> >>user the vip info.  This means the user will know the VIP before the
> >>driver even finishes creating the load balancer.
> >>
> >>So if Octavia is just going to create a floating IP and then associate
> >>that floating IP to the neutron port, there is the problem of the user
> >>not ever seeing the correct VIP (which would be the floating iP).
> >>
> >>So really, we need to have a very detailed discussion on what the
> >>options are for us to get this to work for those of us intending to use
> >>floating ips as VIPs while also working for those only requiring a
> >>neutron port.  I'm pretty sure this will require changing the way V2
> >>behaves, but there's more discussion points needed on that.  Luckily, V2
> >>is in a feature branch and not merged into Neutron master, so we can
> >>change it pretty easily.  Phil and I will bring this up in the meeting
> >>tomorrow, which may lead to a meeting topic in the neutron lbaas
> >>meeting.
> >>
> >>Thanks,
> >>Brandon
> >>
> >>
> >>On Mon, 2014-10-06 at 17:40 +0000, Phillip Toohill wrote:
> >>> Hello All,
> >>>
> >>> I wanted to start a discussion on floating IP management and ultimately
> >>> decide how the LBaaS group wants to handle the association.
> >>>
> >>> There is a need to utilize floating IPs(FLIP) and its API calls to
> >>> associate a FLIP to the neutron port that we currently spin up.
> >>>
> >>> See DOCS here:
> >>>
> >>> >
> >>>
> http://docs.openstack.org/api/openstack-network/2.0/content/floatingip_c
> >>>r
> >>>eate.html
> >>>
> >>> Currently, LBaaS will make internal service calls (clean interface :/)
> >>>to create and attach a Neutron port.
> >>> The VIP from this port is added to the Loadbalancer object of the Load
> >>>balancer configuration and returned to the user.
> >>>
> >>> This creates a bit of a problem if we want to associate a FLIP with the
> >>>port and display the FLIP to the user instead of
> >>> the ports VIP because the port is currently created and attached in the
> >>>plugin and there is no code anywhere to handle the FLIP
> >>> association.
> >>>
> >>> To keep this short and to the point:
> >>>
> >>> We need to discuss where and how we want to handle this association. I
> >>>have a few questions to start it off.
> >>>
> >>> Do we want to add logic in the plugin to call the FLIP association API?
> >>>
> >>> If we have logic in the plugin should we have configuration that
> >>>identifies weather to use/return the FLIP instead the port VIP?
> >>>
> >>> Would we rather have logic for FLIP association in the drivers?
> >>>
> >>> If logic is in the drivers would we still return the port VIP to the
> >>>user then later overwrite it with the FLIP?
> >>> Or would we have configuration to not return the port VIP initially,
> >>>but an additional query would show the associated FLIP.
> >>>
> >>>
> >>> Is there an internal service call for this, and if so would we use it
> >>>instead of API calls?
> >>>
> >>>
> >>> Theres plenty of other thoughts and questions to be asked and discussed
> >>>in regards to FLIP handling,
> >>> hopefully this will get us going. I'm certain I may not be completely
> >>>understanding this and
> >>> is the hopes of this email to clarify any uncertainties.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev at lists.openstack.org
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>_______________________________________________
> >>OpenStack-dev mailing list
> >>OpenStack-dev at lists.openstack.org
> >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >_______________________________________________
> >OpenStack-dev mailing list
> >OpenStack-dev at lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141014/033a1708/attachment-0001.html>