Open Stack

Adding openstack-dev mailer as this discussion is gathering good momentum.

This thread was started in the context of the blueprint (
https://blueprints.launchpad.net/neutron/+spec/adv-services-in-vms) and the
initial face-to-face discussions we had with a number of people on this
thread.


Thanks,
~Sumit.



> On Sat, Sep 28, 2013 at 10:41 AM, P Balaji-B37839 <B37839 at freescale.com>wrote:
>
>>  Thanks Greg for pulling into this thread.****
>>
>> ** **
>>
>> Good that we are seeing converging thoughts and discussions.****
>>
>> ** **
>>
>> Please find inline comments.****
>>
>> ** **
>>
>> Regards,****
>>
>> Balaji.P****
>>
>> ** **
>>
>> *From:* Ravi Chunduru [mailto:ravivsn at gmail.com]
>> *Sent:* Saturday, September 28, 2013 12:43 AM
>> *To:* Bob Melander (bmelande)
>> *Cc:* Geoff Arnold; Regnier, Greg J; Cummings, Gregory D; David Chang
>> (dwchang); Elzur, Uri; Gary Duan; Joseph Swaminathan; Kanzhe Jiang;
>> Kuang-Ching Wang; Kyle Mestery (kmestery); Maciocco, Christian; Marc
>> Benoit; P Balaji-B37839; Rajesh Mohan; Rudrajit Tapadar (rtapadar); Sridar
>> Kandaswamy (skandasw); Sumit Naiksatam; Yi Sun
>> *Subject:* Re: Service VM discussion - mgmt ifs****
>>
>> ** **
>>
>> I am glad we are having a good discussion here. Please see my comments in
>> line.****
>>
>> ** **
>>
>> Thanks,****
>>
>> -Ravi.****
>>
>> ** **
>>
>> On Fri, Sep 27, 2013 at 11:11 AM, Bob Melander (bmelande) <
>> bmelande at cisco.com> wrote:****
>>
>> Good points Ravi. Some comments from me inline.
>>
>> Thanks,****
>>
>> Bob****
>>
>>
>> 27 sep 2013 kl. 19:31 skrev "Ravi Chunduru" <ravivsn at gmail.com>:****
>>
>>  Hope I am not too late on this thread. I have definitely missed earlier
>> discussions. So some of my thoughts would have been already discussed. **
>> **
>>
>> ** **
>>
>>  I want to write down my thoughts and be on same page with you all. ****
>>
>> ** **
>>
>> 1) Service VM management Network:  ****
>>
>> Management interface for the service VM must be always reachable. It
>> means even if the virtual networks are blogged, this interface must be
>> reachable to plugin. ****
>>
>> ** **
>>
>> It means if the management interface should be from management network.
>> Does it mean we should have a bridge network type for management network?
>> ****
>>
>> I suggest going with approach like in Vmware solution. The hypervisor
>> opens up serial port with each guest appliance. Host sees it as a unix
>> socket and guest sees it as a serial port.  This way reaching management
>> network is reliable and not collide with data traffic. Thoughts?****
>>
>>  ** **
>>
>> I think this a nice feature to provide. I think the framework should
>> provide both options, serial port and network interface for management
>> since it cannot be guaranteed all appliances will support serial port
>> access.****
>>
>> ** **
>>
>> Ravi> I completely agree with this. ****
>>
>> Management network interface can be used provided there is no service VM
>> migration. ****
>>
>> *[P Balaji-B37839] We used Virtio-Serial port on KVM hypervisor for
>> propagation of configuration from OS-Controller to Compute Node in our
>> Network Service Chain PoC implementation. Discussing with Daniel [Nova –
>> Libvirt Lead] to make it more generic and secure. As Virtio-Serial port
>> will give access to Host from Guest VM, We should think of Host security.
>> Right now Apparmor for Libvirt is taking care of it. We are working on BP
>> for this.*****
>>
>>
>>
>> ****
>>
>> 2) Data interfaces  ****
>>
>> It is always possible for user to add/delete networks from the services.
>> For example, user can decide to have fwaas for a given virtual network or
>> delete.  ****
>>
>> The problem becomes complex when a service VM needs to listen on IP from
>> a subnet. Say any L3 services like SSL, Load balancer needs to listen on IP
>> address. These IPs can change and sometimes comes from various networks.*
>> ***
>>
>> ** **
>>
>> Say a loadbalancer has a VIP coming from Network 1, and deleted a VIP
>> from Network 2. It means we need to hotplug a port from Network 1 and
>> delete a port from Network 2 on the service VM. ****
>>
>> ** **
>>
>> If there are 100 networks, then it needs 100 ports? I am fine with it but
>> Should we not think about a single port with IPs coming from all the
>> configured networks?****
>>
>> ** **
>>
>> In service VM work we have been doing we have been making use of VLAN
>> trunking which is one way of dealing with this. The service VM has a single
>> interface (or a small number of them) for data traffic. When attached to a
>> particular tenant network, that network is trunked on the port used by the
>> service VM. Benefits of this is this is that no vif hot plugging is needed
>> and few VIFs are needed (I think PCI will limit the number VIFs possible to
>> something much less than 100) Downside is that the plugin needs to support
>> VIF trunking, which afaik only the Cisco N1kv plugin supports. There is a
>> blueprint by Kyle Mestery to add VLAN trunking to more plugins. ML2 would
>> be a good next candidate. I think the framework should aim to support a few
>> different "attachment" methods, hot-plugging being one.****
>>
>>   ****
>>
>> Ravi> That is a good solution. ****
>>
>> Service VM plugin analyzes the config and can make use of this to add or
>> remove network from multi segment domain.****
>>
>>  ** **
>>
>>   ** **
>>
>> 3) Multi tenancy support****
>>
>> If a service VM is by provider, it definitely should have multi tenancy
>> support. Its quite possible for user to go with shared service VM for cost
>> cut downs. In such cases the how does service VM should address overlapped
>> networks. Unless service VMs implement tenant isolation in its
>> implementation its a limitation.****
>>
>>  ** **
>>
>> Agree completely about the benefit of multi-tenancy. Likely this will
>> involve creating something like virtual contexts inside the service VM and
>> that will be appliance dependent. Therefore it is less clear to me how much
>> of this can be provided by the framework.****
>>
>>  Ravi> We may need to provide reference implementation. Hence asked.****
>>
>> *[P Balaji-B37839]  Tenants will be having multiple configurations based
>> on their deployment. For a Service VM shared by provider applying tenant
>> specific configuration by creating virtual context is good.*
>>
>> *Frame work should address the Sharing of Service VM by Provider to
>> multiple tenants and tenant specific configuration mapping.*
>>
>>  ****
>>
>>
>>
>> ****
>>
>> ** **
>>
>> 4) Service VM functionality****
>>
>> Seeing big picture, a service VM will be replaced with Vendor appliances
>> replacing the reference implementations say HAproxy or iptables firewall
>> etc.,  Following this Blueprint , obvious next target is service
>> insertion/chaining etc., Going by this direction - we should consider a
>> service VM as container of multiple services. ****
>>
>> - Service VM must define set of services it can service****
>>
>> - Service VM must have set of capabilities per service for Service
>> Scheduler to take an effective decision****
>>
>> - Future design of service insertion must be aware of the fact that a
>> service VM has multiple services and utilize one or more services with out
>> leaving the VM( greatly helps in improving latency)****
>>
>> With this in consideration I wanted to name it as appliance and each
>> service as  a network function. But I understand it draws up big
>> discussions :)****
>>
>> ** **
>>
>> The way I interpret your text above we have a similar abstraction but in
>> our case we call it a "hosting entity". A hosting entity hosts service
>> instances (in our case we only have one type of service so far - routing)
>> that materializes the logical resources of the service (routers in our
>> case).****
>>
>> ** **
>>
>> We do scheduling of logical service resources across hosting entities and
>> if need be spin up new VMs.****
>>
>>  Ravi> Right, we need a good scheduling logic that takes on attributes
>> what can a service VM support and decide on selection.****
>>
>> *[P Balaji-B37839] Correct Ravi, I think this frame work should address
>> generic scheduling so that the appliance vendors can plug-in their Service
>> VM Attributes.*****
>>
>> ** **
>>
>>
>>
>> ****
>>
>> ** **
>>
>> 5) High availability****
>>
>> In any realistic deployments no user would want a bottleneck for a
>> service with only one service VM.
>> We require redundancy in the solution for real deployment to happen.****
>>
>> ** **
>>
>> When it comes to HA and bottlenecks I would also like to being into the
>> picture the task of making configurations inside the service VMs. This can
>> be done using a variety of protocols/mechanisms and can be quite heavy in
>> the sense of requiring many roundtrips. We've found it to be useful if that
>> can be offloaded and distributed. In our case we have special configuration
>> agents that can run in arbitrary numbers and which take on configuration
>> tasks. The agent is very similar to the l3agent with one difference being
>> that it applies configurations on remote appliances. Personally I would
>> like if the l3agent would be evolved to do this****
>>
>>  ** **
>>
>>     ** **
>>
>> 6) Service VM integration with APIs****
>>
>> I have some thoughts on it. First I would like to know thoughts from you
>> all on the above points.****
>>
>> ** **
>>
>> Thanks,****
>>
>> -Ravi.****
>>
>> ** **
>>
>> (For those whom I am interacting first... a bit Intro: I am Ravi
>> Chunduru, I worked on firewalls, Alteon Load balancers development and
>> currently with PayPal working in Cloud Engineering team)****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> On Thu, Sep 26, 2013 at 5:56 PM, Geoff Arnold <arnoldg at brocade.com>
>> wrote:****
>>
>> Good discussion. ****
>>
>> ** **
>>
>> When I drafted the DNRM blueprint, I made the pessimistic assumption that
>> the variety of VIF plugging patterns, and the number of ways in which
>> services instances might need to adapt to different network environments,
>> were going to be so great that it would be premature to try to capture VIF
>> plugging into a single scheme driven by some kind of service metadata. I
>> based this on conversations with developers of various virtual appliances
>> in the L3-L7 space, and the kinds of adaptations they had been forced to
>> make to get their products working with different IaaS providers. So I
>> decided to package a service as (effectively) a triple:****
>>
>>    - a Glance image for the service****
>>    - a provisioning policy (including Nova parameterization and
>>    lifecycle)****
>>    - a plugin that handled the initialization of a newly-activated
>>    service instance****
>>
>>  ** **
>>
>> If a single metadata-driven scheme for VIF plugging can be made to work,
>> that would be great. (And DNRM could accommodate it using a generic
>> plugin.) I am skeptical, however.****
>>
>> ** **
>>
>> The comment about extending “beyond VMs and also support physical devices”
>> is exactly right, of course. One of the DNRM use cases is policy-based
>> selection of virtual or physical LB instances. (We have a customer waiting…)
>> ****
>>
>> ** **
>>
>> Geoff ****
>>
>> ** **
>>
>> *From: *<Regnier>, "greg.j.regnier at intel.com" <greg.j.regnier at intel.com>
>> *Date: *Thursday, September 26, 2013 at 5:19 PM
>> *To: *"Bob Melander (bmelande)" <bmelande at cisco.com>
>> *Cc: *"Bob Melander (bmelande" <bmelande at cisco.com>, "Cummings, Gregory
>> D" <gregory.d.cummings at intel.com>, "David Chang (dwchang" <
>> dwchang at cisco.com>, "Elzur, Uri" <uri.elzur at intel.com>, "
>> gduan at varmour.com" <gduan at varmour.com>, Joseph Swaminathan <
>> joeswaminathan at gmail.com>, Kanzhe Jiang <kanzhe.jiang at bigswitch.com>,
>> Kuang-Ching Wang <kc.wang at bigswitch.com>, "kmestery at cisco.com" <
>> kmestery at cisco.com>, "Maciocco, Christian" <christian.maciocco at intel.com>,
>> Marc Benoit <mbenoit at paloaltonetworks.com>, P Balaji <
>> B37839 at freescale.com>, Rajesh Mohan <rajesh.mlists at gmail.com>, "
>> ravivsn at gmail.com" <ravivsn at gmail.com>, "Rudrajit Tapadar (rtapadar" <
>> rtapadar at cisco.com>, Geoff Arnold <arnoldg at brocade.com>, "Sridar
>> Kandaswamy (skandasw" <skandasw at cisco.com>, "sumitnaiksatam at gmail.com" <
>> sumitnaiksatam at gmail.com>, Yi Sun <yisun at varmour.com>
>> *Subject: *RE: Service VM discussion - mgmt ifs****
>>
>> ** **
>>
>> + GregC, RaviC, Balaji.P, GeoffA****
>>
>> - DanF****
>>
>>  ****
>>
>> Thanks Bob, good input,****
>>
>>  ****
>>
>> Given that a service VM can host multiple logical services, I agree that
>> the scheduling function you describe will be required.****
>>
>> Also mapping of the logical service interfaces to VIFs…****
>>
>>  ****
>>
>> As Yi pointed out, another aspect is to define the model for how to plug
>> VIFs into the network(s) to build the desired logical topology.****
>>
>> It will be good to work up some use cases, e.g. different insertion
>> modes, single network, spanning networks,…****
>>
>>  ****
>>
>> Regards,****
>>
>>                 Greg****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Bob Melander (bmelande) [mailto:bmelande at cisco.com<bmelande at cisco.com>]
>>
>> *Sent:* Wednesday, September 25, 2013 2:57 PM
>> *To:* Regnier, Greg J; Sumit Naiksatam; Rudrajit Tapadar (rtapadar);
>> David Chang (dwchang); Joseph Swaminathan; Elzur, Uri; Marc Benoit; Sridar
>> Kandaswamy (skandasw); Dan Florea (dflorea); Kanzhe Jiang; Kuang-Ching
>> Wang; Gary Duan; Yi Sun; Rajesh Mohan; Maciocco, Christian; Kyle Mestery
>> (kmestery)
>> *Subject:* Re: Service VM discussion - mgmt ifs****
>>
>>  ****
>>
>> Hi all,****
>>
>>  ****
>>
>> Just so first comments from me.****
>>
>>  ****
>>
>> A service typically defines logical resources. Such a resource will be
>> attached to ****
>>
>> networks through ports. During the lifetime of the resource, it may later
>> be detached ****
>>
>> from some networks. Neutron routers are one concrete example. It is also
>> possible that ****
>>
>> multiple service resources are hosted by the same VM. This is analogous
>> to how multiple ****
>>
>> Neutron routers can be hosted by a network node (to continue the routing
>> service example).****
>>
>>  ****
>>
>> Based on this it will be good if the framework includes its own
>> scheduling component.****
>>
>> Its responsibility is to select an existing VM to host a logical resource
>> or to signal****
>>
>> that a new VM needs to be created (in the case no running VMs are
>> suitable). ****
>>
>> Note that the scheduling component of the service VM framework is
>> different from Nova's ****
>>
>> scheduler. The latter takes care of scheduling of service VMs across
>> compute hosts, ****
>>
>> whereas the former operate on the logical server resources and the
>> entities to host them ****
>>
>> (the VMs) ****
>>
>>  ****
>>
>> The service VM framework scheduler should preferably also allow selection
>> of VIFs to host ****
>>
>> a logical resource's logical interfaces. To clarify the last statement,
>> one "use case" ****
>>
>> could be to spin up a VM with more VIFs than are needed initially (e.g.,
>> if the VM does ****
>>
>> not support vif hot-plugging). Another "use case" is if the plugin
>> supports VLAN trunking ****
>>
>> and attachement of the logical resource's logical interface to a network
>> corresponds to ****
>>
>> trunking of a network on a VIF.****
>>
>>  ****
>>
>> There are at least three (or four) ways to dynamically plug a logical
>> service resource ****
>>
>> inside a VM to networks:****
>>
>> - Create a VM VIF on demand for the logical interface of the service
>> resource ****
>>
>> ("hot-plugging")****
>>
>> - Pre-populate the VM with a set of VIFs that can be allocated to logical
>> interfaces of ****
>>
>> the service resources****
>>
>> - Create a set of VM VIFs (on demand or during VM creation) that carry
>> VLAN trunks for ****
>>
>> which logical (VLAN) interfaces are created and allocated to service
>> resources.****
>>
>>  ****
>>
>> When the service VM framework has its own scheduling component it is
>> fairly easy to extend****
>>
>> it to go beyond VMs and also support physical devices. VMs and physical
>> devices are all****
>>
>> entities that can host logical resources of Neutron services. ****
>>
>>  ****
>>
>> Thanks,****
>>
>> Bob****
>>
>>  ****
>>
>> *From: *<Regnier>, Greg J <greg.j.regnier at intel.com>
>> *Date: *onsdag 25 september 2013 17:09
>> *To: *Sumit Naiksatam <sumitnaiksatam at gmail.com>, "Rudrajit Tapadar
>> (rtapadar)" <rtapadar at cisco.com>, "David Chang (dwchang)" <
>> dwchang at cisco.com>, Joseph Swaminathan <joeswaminathan at gmail.com>,
>> "Elzur, Uri" <uri.elzur at intel.com>, Marc Benoit <
>> mbenoit at paloaltonetworks.com>, "Sridar Kandaswamy (skandasw)" <
>> skandasw at cisco.com>, Dan Florea <dflorea at cisco.com>, Kanzhe Jiang <
>> kanzhe.jiang at bigswitch.com>, Kuang-Ching Wang <kc.wang at bigswitch.com>,
>> Gary Duan <gduan at varmour.com>, Yi Sun <yisun at varmour.com>, Rajesh Mohan <
>> rajesh.mlists at gmail.com>, "Maciocco, Christian" <
>> christian.maciocco at intel.com>, Kyle Mestery <kmestery at cisco.com>, Bob
>> Melander <bmelande at cisco.com>
>> *Cc: *"Regnier, Greg J" <greg.j.regnier at intel.com>
>> *Subject: *Service VM discussion - mgmt ifs****
>>
>>  ****
>>
>> *Hi all,*****
>>
>> * *****
>>
>> *At the face-to-face meeting we had a discussion about the requirements
>> for service VM management.*****
>>
>> *My takeaway was this:*****
>>
>> * *****
>>
>> 1)      *There is a requirement for a management interface for the VM
>> “service VM management port” that was used to manage the VM resource, for
>> example a health monitor in the event a VIF quietly went down.  This
>> management interface could be common across services.*****
>>
>> * *****
>>
>> 2)      *There is also a requirement for a “service management port”
>>  that is used to configure and update the policy/resources of the service,
>> and this interface is service specific.  For example adding a rule to a
>> firewall.*****
>>
>> * *****
>>
>> *Seeking input as to:*****
>>
>> * *****
>>
>> 1)      * Does the above description match the discussion at the meeting?
>> *****
>>
>> 2)      *What are the set of functions needed/desired for the “service
>> VM management” interface?*****
>>
>> 3)      *Other general feedback*****
>>
>> * *****
>>
>> *Thanks,*****
>>
>> *                Greg*****
>>
>> * *****
>>
>> *ps.  Feel free to send your feedback to this list, or comment directly
>> on the blueprint.*****
>>
>>
>>
>> ****
>>
>> ** **
>>
>> --
>> Ravi****
>>
>>
>>
>> ****
>>
>> ** **
>>
>> --
>> Ravi****
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131001/8e4a1491/attachment.html>