Open Stack

Hi Ihar,


wow, good job!!
Sorry for the very slow reply.
I really like your proposal...some comments inline.

On 12/03/2015 04:46 PM, Ihar Hrachyshka wrote:
> Hi,
>
> Small update on the RFE. It was approved for Mitaka, assuming we come up
> with proper details upfront thru neutron-specs process.
>
> In the meantime, we have found more use cases for flow management among
> features in development: QoS DSCP, also the new OF based firewall
> driver. Both authors for those new features independently realized that
> agent does not currently play nice with flows set by external code due
> to its graceful restart behaviour when rules with unknown cookies are
> cleaned up. [The agent uses a random session uuid() to mark rules that
> belong to its current run.]
>
> Before I proceed, full disclosure: I know almost nothing about OpenFlow
> capabilities, so some pieces below may make no sense. I tried to come up
> with high level model first and then try to map it to available OF
> features. Please don’t hesitate to comment, I like to learn new stuff! ;)

I am not an expert either so I encourage people to chime in here.

>
> I am thinking lately on the use cases we collected so far. One common
> need for all features that were seen to be interested in proper
> integration with Open vSwitch agent is to be able to manage feature
> specific flows on br-int and br-tun. There are other things that
> projects may need, like patch ports, though I am still struggling with
> the question of whether it may be postponed or avoided for phase 1.
>
> There are several specific operation 'kinds' that we should cover for
> the RFE:
> - managing flows that modify frames in-place;
> - managing flows that redirect frames.
>
> There are some things that should be considered to make features
> cooperate with the agent and other extensions:
> - feature flows should have proper priorities based on their ‘kind’
> (f.e. in-place modification probably go before redirections);
> - feature flows should survive flow reset that may be triggered by the
> agent;
> - feature flows should survive flow reset without data plane disruption
> (=they should support graceful restart:
> https://review.openstack.org/#/c/182920).
>
> With that in mind, I see the following high level design for the flow
> tables:
>
> - table 0 serves as a dispatcher for specific features;
> - each feature gets one or more tables, one per flow ‘kind’ needed;
> - for each feature table, a new flow entry is added to table 0 that
> would redirect to feature specific table; the rule will be triggered
> only if OF metadata is not updated inside the feature table (see the
> next bullet); the rule will have priority that is defined for the ‘kind’
> of the operation that is implemented by the table it redirects to;
> -  each feature table will have default actions that will 1) mark OF
> metadata for the frame as processed by the feature; 2) redirect back to
> table 0;
> - all feature specific flow rules (except dispatcher rules) belong to
> feature tables;
>
> Now, the workflow for extensions that are interested in setting flows
> would be:
> - on initialize() call, extension defines feature tables it will need;

Do you mean this in a dynamic way or every extension will have tables 
assigned, basically hard-coded? I prefer the second way so we have more 
controls of the tables that are currently used.

> it passes the name of the feature table and the ‘kind’ of the actions it
> will execute; with that, the following is initialized by the agent: 1)

It would be nice to pass also a filter to match some packets. We 
probably don't want to send all the packet to the feature table, the 
extension can define that.

> table 0 dispatcher entry to redirect frames into feature table; the
> entry has the priority according to the ‘kind’ of the table; 2) the

I think we need to define the priority better. According to what you 
wrote we assign priority based on "in-place modification probably go 
before redirections" not sure if it's enough. What happens if we have 
two features that both requires in place-modifications? How do we 
prioritize them? Are we going to allow 2 extension at the same time? Let 
me think more about this...It would be nice to have some real world 
example...

> actual feature table with two default rules (update metadata and push
> back to table 0);
> - whenever extension needs to add a new flow rule, it passes the
> following into the agent: 1) table name; 2) flow specific parameters
> (actions, priority, ...)
>
> Since the agent will manage setting flows for extensions, it will be
> able to use the active agent cookie for all feature flows; next time the
> agent is restarted, it should be able to respin extension flows with no
> data plane disruption. [Note: we should make sure that on agent restart,
> we call to extensions *before* we clean up stale flow rules.]

I like this :)
>
> That design will hopefully allow us to abstract interaction with flows
> from extensions into management code inside the agent. It should
> guarantee extensions cooperate properly assuming they properly define
> their priorities thru ‘kinds’ of tables they have.
>
> It is also assumed that existing flow based features integrated into the
> agent (dvr? anti-spoofing?) will eventually move to the new flow table
> management model.

Good candidates for a POC of this model?

thanks,

Rossella
>
> I understand that the model does not reflect how do feature processing
> for existing OF based features in the agent. It may require some smart
> workarounds to allow non-disruptive migration to new flow table setup.
>
> It would be great to see the design bashed hard before I start to put it
> into spec format. Especially if it’s not sane. :)
>
> Ihar
>
> Mathieu Rohon <mathieu.rohon at gmail.com> wrote:
>
>> Thanks ihar!
>>
>> On Thu, Nov 19, 2015 at 2:32 PM, Ihar Hrachyshka <ihrachys at redhat.com>
>> wrote:
>> UPD: now that we have some understanding what’s needed from l2 agent
>> extension mechanism to cater for interested subprojects (and now that
>> we see that probably the agent in focus right now is OVS only), we may
>> move to RFE step. I reported the following RFE for the feature:
>>
>> https://bugs.launchpad.net/neutron/+bug/1517903
>>
>> It may require BP if drivers team will request one.
>>
>> Cheers,
>>
>> Ihar
>>
>> Ihar Hrachyshka <ihrachys at redhat.com> wrote:
>>
>> Reviving the thread.
>>
>> On the design summit session dedicated to agent and plugin extensions
>> [1] the following was stated for l2 agent extensions (I appreciate if
>> someone checks me on the following though):
>>
>> - current l2 agent extensions mechanism lacks insight into agent
>> details like bridges or vlan maps;
>>
>> - in some cases, we don’t care about extension portability across
>> multiple agents, so it’s not of concern if some of them use
>> implementation details like bridges to set specific flows, or to wire
>> up some additional ports to them;
>>
>> - that said, we still don’t want extensions to have unlimited access
>> to agent details; the rationale for hard constraints on what is seen
>> inside extensions is that we cannot support backwards compatibility
>> for *all* possible internal attributes of an agent; instead, we should
>> explicitly define where we can make an effort to provide stable API
>> into agent details, and what’s, on contrary, beyond real life use
>> cases and hence can be left to be broken/refactored as neutron
>> developers see fit; this API can be agent specific though;
>>
>> - agent details that are to be passed into extensions should be driven
>> by actual use cases. There were several subprojects mentioned in the
>> session that are assumed to lack enough access to agent attributes to
>> do their job without patching core ovs agent files. Those are:
>> BGP-VPN, SFC, (anything else?) Those subprojects that are interested
>> in extending l2 agent extension framework are expected to come up with
>> a list of things missing in current implementation, so that neutron
>> developers can agree on proper abstractions to provide missing details
>> to extensions. For that goal, I set up a new etherpad to collect
>> feedback from subprojects [2].
>>
>> Once we collect use cases there and agree on agent API for extensions
>> (even if per agent type), we will implement it and define as stable
>> API, then pass objects that implement the API into extensions thru
>> extension manager. If extensions support multiple agent types, they
>> can still distinguish between which API to use based on agent type
>> string passed into extension manager.
>>
>> I really hope we start to collect use cases early so that we have time
>> to polish agent API and make it part of l2 extensions earlier in
>> Mitaka cycle.
>>
>> [1]: https://etherpad.openstack.org/p/mitaka-neutron-core-extensibility
>> [2]: https://etherpad.openstack.org/p/l2-agent-extensions-api-expansion
>>
>> Ihar
>>
>> Ihar Hrachyshka <ihrachys at redhat.com> wrote:
>>
>> On 30 Sep 2015, at 12:53, Miguel Angel Ajo <mangelajo at redhat.com> wrote:
>>
>>
>>
>> Ihar Hrachyshka wrote:
>> On 30 Sep 2015, at 12:08, thomas.morin at orange.com wrote:
>>
>> Hi Ihar,
>>
>> Ihar Hrachyshka :
>> Miguel Angel Ajo :
>> Do you have a rough idea of what operations you may need to do?
>> Right now, what bagpipe driver for networking-bgpvpn needs to interact
>> with is:
>> - int_br OVSBridge (read-only)
>> - tun_br OVSBridge (add patch port, add flows)
>> - patch_int_ofport port number (read-only)
>> - local_vlan_map dict (read-only)
>> - setup_entry_for_arp_reply method (called to add static ARP entries)
>> Sounds very tightly coupled to OVS agent.
>> Please bear in mind, the extension interface will be available from
>> different agent types
>> (OVS, SR-IOV, [eventually LB]), so this interface you're talking about
>> could also serve as
>> a translation driver for the agents (where the translation is
>> possible), I totally understand
>> that most extensions are specific agent bound, and we must be able to
>> identify
>> the agent we're serving back exactly.
>> Yes, I do have this in mind, but what we've identified for now seems
>> to be OVS specific.
>> Indeed it does. Maybe you can try to define the needed pieces in high
>> level actions, not internal objects you need to access to. Like ‘-
>> connect endpoint X to Y’, ‘determine segmentation id for a network’ etc.
>> I've been thinking about this, but would tend to reach the conclusion
>> that the things we need to interact with are pretty hard to abstract
>> out into something that would be generic across different agents.
>> Everything we need to do in our case relates to how the agents use
>> bridges and represent networks internally: linuxbridge has one bridge
>> per Network, while OVS has a limited number of bridges playing
>> different roles for all networks with internal segmentation.
>>
>> To look at the two things you  mention:
>> - "connect endpoint X to Y" : what we need to do is redirect the
>> traffic destinated to the gateway of a Neutron network, to the thing
>> that will do the MPLS forwarding for the right BGP VPN context (called
>> VRF), in our case br-mpls (that could be done with an OVS table too) ;
>> that action might be abstracted out to hide the details specific to
>> OVS, but I'm not sure on how to  name the destination in a way that
>> would be agnostic to these details, and this is not really relevant to
>> do until we have a relevant context in which the linuxbridge would
>> pass packets to something doing MPLS forwarding (OVS is currently the
>> only option we support for MPLS forwarding, and it does not really
>> make sense to mix linuxbridge for Neutron L2/L3 and OVS for MPLS)
>> - "determine segmentation id for a network": this is something really
>> OVS-agent-specific, the linuxbridge agent uses multiple linux bridges,
>> and does not rely on internal segmentation
>>
>> Completely abstracting out packet forwarding pipelines in OVS and
>> linuxbridge agents would possibly allow defining an interface that
>> agent extension could use without to know about anything specific to
>> OVS or the linuxbridge, but I believe this is a very significant taks
>> to tackle.
>>
>> If you look for a clean way to integrate with reference agents, then
>> it’s something that we should try to achieve. I agree it’s not an easy
>> thing.
>>
>> Just an idea: can we have a resource for traffic forwarding, similar
>> to security groups? I know folks are not ok with extending security
>> groups API due to compatibility reasons, so maybe fwaas is the place
>> to experiment with it.
>>
>> Hopefully it will be acceptable to create an interface, even it
>> exposes a set of methods specific to the linuxbridge agent and a set
>> of methods specific to the OVS agent.  That would mean that the agent
>> extension that can work in both contexts (not our case yet) would
>> check the agent type before using the first set or the second set.
>>
>> The assumption of the whole idea of l2 agent extensions is that they
>> are agent agnostic. In case of QoS, we implemented a common QoS
>> extension that can be plugged in any agent [1], and a set of backend
>> drivers (atm it’s just sr-iov [2] and ovs [3]) that are selected based
>> on the driver type argument passed into the extension manager [4][5].
>> Your extension could use similar approach to select the backend.
>>
>> [1]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/agent/l2/extensions/qos.py#n169
>>
>> [2]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/mech_sriov/agent/extension_drivers/qos_driver.py
>>
>> [3]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/openvswitch/agent/extension_drivers/qos_driver.py
>>
>> [4]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#n395
>>
>> [5]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py#n155
>>
>>
>> I disagree on the agent-agnostic thing. QoS extension for SR-IOV is
>> totally not agnostic for OVS or LB, in the QoS case, it's just
>> accidental that OVS & LB share common bridges now due to the OVS
>> Hybrid implementation that leverages linux bridge
>> and iptables.
>>
>> Wait. The QoS extension has nothing agent backend specific. All it
>> does is it receives rpc updates for tracked resources and pass them
>> into qos drivers. Those latter are the bits that implement backend
>> specific operations. So I am not sure why you say the extension itself
>> is agent specific: any other amqp based agent in the wild can adopt
>> the extension as-is, only providing a new backend to load.
>>
>> I agree on having a well defined interface, on which API is available
>> to talking back to each agent, and it has to be common, where
>> it's possible to be common.
>>
>> It doesn't have to be easy, but it's the way if we want a world where
>> those commonalities and reusability of extensions can
>> exist and not be just accidental, but it's not realistic in my opinion
>> to AIM for it on every shot. I believe we should try where we can
>> but we should be open to agent specific extensions. The idea of the
>> extensions is that you can extend specific agents without
>> being forced to have the main loop hijacked, or eventually having off
>> tree code plugged into our agents.
>>
>> Partially, yes. The culprit here is how much the extension API should
>> know about an agent. We can probably make the extension API completely
>> extendable by allowing agents to pass any random kwargs into the
>> extension manager that will forward them into extensions. Note that it
>> breaks current API for extensions and technically breaks it (not that
>> I know of any external extensions that could be affected so far).
>>
>> There we should add support to identify the type of agent the
>> extension works with (compatibility, versioning, etc..)
>>
>> We already pass the type into extension manager, and that’s how we
>> plug in the proper backend driver in QoS.
>>
>> Does this approach make sense ?
>>
>> -Thomas
>>
>> _________________________________________________________________________________________________________________________
>>
>>
>> Ce message et ses pieces jointes peuvent contenir des informations
>> confidentielles ou privilegiees et ne doivent donc
>> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
>> recu ce message par erreur, veuillez le signaler
>> a l'expediteur et le detruire ainsi que les pieces jointes. Les
>> messages electroniques etant susceptibles d'alteration,
>> Orange decline toute responsabilite si ce message a ete altere,
>> deforme ou falsifie. Merci.
>>
>> This message and its attachments may contain confidential or
>> privileged information that may be protected by law;
>> they should not be distributed, used or copied without authorisation.
>> If you have received this email in error, please notify the sender and
>> delete this message and its attachments.
>> As emails may be altered, Orange is not liable for messages that have
>> been modified, changed or falsified.
>> Thank you.
>>
>> Note that you should really avoid putting that ^^ kind of signature
>> into your emails intended for public mailing lists. If it’s
>> confidential, why do you send it to everyone? And sorry, folks will
>> copy it without authorisation, for archiving and indexing reasons and
>> whatnot.
>>
>> Ihar
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev