Open Stack

hi,

On Fri, Dec 4, 2015 at 12:46 AM, Ihar Hrachyshka <ihrachys at redhat.com> wrote:
> Hi,
>
> Small update on the RFE. It was approved for Mitaka, assuming we come up
> with proper details upfront thru neutron-specs process.
>
> In the meantime, we have found more use cases for flow management among
> features in development: QoS DSCP, also the new OF based firewall driver.
> Both authors for those new features independently realized that agent does
> not currently play nice with flows set by external code due to its graceful
> restart behaviour when rules with unknown cookies are cleaned up. [The agent
> uses a random session uuid() to mark rules that belong to its current run.]
>
> Before I proceed, full disclosure: I know almost nothing about OpenFlow
> capabilities, so some pieces below may make no sense. I tried to come up
> with high level model first and then try to map it to available OF features.
> Please don’t hesitate to comment, I like to learn new stuff! ;)
>
> I am thinking lately on the use cases we collected so far. One common need
> for all features that were seen to be interested in proper integration with
> Open vSwitch agent is to be able to manage feature specific flows on br-int
> and br-tun. There are other things that projects may need, like patch ports,
> though I am still struggling with the question of whether it may be
> postponed or avoided for phase 1.

i suspect port management is mandatory for many of usecases.

>
> There are several specific operation 'kinds' that we should cover for the
> RFE:
> - managing flows that modify frames in-place;
> - managing flows that redirect frames.
>
> There are some things that should be considered to make features cooperate
> with the agent and other extensions:
> - feature flows should have proper priorities based on their ‘kind’ (f.e.
> in-place modification probably go before redirections);
> - feature flows should survive flow reset that may be triggered by the
> agent;
> - feature flows should survive flow reset without data plane disruption
> (=they should support graceful restart:
> https://review.openstack.org/#/c/182920).
>
> With that in mind, I see the following high level design for the flow
> tables:
>
> - table 0 serves as a dispatcher for specific features;
> - each feature gets one or more tables, one per flow ‘kind’ needed;
> - for each feature table, a new flow entry is added to table 0 that would
> redirect to feature specific table; the rule will be triggered only if OF
> metadata is not updated inside the feature table (see the next bullet); the
> rule will have priority that is defined for the ‘kind’ of the operation that
> is implemented by the table it redirects to;
> -  each feature table will have default actions that will 1) mark OF
> metadata for the frame as processed by the feature; 2) redirect back to
> table 0;
> - all feature specific flow rules (except dispatcher rules) belong to
> feature tables;
>
> Now, the workflow for extensions that are interested in setting flows would
> be:
> - on initialize() call, extension defines feature tables it will need; it
> passes the name of the feature table and the ‘kind’ of the actions it will
> execute; with that, the following is initialized by the agent: 1) table 0
> dispatcher entry to redirect frames into feature table; the entry has the
> priority according to the ‘kind’ of the table; 2) the actual feature table
> with two default rules (update metadata and push back to table 0);
> - whenever extension needs to add a new flow rule, it passes the following
> into the agent: 1) table name; 2) flow specific parameters (actions,
> priority, ...)

"actions" here means openflow actions?

passing openflow actions as parameters is not simple as it might sound
because they are complex objects.  esp. when we have two backends.
(ovs-ofctl and native of_interface)

>
> Since the agent will manage setting flows for extensions, it will be able to
> use the active agent cookie for all feature flows; next time the agent is
> restarted, it should be able to respin extension flows with no data plane
> disruption. [Note: we should make sure that on agent restart, we call to
> extensions *before* we clean up stale flow rules.]
>
> That design will hopefully allow us to abstract interaction with flows from
> extensions into management code inside the agent. It should guarantee
> extensions cooperate properly assuming they properly define their priorities
> thru ‘kinds’ of tables they have.
>
> It is also assumed that existing flow based features integrated into the
> agent (dvr? anti-spoofing?) will eventually move to the new flow table
> management model.
>
> I understand that the model does not reflect how do feature processing for
> existing OF based features in the agent. It may require some smart
> workarounds to allow non-disruptive migration to new flow table setup.
>
> It would be great to see the design bashed hard before I start to put it
> into spec format. Especially if it’s not sane. :)

i tend to think it's better to start from a crude experimental api (eg. give
the bridge objects to extension drivers) to avoid blocking subprojects too long.

>
> Ihar
>
>
> Mathieu Rohon <mathieu.rohon at gmail.com> wrote:
>
>> Thanks ihar!
>>
>> On Thu, Nov 19, 2015 at 2:32 PM, Ihar Hrachyshka <ihrachys at redhat.com>
>> wrote:
>> UPD: now that we have some understanding what’s needed from l2 agent
>> extension mechanism to cater for interested subprojects (and now that we see
>> that probably the agent in focus right now is OVS only), we may move to RFE
>> step. I reported the following RFE for the feature:
>>
>> https://bugs.launchpad.net/neutron/+bug/1517903
>>
>> It may require BP if drivers team will request one.
>>
>> Cheers,
>>
>> Ihar
>>
>> Ihar Hrachyshka <ihrachys at redhat.com> wrote:
>>
>> Reviving the thread.
>>
>> On the design summit session dedicated to agent and plugin extensions [1]
>> the following was stated for l2 agent extensions (I appreciate if someone
>> checks me on the following though):
>>
>> - current l2 agent extensions mechanism lacks insight into agent details
>> like bridges or vlan maps;
>>
>> - in some cases, we don’t care about extension portability across multiple
>> agents, so it’s not of concern if some of them use implementation details
>> like bridges to set specific flows, or to wire up some additional ports to
>> them;
>>
>> - that said, we still don’t want extensions to have unlimited access to
>> agent details; the rationale for hard constraints on what is seen inside
>> extensions is that we cannot support backwards compatibility for *all*
>> possible internal attributes of an agent; instead, we should explicitly
>> define where we can make an effort to provide stable API into agent details,
>> and what’s, on contrary, beyond real life use cases and hence can be left to
>> be broken/refactored as neutron developers see fit; this API can be agent
>> specific though;
>>
>> - agent details that are to be passed into extensions should be driven by
>> actual use cases. There were several subprojects mentioned in the session
>> that are assumed to lack enough access to agent attributes to do their job
>> without patching core ovs agent files. Those are: BGP-VPN, SFC, (anything
>> else?) Those subprojects that are interested in extending l2 agent extension
>> framework are expected to come up with a list of things missing in current
>> implementation, so that neutron developers can agree on proper abstractions
>> to provide missing details to extensions. For that goal, I set up a new
>> etherpad to collect feedback from subprojects [2].
>>
>> Once we collect use cases there and agree on agent API for extensions
>> (even if per agent type), we will implement it and define as stable API,
>> then pass objects that implement the API into extensions thru extension
>> manager. If extensions support multiple agent types, they can still
>> distinguish between which API to use based on agent type string passed into
>> extension manager.
>>
>> I really hope we start to collect use cases early so that we have time to
>> polish agent API and make it part of l2 extensions earlier in Mitaka cycle.
>>
>> [1]: https://etherpad.openstack.org/p/mitaka-neutron-core-extensibility
>> [2]: https://etherpad.openstack.org/p/l2-agent-extensions-api-expansion
>>
>> Ihar
>>
>> Ihar Hrachyshka <ihrachys at redhat.com> wrote:
>>
>> On 30 Sep 2015, at 12:53, Miguel Angel Ajo <mangelajo at redhat.com> wrote:
>>
>>
>>
>> Ihar Hrachyshka wrote:
>> On 30 Sep 2015, at 12:08, thomas.morin at orange.com wrote:
>>
>> Hi Ihar,
>>
>> Ihar Hrachyshka :
>> Miguel Angel Ajo :
>> Do you have a rough idea of what operations you may need to do?
>> Right now, what bagpipe driver for networking-bgpvpn needs to interact
>> with is:
>> - int_br OVSBridge (read-only)
>> - tun_br OVSBridge (add patch port, add flows)
>> - patch_int_ofport port number (read-only)
>> - local_vlan_map dict (read-only)
>> - setup_entry_for_arp_reply method (called to add static ARP entries)
>> Sounds very tightly coupled to OVS agent.
>> Please bear in mind, the extension interface will be available from
>> different agent types
>> (OVS, SR-IOV, [eventually LB]), so this interface you're talking about
>> could also serve as
>> a translation driver for the agents (where the translation is possible), I
>> totally understand
>> that most extensions are specific agent bound, and we must be able to
>> identify
>> the agent we're serving back exactly.
>> Yes, I do have this in mind, but what we've identified for now seems to be
>> OVS specific.
>> Indeed it does. Maybe you can try to define the needed pieces in high
>> level actions, not internal objects you need to access to. Like ‘- connect
>> endpoint X to Y’, ‘determine segmentation id for a network’ etc.
>> I've been thinking about this, but would tend to reach the conclusion that
>> the things we need to interact with are pretty hard to abstract out into
>> something that would be generic across different agents.  Everything we need
>> to do in our case relates to how the agents use bridges and represent
>> networks internally: linuxbridge has one bridge per Network, while OVS has a
>> limited number of bridges playing different roles for all networks with
>> internal segmentation.
>>
>> To look at the two things you  mention:
>> - "connect endpoint X to Y" : what we need to do is redirect the traffic
>> destinated to the gateway of a Neutron network, to the thing that will do
>> the MPLS forwarding for the right BGP VPN context (called VRF), in our case
>> br-mpls (that could be done with an OVS table too) ; that action might be
>> abstracted out to hide the details specific to OVS, but I'm not sure on how
>> to  name the destination in a way that would be agnostic to these details,
>> and this is not really relevant to do until we have a relevant context in
>> which the linuxbridge would pass packets to something doing MPLS forwarding
>> (OVS is currently the only option we support for MPLS forwarding, and it
>> does not really make sense to mix linuxbridge for Neutron L2/L3 and OVS for
>> MPLS)
>> - "determine segmentation id for a network": this is something really
>> OVS-agent-specific, the linuxbridge agent uses multiple linux bridges, and
>> does not rely on internal segmentation
>>
>> Completely abstracting out packet forwarding pipelines in OVS and
>> linuxbridge agents would possibly allow defining an interface that agent
>> extension could use without to know about anything specific to OVS or the
>> linuxbridge, but I believe this is a very significant taks to tackle.
>>
>> If you look for a clean way to integrate with reference agents, then it’s
>> something that we should try to achieve. I agree it’s not an easy thing.
>>
>> Just an idea: can we have a resource for traffic forwarding, similar to
>> security groups? I know folks are not ok with extending security groups API
>> due to compatibility reasons, so maybe fwaas is the place to experiment with
>> it.
>>
>> Hopefully it will be acceptable to create an interface, even it exposes a
>> set of methods specific to the linuxbridge agent and a set of methods
>> specific to the OVS agent.  That would mean that the agent extension that
>> can work in both contexts (not our case yet) would check the agent type
>> before using the first set or the second set.
>>
>> The assumption of the whole idea of l2 agent extensions is that they are
>> agent agnostic. In case of QoS, we implemented a common QoS extension that
>> can be plugged in any agent [1], and a set of backend drivers (atm it’s just
>> sr-iov [2] and ovs [3]) that are selected based on the driver type argument
>> passed into the extension manager [4][5]. Your extension could use similar
>> approach to select the backend.
>>
>> [1]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/agent/l2/extensions/qos.py#n169
>> [2]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/mech_sriov/agent/extension_drivers/qos_driver.py
>> [3]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/openvswitch/agent/extension_drivers/qos_driver.py
>> [4]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#n395
>> [5]:
>> https://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/mech_sriov/agent/sriov_nic_agent.py#n155
>>
>> I disagree on the agent-agnostic thing. QoS extension for SR-IOV is
>> totally not agnostic for OVS or LB, in the QoS case, it's just
>> accidental that OVS & LB share common bridges now due to the OVS Hybrid
>> implementation that leverages linux bridge
>> and iptables.
>>
>> Wait. The QoS extension has nothing agent backend specific. All it does is
>> it receives rpc updates for tracked resources and pass them into qos
>> drivers. Those latter are the bits that implement backend specific
>> operations. So I am not sure why you say the extension itself is agent
>> specific: any other amqp based agent in the wild can adopt the extension
>> as-is, only providing a new backend to load.
>>
>> I agree on having a well defined interface, on which API is available to
>> talking back to each agent, and it has to be common, where
>> it's possible to be common.
>>
>> It doesn't have to be easy, but it's the way if we want a world where
>> those commonalities and reusability of extensions can
>> exist and not be just accidental, but it's not realistic in my opinion to
>> AIM for it on every shot. I believe we should try where we can
>> but we should be open to agent specific extensions. The idea of the
>> extensions is that you can extend specific agents without
>> being forced to have the main loop hijacked, or eventually having off tree
>> code plugged into our agents.
>>
>> Partially, yes. The culprit here is how much the extension API should know
>> about an agent. We can probably make the extension API completely extendable
>> by allowing agents to pass any random kwargs into the extension manager that
>> will forward them into extensions. Note that it breaks current API for
>> extensions and technically breaks it (not that I know of any external
>> extensions that could be affected so far).
>>
>> There we should add support to identify the type of agent the extension
>> works with (compatibility, versioning, etc..)
>>
>> We already pass the type into extension manager, and that’s how we plug in
>> the proper backend driver in QoS.
>>
>> Does this approach make sense ?
>>
>> -Thomas
>>
>>
>> _________________________________________________________________________________________________________________________
>>
>> Ce message et ses pieces jointes peuvent contenir des informations
>> confidentielles ou privilegiees et ne doivent donc
>> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
>> recu ce message par erreur, veuillez le signaler
>> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
>> electroniques etant susceptibles d'alteration,
>> Orange decline toute responsabilite si ce message a ete altere, deforme ou
>> falsifie. Merci.
>>
>> This message and its attachments may contain confidential or privileged
>> information that may be protected by law;
>> they should not be distributed, used or copied without authorisation.
>> If you have received this email in error, please notify the sender and
>> delete this message and its attachments.
>> As emails may be altered, Orange is not liable for messages that have been
>> modified, changed or falsified.
>> Thank you.
>>
>> Note that you should really avoid putting that ^^ kind of signature into
>> your emails intended for public mailing lists. If it’s confidential, why do
>> you send it to everyone? And sorry, folks will copy it without
>> authorisation, for archiving and indexing reasons and whatnot.
>>
>> Ihar
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev