Open Stack

Awesome information!

Thanks for sharing with the rest of the OpenStack community.

Edgar

From: Matt Kassawara <mkassawara at gmail.com<mailto:mkassawara at gmail.com>>
Date: Wednesday, December 10, 2014 at 8:52 AM
To: "Vasudevan, Swaminathan (PNB Roseville)" <swaminathan.vasudevan at hp.com<mailto:swaminathan.vasudevan at hp.com>>
Cc: Edgar Magana <edgar.magana at workday.com<mailto:edgar.magana at workday.com>>, Phil Hopkins <phil.hopkins at gmail.com<mailto:phil.hopkins at gmail.com>>, Nicholas Chase <nchase at mirantis.com<mailto:nchase at mirantis.com>>, "openstack-docs at lists.openstack.org<mailto:openstack-docs at lists.openstack.org>" <openstack-docs at lists.openstack.org<mailto:openstack-docs at lists.openstack.org>>
Subject: Re: DVR network traffic flow details

Thanks... I'll add this as soon as possible.

On Tue, Dec 9, 2014 at 8:25 PM, Vasudevan, Swaminathan (PNB Roseville) <swaminathan.vasudevan at hp.com<mailto:swaminathan.vasudevan at hp.com>> wrote:
Hi Matt,
I have put together the details and functions of each entity for the East-West routing and North-South without FloatingIP.
I will send you an update on the “North-South with FloatingIP” tomorrow.

Please let me know if you have any questions or concerns.


1)     East-west for instances with or without a floating IP.



[cid:image003.jpg at 01D013DD.8DFE0740]
1  -->The frame with destination ip as ‘vm2 ip’ is sent by vm1 towards its default gateway mac for red network which is r1-red-mac.    The integration bridge forwards this frame to DVR router r1. ( br-int)

2 -->The DVR router r1’s red subnet interface picks this frame and it then routes the IP packet in the frame.  ( router-namespace)
NOTE: The router_namespace arp table is populated by the l3_agent dynamically.
3 --> After routing, the DVR router r1 puts the routed frame out of its green subnet interface. (router-namespace)

4 --> This frame is switched by the integration bridge towards the tunnel bridge along with tagging the frame with green network’s local-vlan tag. (br-int)

5 --> The tunnel bridge on CN1, replaces the frame’s source mac address with a Unique DVR MAC Address of its node (one unique dvr mac address is assigned per compute node by the controller).  The resulting frame is forwarded to CN2 by this tunnel bridge.  Before forwarding, it also strips the local green-vlan tag and tunnels the frame with green-vni VXLAN id. ( br-tun)

6 --> The tunnel bridge on CN2, picks up the tunneled frame, de-tunnels it and strips off the green-vni tag.  It then adds its local green network vlan tag to the frame and forwards the frame to the integration bridge. ( br-tun)

7 --> The integration bridge on CN2, identifies the incoming frame’s source mac address is a DVR Unique MAC Address (every compute node l2-agent knows all dvr unique mac addresses used in the cloud).  It then replaces the DVR Unique MAC Address with the green subnet interface MAC address and forwards the frame to vm2. (br-int)

br-int:
Operations:
If the packet is for the local VMs residing on the same subnet then it would be a regular switching that would happen between the VMs.

If the packet is for the local VMs residing on a different subnet then the packet from VM1 would come with a destination address of the Default gateway on the same subnet. Then the “br-int” will forward those packets to the “router-namespace’s”.

The br-int will also receive the routed packets from the “router-namespace” and then will switch those packets to the “br-tun” since the destination is on a different node.

Router_Namespace:
Operations:
The router_namespace has rules to route or forward the packet to the destination interface.
The routed packet then enters the br-int again.

br-tun:
Operations:
The br-tun replaces the source Mac with the “DVR Mac or the LMac” for the forwarding packets.
Also replaces the vlan tag with the  destination vni tag and forwards the packet to the destination Node through the tunnel.
( Note: L2 population helps to identify the VNI’s that belong to different VXLAN ids and where the nodes reside).

The br-tun also replace the vni tag with a corresponding vlan tag for the received packets.

2. North-South for instances without a Floating IP:

[cid:image005.png at 01D013DD.8DFE0740]

1  -->The frame with destination ip as ‘remote external network ip’ is sent by vm1 towards its default gateway mac for red network which is r1-red-mac.    The integration bridge forwards this frame to DVR router r1. ( br-int)

2 -->The DVR router r1’s red subnet interface picks this frame and it then redirects the IP packet in the frame on the same-interface since it’s destination is an external network’s ip address.  ( router-namespace)
NOTE: The router_namespace arp table is populated by the l3_agent dynamically.
3 --> After routing, the DVR router r1 puts the routed frame out of its red subnet interface. (router-namespace)


4 --> This frame is switched by the integration bridge towards the tunnel bridge along with tagging the frame with red network’s local-vlan tag. (br-int)

5 --> The tunnel bridge on CN1, replaces the frame’s source mac address with a Unique DVR MAC Address of its node (one unique dvr mac address is assigned per compute node by the controller).  The resulting frame is forwarded to Network Node or Service Node by this tunnel bridge.  Before forwarding, it also strips the local red-vlan tag and tunnels the frame with red-vni VXLAN id for the Network-Node or Service Node. ( br-tun)

6 --> The tunnel bridge on Network Node or Service Node, picks up the tunneled frame, de-tunnels it and strips off the red-vni tag.  It then adds its local red network vlan tag to the frame and forwards the frame to the integration bridge. ( br-tun)

7 --> The integration bridge on Network Node or Service Node, identifies the incoming frame’s source mac address is a DVR Unique MAC Address (every compute node l2-agent knows all dvr unique mac addresses used in the cloud).  It then replaces the DVR Unique MAC Address with the red subnet interface MAC address and forwards the frame to router-namespace since the MAC belongs to the gateway MAC. (br-int)

8--> The router-namespace in the Network Node will then receive the packet through the “red” interface and then will forward the packet to the “red” router-gateway interface in the SNAT Namespace. ( router_namespace)

9-->The snat-namespace will then forward the packet to the External Network after replacing the source address with the Gateway address. ( snat_namespace).

Similarly the return packet’s destination address when received by the “SNAT-Namespace” is replaced by the internal private address and then forwarded to the right interface that it belongs to.

Thanks
Swami


From: Vasudevan, Swaminathan (PNB Roseville)
Sent: Monday, December 08, 2014 6:09 PM
To: 'Matt Kassawara'; Edgar Magana; Phil Hopkins; Nicholas Chase; openstack-docs at lists.openstack.org<mailto:openstack-docs at lists.openstack.org>
Subject: RE: DVR network traffic flow details

Hi Matt,
Thanks for the information.
Regarding the details on “tables and flows” – I don’t think that is required for a regular user. But if we wanted to include a session on “Under the hood” or something, then we can add those details in the admin-guide.
Let me know if you wanted to add an Under the hood session and I can let you know the top level rules and tables just added for DVR in br-int and br-tun.

I will review the doc with my team and will get back to your on the feedback within an couple of days.
Thanks
Swami

From: Matt Kassawara [mailto:mkassawara at gmail.com]
Sent: Monday, December 08, 2014 6:06 PM
To: Vasudevan, Swaminathan (PNB Roseville); Edgar Magana; Phil Hopkins; Nicholas Chase; openstack-docs at lists.openstack.org<mailto:openstack-docs at lists.openstack.org>
Subject: DVR network traffic flow details

Swami/Edgar,

I see three distinct traffic flows with DVR:

1) North-south for instances without a floating IP.
2) North-south for instances with a floating IP.
3) East-west for instances with or without a floating IP.

I drew a diagrams for each flow that includes the primary components and their connections. For simplicity (and space considerations), I left out components that don't directly apply to DVR (such as metadata).

I also wrote a series of steps describing flow #1 in detail without getting too technical. However, I'm not exactly sure about our target audience. I would expect them to know enough about networking and neutron (perhaps from other parts of the networking guide) prior to attempting DVR, but perhaps not enough to understand tables and flows in Open vSwitch. How much detail should we provide?

I would appreciate if someone could review the diagrams for all flows and the steps describing flow #1 and provide feedback/corrections. In the interest of time, can someone also provide the steps describing flows #2 and #3 so I don't have to reverse-engineer them too? You can follow my level of detail in flow #1 or suggest something with more or less detail based on the target audience.

Ideally, I would like to submit most if not all of this content by the end of this week for inclusion into the official networking guide. I already came across several people who stumbled upon it via search engines and successfully deployed DVR in a test environment, so at least the configuration works.

https://github.com/ionosphere80/openstack-networking-guide/blob/master/scenario-dvr/scenario-dvr.md

Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-docs/attachments/20141210/9fc81e4e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 44545 bytes
Desc: image003.jpg
URL: <http://lists.openstack.org/pipermail/openstack-docs/attachments/20141210/9fc81e4e/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 163531 bytes
Desc: image005.png
URL: <http://lists.openstack.org/pipermail/openstack-docs/attachments/20141210/9fc81e4e/attachment-0001.png>