Open Stack

On Aug 19, 2016, at 5:35 PM, Neil Jerram <neil at tigera.io<mailto:neil at tigera.io>> wrote:

On Fri, Aug 19, 2016 at 9:25 PM John Belamaric <jbelamaric at infoblox.com<mailto:jbelamaric at infoblox.com>> wrote:
>
> Yes, and that was what I was alluding to in order to make packets go out the correct interface based on the source IP in the packets.
>

You can configure it to send the packets out the interface the original packet came in on. That solves this problem. You don’t have to use source IP.


Am I right that we are talking here about a configuration that can be done in the instance OS?  If so, could you say exactly how to configure that, John, when the instance OS is Linux?


Yes, it’s in the instance. You can explicitly create a routing table and associate it to the interface with ”ip rule”. I just tried this and it worked. I created an instance with two interfaces on two different networks. There are two different routers for those networks (but just one external net):


[cid:7C6CEDF9-0675-40DF-A630-54D2B29ADCB6 at inmd.infoblox.com]




I associated one floating IP with eth0 and one with eth1. In the route-test instance, eth0 was configured via DHCP with 10.222.243.59/24 and I manually configured eth1 like this:

# ip a add 10.250.250.82/24 dev eth1
# ip link set dev eth1 up
# ip route add 10.250.250.0/24 table 100 scope link dev eth1
# ip rule add from 10.250.250.82 lookup 100
# ip route add default via 10.250.250.1 table 100 dev eth1


Then doing TCP dump and pinging from another machine to the floating IP I see this (note the return packet coming from source IP 10.250.250.82, NOT 10.222.243.59):

15:01:13.671491 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:14.939527 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:15.939436 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:16.939410 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:17.937020 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 1, length 64
15:01:17.937056 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 1, length 64
15:01:18.560065 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:18.937874 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 2, length 64
15:01:18.937925 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 2, length 64
15:01:19.559314 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:19.939402 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 3, length 64
15:01:19.939428 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 3, length 64
15:01:20.559428 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:20.940835 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 4, length 64
15:01:20.940887 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 4, length 64
15:01:21.732527 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28
15:01:21.942237 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 5, length 64
15:01:21.942262 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 5, length 64


FYI here is what the setup looks like when it’s done:

root at route-test:~# ip route show
default via 10.222.243.1 dev eth0
10.222.243.0/24 dev eth0  proto kernel  scope link  src 10.222.243.59
10.250.250.0/24 dev eth1  proto kernel  scope link  src 10.250.250.82
169.254.169.254 via 10.222.243.1 dev eth0
root at route-test:~# ip route show table 100
default via 10.250.250.1 dev eth1
10.250.250.0/24 dev eth1  scope link
root at route-test:~# ip rule
0:     from all lookup local
32765: from 10.250.250.82 lookup 100
32766: from all lookup main
32767: from all lookup default
root at route-test:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:70:c2:1b brd ff:ff:ff:ff:ff:ff
    inet 10.222.243.59/24 brd 10.222.243.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe70:c21b/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:b3:5c:49 brd ff:ff:ff:ff:ff:ff
    inet 10.250.250.82/24 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feb3:5c49/64 scope link
       valid_lft forever preferred_lft forever
root at route-test:~#

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160822/863bb180/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: route-test-topology.png
Type: image/png
Size: 111551 bytes
Desc: route-test-topology.png
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160822/863bb180/attachment.png>