<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Aug 19, 2016, at 5:35 PM, Neil Jerram <<a href="mailto:neil@tigera.io" class="">neil@tigera.io</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Fri, Aug 19, 2016 at 9:25 PM John Belamaric <<a href="mailto:jbelamaric@infoblox.com" class="">jbelamaric@infoblox.com</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
><br class="">
> Yes, and that was what I was alluding to in order to make packets go out the correct interface based on the source IP in the packets.<br class="">
><br class="">
<br class="">
You can configure it to send the packets out the interface the original packet came in on. That solves this problem. You don’t have to use source IP.<br class="">
<br class="">
</blockquote>
<div class=""><br class="">
</div>
<div class="">Am I right that we are talking here about a configuration that can be done in the instance OS? If so, could you say exactly how to configure that, John, when the instance OS is Linux?<br class="">
<br class="">
</div>
</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>Yes, it’s in the instance. You can explicitly create a routing table and associate it to the interface with ”ip rule”. I just tried this and it worked. I created an instance with two interfaces on two different networks. There are two different routers
for those networks (but just one external net):</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><img apple-inline="yes" id="2C33173B-397D-40C8-A6D8-DD89C2717EC4" height="512" width="796" apple-width="yes" apple-height="yes" src="cid:7C6CEDF9-0675-40DF-A630-54D2B29ADCB6@inmd.infoblox.com" class=""></div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div>I associated one floating IP with eth0 and one with eth1. In the route-test instance, eth0 was configured via DHCP with 10.222.243.59/24 and I manually configured eth1 like this:</div>
<div><br class="">
</div>
<div># ip a add 10.250.250.82/24 dev eth1</div>
<div># ip link set dev eth1 up</div>
<div># ip route add 10.250.250.0/24 table 100 scope link dev eth1</div>
<div># ip rule add from 10.250.250.82 lookup 100</div>
<div># ip route add default via 10.250.250.1 table 100 dev eth1</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div>Then doing TCP dump and pinging from another machine to the floating IP I see this (note the return packet coming from source IP 10.250.250.82, NOT 10.222.243.59):</div>
<div><br class="">
</div>
<div>
<div>15:01:13.671491 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:14.939527 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:15.939436 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:16.939410 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:17.937020 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 1, length 64</div>
<div>15:01:17.937056 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 1, length 64</div>
<div>15:01:18.560065 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:18.937874 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 2, length 64</div>
<div>15:01:18.937925 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 2, length 64</div>
<div>15:01:19.559314 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:19.939402 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 3, length 64</div>
<div>15:01:19.939428 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 3, length 64</div>
<div>15:01:20.559428 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:20.940835 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 4, length 64</div>
<div>15:01:20.940887 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 4, length 64</div>
<div>15:01:21.732527 ARP, Request who-has 10.250.250.71 tell 10.250.250.1, length 28</div>
<div>15:01:21.942237 IP 172.22.138.232 > 10.250.250.82: ICMP echo request, id 3120, seq 5, length 64</div>
<div>15:01:21.942262 IP 10.250.250.82 > 172.22.138.232: ICMP echo reply, id 3120, seq 5, length 64</div>
</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div>FYI here is what the setup looks like when it’s done:</div>
<div><br class="">
</div>
<div>
<div>root@route-test:~# ip route show</div>
<div>default via 10.222.243.1 dev eth0</div>
<div>10.222.243.0/24 dev eth0 proto kernel scope link src 10.222.243.59</div>
<div>10.250.250.0/24 dev eth1 proto kernel scope link src 10.250.250.82</div>
<div>169.254.169.254 via 10.222.243.1 dev eth0</div>
<div>root@route-test:~# ip route show table 100</div>
<div>default via 10.250.250.1 dev eth1</div>
<div>10.250.250.0/24 dev eth1 scope link</div>
<div>root@route-test:~# ip rule</div>
<div>0: <span class="Apple-tab-span" style="white-space:pre"></span>from all lookup local</div>
<div>32765: <span class="Apple-tab-span" style="white-space:pre"></span>from 10.250.250.82 lookup 100</div>
<div>32766: <span class="Apple-tab-span" style="white-space:pre"></span>from all lookup main</div>
<div>32767: <span class="Apple-tab-span" style="white-space:pre"></span>from all lookup default</div>
<div>root@route-test:~# ip a</div>
<div>1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default</div>
<div> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00</div>
<div> inet 127.0.0.1/8 scope host lo</div>
<div> valid_lft forever preferred_lft forever</div>
<div> inet6 ::1/128 scope host</div>
<div> valid_lft forever preferred_lft forever</div>
<div>2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000</div>
<div> link/ether fa:16:3e:70:c2:1b brd ff:ff:ff:ff:ff:ff</div>
<div> inet 10.222.243.59/24 brd 10.222.243.255 scope global eth0</div>
<div> valid_lft forever preferred_lft forever</div>
<div> inet6 fe80::f816:3eff:fe70:c21b/64 scope link</div>
<div> valid_lft forever preferred_lft forever</div>
<div>3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000</div>
<div> link/ether fa:16:3e:b3:5c:49 brd ff:ff:ff:ff:ff:ff</div>
<div> inet 10.250.250.82/24 scope global eth1</div>
<div> valid_lft forever preferred_lft forever</div>
<div> inet6 fe80::f816:3eff:feb3:5c49/64 scope link</div>
<div> valid_lft forever preferred_lft forever</div>
<div>root@route-test:~#</div>
<div><br class="">
</div>
</div>
</div>
</body>
</html>