Open Stack

Damn. Found it.
I stupidly forgot to add manual route to 192.168.1.252 for qg (gateway) 
network !
I had all the keys, I knew that for metadata traffic, you need external 
mapping to router IP, I saw that iptables was saying 'outbound traffic 
thru 192.168.10.1' (ie. qg - router), but I didn't noticed to add it.

Correct route is as follows :
sylvain at folsom02:~$ route -n
Table de routage IP du noyau
Destination     Passerelle      Genmask         Indic Metric Ref    Use 
Iface
0.0.0.0         192.168.1.252   0.0.0.0         UG    0 0        0 
qg-f2397006-20
0.0.0.0         192.168.1.252   0.0.0.0         UG    0 0        0 br-ex
10.0.0.0        0.0.0.0         255.255.255.0   U     0 0        0 
tap2ed3cd8a-03
10.0.0.0        0.0.0.0         255.255.255.0   U     0 0        0 
qr-f76e4668-fa
169.254.0.0     0.0.0.0         255.255.0.0     U     1000 0        0 eth0
172.16.0.0      0.0.0.0         255.255.240.0   U     0 0        0 eth0
192.168.1.252   0.0.0.0         255.255.255.255 UH    0 0        0 
qg-f2397006-20
192.168.1.252   0.0.0.0         255.255.255.255 UH    0 0        0 br-ex
192.168.10.0    0.0.0.0         255.255.255.0   U     0 0        0 
qg-f2397006-20
192.168.10.0    0.0.0.0         255.255.255.0   U     0 0        0 br-ex

(I also removed the default entry for eth0 (mngmt interface), as it was 
nonsense).

-Sylvain

Le 19/02/2013 20:39, Sylvain Bauza a écrit :
> Hi,
>
> I have a classic "Provider Network, private networks" as follows :
>  - internal network 10.0.0.0/24
>  - external network 192.168.10.0/24 gw 192.168.1.252 (I know, I have 
> to add a manual route on both gw and network node)
>  - br-ex is having 192.168.10.254
>
> I have a floating IP 192.168.10.2 NAT'd to 10.0.0.4 and my external 
> router IP address is 192.168.10.1
>
> For some reason, on br-ex, I see traffic from 10.0.0.4 to 
> 192.168.1.154 (my own computer) with tcpdump :
> 19:55:58.938524 00:13:8f:51:96:9d > bc:ae:c5:d6:9d:60, ethertype IPv4 
> (0x0800), length 98: 10.0.0.4 > 192.168.1.154: ICMP echo request, id 
> 62465, seq 1189, length 64
>
> I should at least see a second ICMP request from 192.168.10.2 to 
> 192.168.1.154 (with same seq. number) if L3 agent would work correctly.
> DNAT is working correctly : I can ping from 192.168.1.154 to 
> 192.168.10.2.
> As a consequence, I can't access Internet from my VMs, but I can reach 
> them thru SSH.
>
> Do you have any idea for the root cause ?
> As said, the problem is bounded to a SNAT issue with Quantum L3 agent, 
> as DNAT is working perfectly.
>
> Please find below my iproute2 status, my route configuration, plus 
> iptables -t nat.
> sylvain at folsom02:~$ route -n
> Table de routage IP du noyau
> Destination     Passerelle      Genmask         Indic Metric Ref Use 
> Iface
> 0.0.0.0         192.168.1.252   0.0.0.0         UG    0 0        0 br-ex
> 0.0.0.0         172.16.0.2      0.0.0.0         UG    0 0        0 eth0
> 10.0.0.0        0.0.0.0         255.255.255.0   U     0 0        0 
> tap2ed3cd8a-03
> 10.0.0.0        0.0.0.0         255.255.255.0   U     0 0        0 
> qr-f76e4668-fa
> 169.254.0.0     0.0.0.0         255.255.0.0     U     1000 0        0 
> eth0
> 172.16.0.0      0.0.0.0         255.255.240.0   U     0 0        0 eth0
> 192.168.1.252   0.0.0.0         255.255.255.255 UH    0 0        0 br-ex
> 192.168.10.0    0.0.0.0         255.255.255.0   U     0 0        0 
> qg-f2397006-20
> 192.168.10.0    0.0.0.0         255.255.255.0   U     0 0        0 br-ex
>
> sylvain at folsom02:~$ ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>     inet 127.0.0.1/8 scope host lo
>     inet6 ::1/128 scope host
>        valid_lft forever preferred_lft forever
> 2: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc 
> pfifo_fast state UP qlen 1000
>     link/ether 00:13:8f:51:96:9d brd ff:ff:ff:ff:ff:ff
> 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast 
> state UNKNOWN qlen 1000
>     link/ether 00:e0:7d:b5:df:ae brd ff:ff:ff:ff:ff:ff
>     inet 172.16.0.2/20 brd 172.16.15.255 scope global eth0
>     inet6 fe80::2e0:7dff:feb5:dfae/64 scope link
>        valid_lft forever preferred_lft forever
> 6: br-ex: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc 
> noqueue state UNKNOWN
>     link/ether 00:13:8f:51:96:9d brd ff:ff:ff:ff:ff:ff
>     inet 192.168.10.254/24 scope global br-ex
> 11: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>     link/ether ca:0d:5d:80:15:49 brd ff:ff:ff:ff:ff:ff
> 12: qr-f76e4668-fa: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 
> qdisc noqueue state UNKNOWN
>     link/ether fa:16:3e:4a:82:b6 brd ff:ff:ff:ff:ff:ff
>     inet 10.0.0.1/24 brd 10.0.0.255 scope global qr-f76e4668-fa
>     inet6 fe80::f816:3eff:fe4a:82b6/64 scope link
>        valid_lft forever preferred_lft forever
> 13: qg-f2397006-20: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 
> qdisc noqueue state UNKNOWN
>     link/ether fa:16:3e:e1:95:20 brd ff:ff:ff:ff:ff:ff
>     inet 192.168.10.1/24 brd 192.168.10.255 scope global qg-f2397006-20
>     inet 192.168.10.2/32 brd 192.168.10.2 scope global qg-f2397006-20
>     inet6 fe80::f816:3eff:fee1:9520/64 scope link
>        valid_lft forever preferred_lft forever
> 14: tap2ed3cd8a-03: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 
> qdisc noqueue state UNKNOWN
>     link/ether fa:16:3e:8f:3c:92 brd ff:ff:ff:ff:ff:ff
>     inet 10.0.0.2/24 brd 10.0.0.255 scope global tap2ed3cd8a-03
>     inet6 fe80::f816:3eff:fe8f:3c92/64 scope link
>        valid_lft forever preferred_lft forever
> 15: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>     link/ether 4e:46:e7:ec:52:49 brd ff:ff:ff:ff:ff:ff
>
> sylvain at folsom02:~$ sudo iptables -t nat -L -n
> Chain PREROUTING (policy ACCEPT)
> target     prot opt source               destination
> quantum-l3-agent-PREROUTING  all  --  0.0.0.0/0 0.0.0.0/0
>
> Chain INPUT (policy ACCEPT)
> target     prot opt source               destination
>
> Chain OUTPUT (policy ACCEPT)
> target     prot opt source               destination
> quantum-l3-agent-OUTPUT  all  --  0.0.0.0/0 0.0.0.0/0
>
> Chain POSTROUTING (policy ACCEPT)
> target     prot opt source               destination
> quantum-l3-agent-POSTROUTING  all  --  0.0.0.0/0 0.0.0.0/0
> quantum-postrouting-bottom  all  --  0.0.0.0/0 0.0.0.0/0
>
> Chain quantum-l3-agent-OUTPUT (1 references)
> target     prot opt source               destination
> DNAT       all  --  0.0.0.0/0            192.168.10.2 to:10.0.0.4
>
> Chain quantum-l3-agent-POSTROUTING (1 references)
> target     prot opt source               destination
> ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ! 
> ctstate DNAT
> ACCEPT     all  --  10.0.0.0/24          172.16.0.1
>
> Chain quantum-l3-agent-PREROUTING (1 references)
> target     prot opt source               destination
> DNAT       tcp  --  0.0.0.0/0            169.254.169.254      tcp 
> dpt:80 to:172.16.0.1:8775
> DNAT       all  --  0.0.0.0/0            192.168.10.2 to:10.0.0.4
>
> Chain quantum-l3-agent-float-snat (1 references)
> target     prot opt source               destination
> SNAT       all  --  10.0.0.4             0.0.0.0/0 to:192.168.10.2
>
> Chain quantum-l3-agent-snat (1 references)
> target     prot opt source               destination
> quantum-l3-agent-float-snat  all  --  0.0.0.0/0 0.0.0.0/0
> SNAT       all  --  10.0.0.0/24          0.0.0.0/0 to:192.168.10.1
>
> Chain quantum-postrouting-bottom (1 references)
> target     prot opt source               destination
> quantum-l3-agent-snat  all  --  0.0.0.0/0            0.0.0.0/0
>
>
> Thanks for your help,
> -Sylvain