<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">On 11.03.2015 19:31, Ian Wells wrote:<br>
</div>
<blockquote
cite="mid:CAPoubz6LgqZ8vi+ZNZLT0DrKqggFh0WfcecVwnB+zr2Y5nGVBg@mail.gmail.com"
type="cite">
<div dir="ltr">On 11 March 2015 at 04:27, Fredy Neeser <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:Fredy.Neeser@solnet.ch" target="_blank">Fredy.Neeser@solnet.ch</a>></span>
wrote:<br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">7:
br-ex.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc noqueue state UNKNOWN group default<br>
<div>
<div class="h5">
link/ether e0:3f:49:b4:7c:a7 brd ff:ff:ff:ff:ff:ff<br>
inet <a moz-do-not-send="true"
href="http://192.168.1.14/24" target="_blank">192.168.1.14/24</a>
brd 192.168.1.255 scope global br-ex.1<br>
valid_lft forever preferred_lft forever<br>
<br>
8: br-ex.12: <BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1554 qdisc noqueue state UNKNOWN group default<br>
link/ether e0:3f:49:b4:7c:a7 brd ff:ff:ff:ff:ff:ff<br>
inet <a moz-do-not-send="true"
href="http://192.168.1.14/24" target="_blank">192.168.1.14/24</a>
brd 192.168.1.255 scope global br-ex.12<br>
valid_lft forever preferred_lft forever<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I find it hard to believe that you want the same
address configured on *both* of these interfaces - which
one do you think will be sending packets?<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Ian, thanks for your feedback!<br>
<br>
I did choose the same address for the two interfaces, for three
reasons:<br>
<br>
1. Within my home single-LAN (underlay) environment, traffic is
switched, and VXLAN traffic is confined to VLAN 12, so there is
never a conflict between IP 192.168.1.14 on VLAN 1 and the same IP
on VLAN 12.<br>
OTOH, for a more scalable VXLAN setup (with multiple underlays and
L3 routing in between), I would like to use different IPs for
br-ex.1 and br-ex.12 -- for example by using separate subnets<br>
192.168.1.0/26 for VLAN 1<br>
192.168.12.0/26 for VLAN 12<br>
However, I'm not quite there yet (see 3.).<br>
<br>
2. I'm using policy routing on my hosts to steer VXLAN traffic (UDP
dest. port 4789) to interface br-ex.12 -- all other traffic from
192.168.1.14 is source routed from br-ex.1, presumably because
br-ex.1 is a lower-numbered interface than br-ex.12 (?) --
interesting question whether I'm relying here on the order in which
I created these two interfaces.<br>
<br>
[root@langrain ~]# ip a<br>
...<br>
7: br-ex.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UNKNOWN group default <br>
link/ether e0:3f:49:b4:7c:a7 brd ff:ff:ff:ff:ff:ff<br>
inet 192.168.1.14/24 brd 192.168.1.255 scope global br-ex.1<br>
valid_lft forever preferred_lft forever<br>
8: br-ex.12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1554
qdisc noqueue state UNKNOWN group default <br>
link/ether e0:3f:49:b4:7c:a7 brd ff:ff:ff:ff:ff:ff<br>
inet 192.168.1.14/24 brd 192.168.1.255 scope global br-ex.12<br>
valid_lft forever preferred_lft forever<br>
<br>
3. It's not clear to me how to setup multiple nodes with packstack
if a node's tunnel IP does not equal its admin IP (or the OpenStack
API IP in case of a controller node). With packstack, I can only
specify the compute node IPs through CONFIG_COMPUTE_HOSTS.
Presumably, these IPs are used for both packstack deployment (admin
IP) and for configuring the VXLAN tunnel IPs (local_ip and remote_ip
parameters). How would I specify different IPs for these purposes?
(Recall that my hosts have a single NIC).<br>
<br>
<br>
In any case, native traffic on bridge br-ex is sent via br-ex.1
(VLAN 1), which is also the reason the Neutron gateway port qg-XXX
needs to be an access port for VLAN 1 (tag: 1). VXLAN traffic is
sent from br-ex.12 on all compute nodes. See the 2 cases below:<br>
<br>
<br>
Case 1. Max-size ping from compute node 'langrain' (192.168.1.14) to
another host on same LAN<br>
=> Native traffic sent from br-ex.1; no traffic sent
from br-ex.12<br>
<br>
[fn@langrain ~]$ ping -M do -s 1472 -c 1 192.168.1.54<br>
PING 192.168.1.54 (192.168.1.54) 1472(1500) bytes of data.<br>
1480 bytes from 192.168.1.54: icmp_seq=1 ttl=64 time=0.766 ms<br>
<br>
[root@langrain ~]# tcpdump -n -i br-ex.1 dst 192.168.1.54<br>
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode<br>
listening on br-ex.1, link-type EN10MB (Ethernet), capture size
65535 bytes<br>
10:32:37.666572 IP 192.168.1.14 > 192.168.1.54: ICMP echo
request, id 10432, seq 1, length 1480<br>
10:32:42.673665 ARP, Request who-has 192.168.1.54 tell 192.168.1.14,
length 28<br>
<br>
<br>
Case 2: Max-size ping from a guest1 (10.0.0.1) on compute node
'langrain' (192.168.1.14)<br>
to a guest2 (10.0.0.3) on another compute node
(192.168.1.21) via VXLAN tunnel.<br>
Guests are on the same virtual network 10.0.0.0/24<br>
=> Encapsulated traffic sent from br-ex.12; no
traffic sent from br-ex.1<br>
<br>
$ ping -M do -s 1472 -c 1 10.0.0.3<br>
PING 10.0.0.3 (10.0.0.3) 1472(1500) bytes of data.<br>
1480 bytes from 10.0.0.3: icmp_seq=1 ttl=64 time=2.22 ms<br>
<br>
[root@langrain ~]# tcpdump -n -i br-ex.12<br>
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode<br>
listening on br-ex.12, link-type EN10MB (Ethernet), capture size
65535 bytes<br>
<br>
11:02:56.916265 IP 192.168.1.14.47872 > 192.168.1.21.4789: VXLAN,
flags [I] (0x08), vni 10<br>
ARP, Request who-has 10.0.0.3 tell 10.0.0.1, length 28<br>
11:02:56.916991 IP 192.168.1.21.51408 > 192.168.1.14.4789: VXLAN,
flags [I] (0x08), vni 10<br>
ARP, Reply 10.0.0.3 is-at fa:16:3e:e6:e1:c8, length 28<br>
11:02:56.917282 IP 192.168.1.14.57836 > 192.168.1.21.4789: VXLAN,
flags [I] (0x08), vni 10<br>
IP 10.0.0.1 > 10.0.0.3: ICMP echo request, id 25474, seq 1,
length 1480<br>
11:02:56.918110 IP 192.168.1.21.44153 > 192.168.1.14.4789: VXLAN,
flags [I] (0x08), vni 10<br>
IP 10.0.0.3 > 10.0.0.1: ICMP echo reply, id 25474, seq 1, length
1480<br>
11:03:01.918885 IP 192.168.1.21.51408 > 192.168.1.14.4789: VXLAN,
flags [I] (0x08), vni 10<br>
ARP, Request who-has 10.0.0.1 tell 10.0.0.3, length 28<br>
11:03:01.919207 IP 192.168.1.14.57760 > 192.168.1.21.4789: VXLAN,
flags [I] (0x08), vni 10<br>
ARP, Reply 10.0.0.1 is-at fa:16:3e:f4:1d:89, length 28<br>
11:03:01.920502 ARP, Request who-has 192.168.1.14 tell 192.168.1.21,
length 46<br>
11:03:01.920519 ARP, Reply 192.168.1.14 is-at e0:3f:49:b4:7c:a7,
length 28<br>
<br>
<br>
<blockquote
cite="mid:CAPoubz6LgqZ8vi+ZNZLT0DrKqggFh0WfcecVwnB+zr2Y5nGVBg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>You may find that configuring a VLAN interface for
eth1.12 (not in a bridge, with a local address suitable
for communication with compute nodes, for VXLAN traffic)
and eth1.1 (in br-ex, for external traffic to use) does
better for you.<br>
</div>
</div>
</div>
</div>
</blockquote>
Hmm, I only have one NIC (eth0). In order to attach eth0 to br-ex,
I had to configure it as an OVSPort. <br>
Maybe I misunderstand your alternative, but are you suggesting to
configure eth0.1 as an OVSPort (connected to br-ex), and eth0.12
as a standalone interface? (Not sure a physical interface can be
"brain split" in such a way.)<br>
<br>
<blockquote
cite="mid:CAPoubz6LgqZ8vi+ZNZLT0DrKqggFh0WfcecVwnB+zr2Y5nGVBg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">I'm also not clear what your
Openstack API endpoint address or MTU is - maybe that's why
the eth1.1 interface is addressed?</div>
</div>
</div>
</blockquote>
It's 192.168.1.14, and br-ex.1 is always used for native traffic, so
the MTU is 1500.<br>
<br>
Note that my physical switch uses a native VLAN of 1 and is
configured with "Untag all ports" for VLAN 1. Moreover, OVSPort eth0
(attached to br-ex) is configured for VLAN trunking with a native
VLAN of 1 (vlan_mode: native-untagged, trunks: [1,12], tag: 1), so
within bridge br-ex, native packets are tagged 1.<br>
<br>
<blockquote
cite="mid:CAPoubz6LgqZ8vi+ZNZLT0DrKqggFh0WfcecVwnB+zr2Y5nGVBg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"> I can tell you that if you want
your API to be on the same address 192.168.1.14 as the VXLAN
tunnel endpoints then it has to be one address on one
interface and the two functions will share the same MTU -
almost certainly not what you're looking for.</div>
</div>
</div>
</blockquote>
With my current setup (thanks to policy routing), I have the same IP
on two interfaces br-ex.1 and br-ex.12, with MTUs 1500 and 1554,
respectively.<br>
<br>
<blockquote
cite="mid:CAPoubz6LgqZ8vi+ZNZLT0DrKqggFh0WfcecVwnB+zr2Y5nGVBg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"> If you source VXLAN packets from a
different IP address then you can put it on a different
interface and give it a different MTU - which appears to fit
what you want much better.<br>
</div>
</div>
</div>
</blockquote>
Selecting different compute host IPs for admin
(CONFIG_COMPUTE_HOSTS) and tunnel IPs would eliminate the need for
policy routing and is also more suitable for scaling a VXLAN
deployment across multiple independent L2 BC domains, but for that
I'll need to resolve point 3. above -- pointers in that direction
are much appreciated.<br>
<br>
Thanks,<br>
- Fredy<br>
<br>
</body>
</html>