[Openstack] Neutron Issues...
Erich Weiler
weiler at soe.ucsc.edu
Tue Apr 15 23:55:33 UTC 2014
I think I've identified a problem - when I create a new network and
subnet via [neutron net-create and neutron subnet-create], it creates
the networks fine, and then I start the dhcp service on the network node:
/etc/init.d/neutron-dhcp-agent start
It seems to add a tap interface to br-int:
# ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000c2864f59274e
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC
SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
1(tapbd844c9f-90): addr:4a:01:00:00:00:00 <------ ADDS THIS
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
4(int-br-eth2): addr:52:76:9c:cb:4a:47
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
5(int-br-ex): addr:2e:d8:0d:0b:b9:e8
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
LOCAL(br-int): addr:c2:86:4f:59:27:4e
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
But as you note, the link is down. I would expect a corresponding
interface called "tapbd844c9f-90" be created when I do a simple
"ifconfig" on the network node, but it's not there:
# ifconfig
br-eth2 Link encap:Ethernet HWaddr 00:14:4F:CB:26:F0
inet6 addr: fe80::68a6:5ff:fe23:18e8/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:354 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:27568 (26.9 KiB) TX bytes:468 (468.0 b)
br-ex Link encap:Ethernet HWaddr 00:14:4F:CB:26:EF
inet6 addr: fe80::214:4fff:fecb:26ef/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:589 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:40950 (39.9 KiB) TX bytes:936 (936.0 b)
br-int Link encap:Ethernet HWaddr C2:86:4F:59:27:4E
inet6 addr: fe80::58ad:bff:feb6:2a38/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:369 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:28334 (27.6 KiB) TX bytes:468 (468.0 b)
eth0 Link encap:Ethernet HWaddr 00:14:4F:CB:26:EE
inet addr:10.1.1.145 Bcast:10.1.255.255 Mask:255.255.0.0
inet6 addr: fe80::214:4fff:fecb:26ee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12610 errors:0 dropped:0 overruns:0 frame:0
TX packets:9659 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1431159 (1.3 MiB) TX bytes:2084840 (1.9 MiB)
Memory:fab60000-fab80000
eth1 Link encap:Ethernet HWaddr 00:14:4F:CB:26:EF
inet6 addr: fe80::214:4fff:fecb:26ef/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:3422 errors:0 dropped:0 overruns:0 frame:0
TX packets:59 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:205320 (200.5 KiB) TX bytes:9464 (9.2 KiB)
Memory:fabe0000-fac00000
eth2 Link encap:Ethernet HWaddr 00:14:4F:CB:26:F0
inet6 addr: fe80::214:4fff:fecb:26f0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:25018 errors:0 dropped:0 overruns:0 frame:0
TX packets:344 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1801516 (1.7 MiB) TX bytes:19639 (19.1 KiB)
Memory:fac60000-fac80000
int-br-eth2 Link encap:Ethernet HWaddr 52:76:9C:CB:4A:47
inet6 addr: fe80::5076:9cff:fecb:4a47/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:468 (468.0 b) TX bytes:468 (468.0 b)
int-br-ex Link encap:Ethernet HWaddr 2E:D8:0D:0B:B9:E8
inet6 addr: fe80::2cd8:dff:fe0b:b9e8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:21 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1368 (1.3 KiB) TX bytes:468 (468.0 b)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
phy-br-eth2 Link encap:Ethernet HWaddr CA:37:ED:2E:1A:57
inet6 addr: fe80::c837:edff:fe2e:1a57/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:468 (468.0 b) TX bytes:468 (468.0 b)
phy-br-ex Link encap:Ethernet HWaddr 22:F0:60:46:26:FD
inet6 addr: fe80::20f0:60ff:fe46:26fd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:468 (468.0 b) TX bytes:1368 (1.3 KiB)
I think becasue it's not there, that's why "ovs-ofctl show br-int" shows
the link is down. Does anyone know why it doesn't create that
interface? Or if I can create it manually...?
On 04/14/14 14:09, Erich Weiler wrote:
> I think I'm getting a little closer.. I see that on my neutron network
> node my DHCP server (dnsmasq) is on 10.200.0.3:
>
> # ip netns
> qrouter-eefaa5c8-95fc-4b3d-a8d2-27eebc449337
> qdhcp-e4448083-ec61-4293-ad0e-62239986965f
>
> # ip netns exec qdhcp-e4448083-ec61-4293-ad0e-62239986965f ifconfig
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:7 errors:0 dropped:0 overruns:0 frame:0
> TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:1158 (1.1 KiB) TX bytes:1158 (1.1 KiB)
>
> tapc5195287-8f Link encap:Ethernet HWaddr FA:16:3E:C2:EC:B5
> inet addr:10.200.0.3 Bcast:10.200.255.255 Mask:255.255.0.0
> inet6 addr: fe80::f816:3eff:fec2:ecb5/64 Scope:Link
> UP BROADCAST RUNNING MTU:1500 Metric:1
> RX packets:10769 errors:0 dropped:0 overruns:0 frame:0
> TX packets:10799 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:647044 (631.8 KiB) TX bytes:454965 (444.3 KiB)
>
> But when I look at my OVS state on the same neutron node:
>
> # ovs-ofctl show br-int
>
> OFPT_FEATURES_REPLY (xid=0x2): dpid:0000c2864f59274e
> n_tables:254, n_buffers:256
> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC
> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
> 1(int-br-eth2): addr:0a:0d:73:7f:91:8f
> config: 0
> state: 0
> current: 10GB-FD COPPER
> speed: 10000 Mbps now, 0 Mbps max
> 2(int-br-ex): addr:d6:10:30:bf:67:15
> config: 0
> state: 0
> current: 10GB-FD COPPER
> speed: 10000 Mbps now, 0 Mbps max
> 3(qr-b87526f1-3d): addr:c6:25:03:f0:51:08
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> 4(tap5a3a1ef2-9c): addr:3e:63:c2:19:54:6a
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> 5(tapc5195287-8f): addr:af:00:00:00:00:00 <--- HUH??
> config: PORT_DOWN
> state: LINK_DOWN
> speed: 0 Mbps now, 0 Mbps max
> LOCAL(br-int): addr:c2:86:4f:59:27:4e
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
>
> It shows the tapc5195287-8f port, the one dnsmasq is on, is PORT_DOWN,
> LINK_DOWN. I wonder why that is? The earlier "ip netns ifconfig" on
> the dnsmasq port showed status "UP". Anyone see anything I'm missing?
>
> Thanks,
> erich
>
> On 04/14/14 11:30, Erich Weiler wrote:
>> And additionally here's my "iptables -L" on the compute node with the VM
>> running on it, if anyone can see anything wrong with it that might block
>> ARP responses... I didn't modify this manually, looks like Open vSwitch
>> did all the tweaking. Remember 10.200.0.3 is the dnsmasq server and
>> 10.200.0.6 is the VM:
>>
>> # iptables -L
>> Chain INPUT (policy ACCEPT)
>> target prot opt source destination
>> neutron-openvswi-INPUT all -- anywhere anywhere
>> ACCEPT udp -- anywhere anywhere udp
>> dpt:domain
>> ACCEPT tcp -- anywhere anywhere tcp
>> dpt:domain
>> ACCEPT udp -- anywhere anywhere udp
>> dpt:bootps
>> ACCEPT tcp -- anywhere anywhere tcp
>> dpt:bootps
>>
>> Chain FORWARD (policy ACCEPT)
>> target prot opt source destination
>> neutron-filter-top all -- anywhere anywhere
>> neutron-openvswi-FORWARD all -- anywhere anywhere
>> ACCEPT all -- anywhere 192.168.122.0/24 state
>> RELATED,ESTABLISHED
>> ACCEPT all -- 192.168.122.0/24 anywhere
>> ACCEPT all -- anywhere anywhere
>> REJECT all -- anywhere anywhere reject-with
>> icmp-port-unreachable
>> REJECT all -- anywhere anywhere reject-with
>> icmp-port-unreachable
>>
>> Chain OUTPUT (policy ACCEPT)
>> target prot opt source destination
>> neutron-filter-top all -- anywhere anywhere
>> neutron-openvswi-OUTPUT all -- anywhere anywhere
>>
>> Chain neutron-filter-top (2 references)
>> target prot opt source destination
>> neutron-openvswi-local all -- anywhere anywhere
>>
>> Chain neutron-openvswi-FORWARD (1 references)
>> target prot opt source destination
>> neutron-openvswi-sg-chain all -- anywhere anywhere
>> PHYSDEV match --physdev-out tapb2c5d867-34 --physdev-is-bridged
>> neutron-openvswi-sg-chain all -- anywhere anywhere
>> PHYSDEV match --physdev-in tapb2c5d867-34 --physdev-is-bridged
>>
>> Chain neutron-openvswi-INPUT (1 references)
>> target prot opt source destination
>> neutron-openvswi-ob2c5d867-3 all -- anywhere anywhere
>> PHYSDEV match --physdev-in tapb2c5d867-34 --physdev-is-bridged
>>
>> Chain neutron-openvswi-OUTPUT (1 references)
>> target prot opt source destination
>>
>> Chain neutron-openvswi-ib2c5d867-3 (1 references)
>> target prot opt source destination
>> DROP all -- anywhere anywhere state
>> INVALID
>> RETURN all -- anywhere anywhere state
>> RELATED,ESTABLISHED
>> RETURN udp -- 10.200.0.3 anywhere udp
>> spt:bootps dpt:bootpc
>> neutron-openvswi-sg-fallback all -- anywhere anywhere
>>
>> Chain neutron-openvswi-local (1 references)
>> target prot opt source destination
>>
>> Chain neutron-openvswi-ob2c5d867-3 (2 references)
>> target prot opt source destination
>> RETURN udp -- anywhere anywhere udp
>> spt:bootpc dpt:bootps
>> neutron-openvswi-sb2c5d867-3 all -- anywhere anywhere
>> DROP udp -- anywhere anywhere udp
>> spt:bootps dpt:bootpc
>> DROP all -- anywhere anywhere state
>> INVALID
>> RETURN all -- anywhere anywhere state
>> RELATED,ESTABLISHED
>> RETURN all -- anywhere anywhere
>> neutron-openvswi-sg-fallback all -- anywhere anywhere
>>
>> Chain neutron-openvswi-sb2c5d867-3 (1 references)
>> target prot opt source destination
>> RETURN all -- 10.200.0.6 anywhere MAC
>> FA:16:3E:5D:11:5B
>> DROP all -- anywhere anywhere
>>
>> Chain neutron-openvswi-sg-chain (2 references)
>> target prot opt source destination
>> neutron-openvswi-ib2c5d867-3 all -- anywhere anywhere
>> PHYSDEV match --physdev-out tapb2c5d867-34 --physdev-is-bridged
>> neutron-openvswi-ob2c5d867-3 all -- anywhere anywhere
>> PHYSDEV match --physdev-in tapb2c5d867-34 --physdev-is-bridged
>> ACCEPT all -- anywhere anywhere
>>
>> Chain neutron-openvswi-sg-fallback (2 references)
>> target prot opt source destination
>> DROP all -- anywhere anywhere
>>
>>
>> On 04/14/14 11:06, Erich Weiler wrote:
>>> No one can see anything wrong?
>>>
>>> Here's "ovs-vsctl show" on the neutron network node, if it helps:
>>>
>>> # ovs-vsctl show
>>> 52702cef-6433-4627-ade8-51561b4e8126
>>> Bridge "br-eth2"
>>> Port "phy-br-eth2"
>>> Interface "phy-br-eth2"
>>> Port "eth2"
>>> Interface "eth2"
>>> Port "br-eth2"
>>> Interface "br-eth2"
>>> type: internal
>>> Bridge br-int
>>> Port int-br-ex
>>> Interface int-br-ex
>>> Port br-int
>>> Interface br-int
>>> type: internal
>>> Port "qr-b87526f1-3d"
>>> tag: 4095
>>> Interface "qr-b87526f1-3d"
>>> type: internal
>>> Port "int-br-eth2"
>>> Interface "int-br-eth2"
>>> Port "tapc5195287-8f"
>>> tag: 1
>>> Interface "tapc5195287-8f"
>>> type: internal
>>> Port "tap5a3a1ef2-9c"
>>> tag: 4095
>>> Interface "tap5a3a1ef2-9c"
>>> type: internal
>>> Bridge br-ex
>>> Port phy-br-ex
>>> Interface phy-br-ex
>>> Port "qg-0b1ba3a7-16"
>>> Interface "qg-0b1ba3a7-16"
>>> type: internal
>>> Port "eth1"
>>> Interface "eth1"
>>> Port br-ex
>>> Interface br-ex
>>> type: internal
>>> ovs_version: "1.11.0"
>>>
>>> On 04/12/14 15:26, Erich Weiler wrote:
>>>> And one more, I think I found something. On the network node, I see on
>>>> phy-br-eth2:
>>>>
>>>> # tcpdump -n -e -vv -i phy-br-eth2
>>>>
>>>> tcpdump: WARNING: phy-br-eth2: no IPv4 address assigned
>>>> tcpdump: listening on phy-br-eth2, link-type EN10MB (Ethernet), capture
>>>> size 65535 bytes
>>>> 15:23:47.394584 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
>>>> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
>>>> 15:23:47.394605 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
>>>> (0x8100), length 46: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4
>>>> (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>>>> 15:23:48.394677 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
>>>> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
>>>> 15:23:48.394693 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
>>>> (0x8100), length 46: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4
>>>> (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>>>> 15:23:49.395364 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
>>>> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
>>>> 15:23:49.395379 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
>>>> (0x8100), length 46: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4
>>>> (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>>>>
>>>> It sees the ARP request on VLAN 200, but it looks like it replies on
>>>> VLAN 1. I'm not sure why it would reply on VLAN 1... I'm not entirely
>>>> sure how phy-br-eth2 works though...
>>>>
>>>> On 4/12/14, 2:07 PM, Erich Weiler wrote:
>>>>> Sorry, one more... ;)
>>>>>
>>>>> Wile trying to ping the dnsmasq host (10.200.0.3) from the VM
>>>>> (statically configured to 10.200.0.6), if I tcpdump specifically on
>>>>> vlan
>>>>> 200 on the network node:
>>>>>
>>>>> # tcpdump -ni eth2 vlan 200
>>>>> tcpdump: WARNING: eth2: no IPv4 address assigned
>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>> decode
>>>>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535
>>>>> bytes
>>>>> 13:55:18.826384 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:55:18.956941 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> 13:55:19.825947 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:55:20.826056 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:55:20.960779 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> 13:55:21.826718 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:55:22.825870 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:55:22.976173 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> 13:55:23.825964 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:55:24.826730 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:55:24.981795 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>
>>>>> I see the ARP requests coming in. But I don't see any replies, at
>>>>> least
>>>>> on vlan 200. If I tcpdump with a specific VLAN, I do see the reply on
>>>>> the network node:
>>>>>
>>>>> # tcpdump -i eth2
>>>>> tcpdump: WARNING: eth2: no IPv4 address assigned
>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>> decode
>>>>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535
>>>>> bytes
>>>>> 13:58:11.830842 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:58:11.830878 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>> Unknown), length 28
>>>>> 13:58:12.833434 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>> length 46
>>>>> 13:58:12.833457 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>> Unknown), length 28
>>>>> 13:58:13.494885 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> 13:58:13.495217 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> 13:58:13.495577 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> 13:58:13.496017 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> 13:58:13.496355 STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>
>>>>> So maybe the ARP reply is being sent out, but maybe just not over VLAN
>>>>> 200 where it needs to go, very strange...
>>>>>
>>>>> Also more verbose tcpdump on the network node:
>>>>>
>>>>> # tcpdump -n -e -vv -i eth2
>>>>>
>>>>> tcpdump: WARNING: eth2: no IPv4 address assigned
>>>>> tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size
>>>>> 65535 bytes
>>>>> 14:04:10.848783 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
>>>>> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>>> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
>>>>> 14:04:10.848815 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>>> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>>>>> 14:04:10.918013 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 205, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80cd.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.918374 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 201, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80c9.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.918705 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 204, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80cc.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.919171 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 202, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80ca.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.919525 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 203, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80cb.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.919889 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 206, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80ce.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.920346 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 208, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80d0.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.920682 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 209, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80d1.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.921010 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 200, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80c8.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:10.921295 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 207, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80cf.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:11.287103 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cc, 802.3, length
>>>>> 188: LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl
>>>>> 0x03: oui Cisco (0x00000c), pid VTP (0x2003): VTPv1, Message Join
>>>>> message (0x04), length 166
>>>>> Domain name: cbse, Rsvd: 0
>>>>> 14:04:11.848874 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
>>>>> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>>> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
>>>>> 14:04:11.848896 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>>> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>>>>> 14:04:12.851406 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
>>>>> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>>> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
>>>>> 14:04:12.851428 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>>> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>>>>> 14:04:12.923686 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 205, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80cd.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>> 14:04:12.924039 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 68: vlan 201, p 7, LLC, dsap SNAP (0xaa) Individual,
>>>>> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
>>>>> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
>>>>> forwarding-delay 15.00s
>>>>> root-id 80c9.00:13:1a:3d:3e:80, root-pathcost 9008
>>>>>
>>>>> That one seems to indicate the ARP reply is being sent on vlan
>>>>> 200..??? :
>>>>>
>>>>> 14:04:12.851428 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype
>>>>> 802.1Q
>>>>> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
>>>>> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>>>>>
>>>>> I'm totally confused... ARP should be working according to that....
>>>>>
>>>>> On 4/12/14, 1:44 PM, Erich Weiler wrote:
>>>>>> I tried additionally checking the dnsmasq instance on the network
>>>>>> node
>>>>>> while trying to ping it from the VM (I set the VMs IP statically to
>>>>>> 10.200.0.6 and tried to ping dnsmasq which is 10.200.0.3). From the
>>>>>> network node, I did:
>>>>>>
>>>>>> # ip netns
>>>>>> qdhcp-e4448083-ec61-4293-ad0e-62239986965f
>>>>>>
>>>>>> [root at cloudnet-1 network-scripts]# ip netns exec
>>>>>> qdhcp-e4448083-ec61-4293-ad0e-62239986965f ifconfig
>>>>>> lo Link encap:Local Loopback
>>>>>> inet addr:127.0.0.1 Mask:255.0.0.0
>>>>>> inet6 addr: ::1/128 Scope:Host
>>>>>> UP LOOPBACK RUNNING MTU:16436 Metric:1
>>>>>> RX packets:17 errors:0 dropped:0 overruns:0 frame:0
>>>>>> TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
>>>>>> collisions:0 txqueuelen:0
>>>>>> RX bytes:3184 (3.1 KiB) TX bytes:3184 (3.1 KiB)
>>>>>>
>>>>>> tapc5195287-8f Link encap:Ethernet HWaddr FA:16:3E:C2:EC:B5
>>>>>> inet addr:10.200.0.3 Bcast:10.200.255.255
>>>>>> Mask:255.255.0.0
>>>>>> inet6 addr: fe80::f816:3eff:fec2:ecb5/64 Scope:Link
>>>>>> UP BROADCAST RUNNING MTU:1500 Metric:1
>>>>>> RX packets:2369 errors:0 dropped:0 overruns:0 frame:0
>>>>>> TX packets:2220 errors:0 dropped:0 overruns:0 carrier:0
>>>>>> collisions:0 txqueuelen:0
>>>>>> RX bytes:158390 (154.6 KiB) TX bytes:94804 (92.5 KiB)
>>>>>>
>>>>>> # ip netns exec qdhcp-e4448083-ec61-4293-ad0e-62239986965f tcpdump -i
>>>>>> tapc5195287-8f
>>>>>>
>>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>>> decode
>>>>>> listening on tapc5195287-8f, link-type EN10MB (Ethernet), capture
>>>>>> size
>>>>>> 65535 bytes
>>>>>> 13:38:03.997934 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:03.997948 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:03.999280 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:04.998033 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:04.998044 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:04.999187 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:05.999197 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:06.001111 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:06.001121 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:06.998171 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:06.998181 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:07.998279 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:07.998289 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:09.001444 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:09.001454 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:09.004258 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:09.998436 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:09.998446 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:10.004161 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:10.998488 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:10.998498 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:11.004181 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:12.001742 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:12.001752 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:12.998647 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:12.998657 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:13.998762 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:13.998772 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:14.009308 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:15.000801 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:15.000812 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:24.019260 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>>>>>> length 28
>>>>>> 13:38:24.998348 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:34.999130 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:34.999143 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:36.003134 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:36.003147 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:36.999278 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:36.999291 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:37.999362 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:37.999375 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:39.003456 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:39.003469 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:40.003526 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:40.003539 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:41.003609 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:41.003622 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:42.003783 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:42.003796 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>> 13:38:43.003781 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>> length 46
>>>>>> 13:38:43.003794 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>> Unknown), length 28
>>>>>>
>>>>>> I assume because of the fact I'm seeing the ARP requests and
>>>>>> replies on
>>>>>> the dnsmasq interface that VLANs are working, as they are supposed
>>>>>> to be
>>>>>> talking on VLAN 200, but ARP isn't working for some reason, or at
>>>>>> least
>>>>>> the VM isn't getting the ARP reply.
>>>>>>
>>>>>> I tried setting my data interfaces on my network node and my compute
>>>>>> node to promiscuous mode (they weren't before), but it didn't seem to
>>>>>> help. Are those interfaces supposed to be in promiscuous mode?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 4/12/14, 1:02 PM, Erich Weiler wrote:
>>>>>>> Hi Y'all,
>>>>>>>
>>>>>>> I'm making some progress on my neutron VLAN deployment issues, but
>>>>>>> it's
>>>>>>> still not working as expected. I have my compute nodes data port
>>>>>>> connected to a switchport that is a trunk, allowing VLANs 200-209 to
>>>>>>> flow over the trunk. The neutron node also has its internal data
>>>>>>> port
>>>>>>> on the same switch, on a trunk port, also allowing VLANs 200-209 to
>>>>>>> flow
>>>>>>> over it.
>>>>>>>
>>>>>>> I *think* I have OVS configured correctly on the neutron network
>>>>>>> node
>>>>>>> and the compute nodes.
>>>>>>>
>>>>>>> To test I just created an internal network (on physnet1) callled
>>>>>>> test-net on segmentation-id 200 (network 10.200.0.0/16). So I
>>>>>>> assume it
>>>>>>> will use VLAN 200. I then created a VM and attached it to that
>>>>>>> network.
>>>>>>> But when the VM boots, it does not get an IP address from dnsmasq.
>>>>>>>
>>>>>>> Back on the network node, I see dnsmasq running on that VLAN, I
>>>>>>> think:
>>>>>>>
>>>>>>> # ps -afe | grep masq
>>>>>>> nobody 16674 1 0 Apr10 ? 00:00:00 dnsmasq --no-hosts
>>>>>>> --no-resolv --strict-order --bind-interfaces
>>>>>>> --interface=tapc5195287-8f
>>>>>>> --except-interface=lo
>>>>>>> --pid-file=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/pid
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --dhcp-hostsfile=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/host
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --addn-hosts=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/addn_hosts
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --dhcp-optsfile=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/opts
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --leasefile-ro --dhcp-range=tag0,10.200.0.0,static,86400s
>>>>>>> --dhcp-lease-max=65536 --conf-file= --domain=openstacklocal
>>>>>>>
>>>>>>> And I tried doing a tcpdump on eth2 (which is the data interface
>>>>>>> with
>>>>>>> the VLANs) while doing a DHCPDISCOVER from the VM:
>>>>>>>
>>>>>>> # tcpdump -i eth2
>>>>>>> tcpdump: WARNING: eth2: no IPv4 address assigned
>>>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>>>> decode
>>>>>>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535
>>>>>>> bytes
>>>>>>> 12:43:46.440221 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.440563 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.440896 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.441414 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.441672 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.442001 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.442502 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.442830 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.443171 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:46.443470 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:47.718633 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
>>>>>>> BOOTP/DHCP,
>>>>>>> Request from fa:16:3e:5d:11:5b (oui Unknown), length 280
>>>>>>> 12:43:47.719183 IP 10.200.0.3.bootps > 10.200.0.6.bootpc:
>>>>>>> BOOTP/DHCP,
>>>>>>> Reply, length 325
>>>>>>> 12:43:48.447761 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.448109 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.448593 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.448896 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.449266 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.449672 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.450010 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.450385 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.450678 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:48.451115 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:49.946348 VTPv1, Message Join message (0x04), length 166
>>>>>>> 12:43:50.459700 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.460184 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.460529 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.460867 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.461195 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.461625 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.461956 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.462335 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.462745 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:50.463072 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.455118 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.455591 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.455930 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.456258 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.456575 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.457053 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.457414 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.457702 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.458171 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.458503 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:52.718355 ARP, Request who-has 10.200.0.6 tell 10.200.0.3,
>>>>>>> length 28
>>>>>>> 12:43:53.718209 ARP, Request who-has 10.200.0.6 tell 10.200.0.3,
>>>>>>> length 28
>>>>>>> 12:43:54.460398 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.460871 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.461187 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.461588 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.461984 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.462361 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.462652 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.462981 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.463474 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:43:54.463751 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>>
>>>>>>> Where 10.200.0.3 is the IP address of the dnsmasq instance on the
>>>>>>> network node and 10.200.0.6 is the IP (supposedly) of the VM, from
>>>>>>> what
>>>>>>> nova tells me. It seems the traffic is getting to the network node,
>>>>>>> but
>>>>>>> the VM never gets the IP. Even if I configure the VM statically
>>>>>>> to be
>>>>>>> 10.200.0.6, I still can't ping the DHCP server on 10.200.0.3.
>>>>>>>
>>>>>>> It looks like ARP requests are getting sent out for the DHCP server
>>>>>>> hen
>>>>>>> I ping (as seen fro the network node):
>>>>>>>
>>>>>>> # tcpdump -i eth2
>>>>>>> tcpdump: WARNING: eth2: no IPv4 address assigned
>>>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>>>> decode
>>>>>>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535
>>>>>>> bytes
>>>>>>> 12:57:08.871232 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.871717 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.872063 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.872421 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.872819 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.873212 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.873507 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.873839 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.874329 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:08.874621 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:09.041413 VTPv1, Message Join message (0x04), length 166
>>>>>>> 12:57:09.289413 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>>> length 46
>>>>>>> 12:57:09.289435 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>>> Unknown), length 28
>>>>>>> 12:57:10.289706 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>>> length 46
>>>>>>> 12:57:10.289728 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>>> Unknown), length 28
>>>>>>> 12:57:10.876689 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.877031 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.877439 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.877843 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.878190 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.878542 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.878824 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.879282 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.879612 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:10.879933 STP 802.1d, Config, Flags [none], bridge-id
>>>>>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>>>>> 12:57:11.289582 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>>> length 46
>>>>>>> 12:57:11.289603 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>>> Unknown), length 28
>>>>>>> 12:57:12.289668 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>>>>>> length 46
>>>>>>> 12:57:12.289689 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>>>>>> Unknown), length 28
>>>>>>>
>>>>>>> But the VM doesn't get any ping replies... Maybe ARP is having
>>>>>>> issues?
>>>>>>>
>>>>>>> Can anyone see anything wrong with this picture?
>>>>>>>
>>>>>>> Thanks!!
>>>>>>> -erich
More information about the Openstack
mailing list