[Openstack] Neutron Issues...
Erich Weiler
weiler at soe.ucsc.edu
Sat Apr 12 22:26:08 UTC 2014
And one more, I think I found something. On the network node, I see on
phy-br-eth2:
# tcpdump -n -e -vv -i phy-br-eth2
tcpdump: WARNING: phy-br-eth2: no IPv4 address assigned
tcpdump: listening on phy-br-eth2, link-type EN10MB (Ethernet), capture
size 65535 bytes
15:23:47.394584 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
(0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
15:23:47.394605 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
(0x8100), length 46: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4
(len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
15:23:48.394677 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
(0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
15:23:48.394693 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
(0x8100), length 46: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4
(len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
15:23:49.395364 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
(0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
15:23:49.395379 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
(0x8100), length 46: vlan 1, p 0, ethertype ARP, Ethernet (len 6), IPv4
(len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
It sees the ARP request on VLAN 200, but it looks like it replies on
VLAN 1. I'm not sure why it would reply on VLAN 1... I'm not entirely
sure how phy-br-eth2 works though...
On 4/12/14, 2:07 PM, Erich Weiler wrote:
> Sorry, one more... ;)
>
> Wile trying to ping the dnsmasq host (10.200.0.3) from the VM
> (statically configured to 10.200.0.6), if I tcpdump specifically on vlan
> 200 on the network node:
>
> # tcpdump -ni eth2 vlan 200
> tcpdump: WARNING: eth2: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
> 13:55:18.826384 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:55:18.956941 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 13:55:19.825947 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:55:20.826056 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:55:20.960779 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 13:55:21.826718 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:55:22.825870 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:55:22.976173 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 13:55:23.825964 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:55:24.826730 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:55:24.981795 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>
> I see the ARP requests coming in. But I don't see any replies, at least
> on vlan 200. If I tcpdump with a specific VLAN, I do see the reply on
> the network node:
>
> # tcpdump -i eth2
> tcpdump: WARNING: eth2: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
> 13:58:11.830842 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:58:11.830878 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
> Unknown), length 28
> 13:58:12.833434 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 13:58:12.833457 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
> Unknown), length 28
> 13:58:13.494885 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 13:58:13.495217 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 13:58:13.495577 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 13:58:13.496017 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 13:58:13.496355 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>
> So maybe the ARP reply is being sent out, but maybe just not over VLAN
> 200 where it needs to go, very strange...
>
> Also more verbose tcpdump on the network node:
>
> # tcpdump -n -e -vv -i eth2
>
> tcpdump: WARNING: eth2: no IPv4 address assigned
> tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 14:04:10.848783 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 14:04:10.848815 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
> 14:04:10.918013 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 205, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80cd.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.918374 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 201, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80c9.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.918705 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 204, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80cc.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.919171 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 202, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80ca.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.919525 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 203, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80cb.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.919889 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 206, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80ce.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.920346 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 208, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80d0.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.920682 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 209, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80d1.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.921010 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 200, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80c8.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:10.921295 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 207, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80cf.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:11.287103 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cc, 802.3, length
> 188: LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl
> 0x03: oui Cisco (0x00000c), pid VTP (0x2003): VTPv1, Message Join
> message (0x04), length 166
> Domain name: cbse, Rsvd: 0
> 14:04:11.848874 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 14:04:11.848896 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
> 14:04:12.851406 fa:16:3e:5d:11:5b > Broadcast, ethertype 802.1Q
> (0x8100), length 64: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
> IPv4 (len 4), Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 14:04:12.851428 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
> 14:04:12.923686 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 205, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80cd.00:13:1a:3d:3e:80, root-pathcost 9008
> 14:04:12.924039 88:43:e1:ff:76:c8 > 01:00:0c:cc:cc:cd, ethertype 802.1Q
> (0x8100), length 68: vlan 201, p 7, LLC, dsap SNAP (0xaa) Individual,
> ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid PVST
> (0x010b): STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> message-age 3.00s, max-age 20.00s, hello-time 2.00s,
> forwarding-delay 15.00s
> root-id 80c9.00:13:1a:3d:3e:80, root-pathcost 9008
>
> That one seems to indicate the ARP reply is being sent on vlan 200..??? :
>
> 14:04:12.851428 fa:16:3e:c2:ec:b5 > fa:16:3e:5d:11:5b, ethertype 802.1Q
> (0x8100), length 46: vlan 200, p 0, ethertype ARP, Ethernet (len 6),
> IPv4 (len 4), Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5, length 28
>
> I'm totally confused... ARP should be working according to that....
>
> On 4/12/14, 1:44 PM, Erich Weiler wrote:
>> I tried additionally checking the dnsmasq instance on the network node
>> while trying to ping it from the VM (I set the VMs IP statically to
>> 10.200.0.6 and tried to ping dnsmasq which is 10.200.0.3). From the
>> network node, I did:
>>
>> # ip netns
>> qdhcp-e4448083-ec61-4293-ad0e-62239986965f
>>
>> [root at cloudnet-1 network-scripts]# ip netns exec
>> qdhcp-e4448083-ec61-4293-ad0e-62239986965f ifconfig
>> lo Link encap:Local Loopback
>> inet addr:127.0.0.1 Mask:255.0.0.0
>> inet6 addr: ::1/128 Scope:Host
>> UP LOOPBACK RUNNING MTU:16436 Metric:1
>> RX packets:17 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:0
>> RX bytes:3184 (3.1 KiB) TX bytes:3184 (3.1 KiB)
>>
>> tapc5195287-8f Link encap:Ethernet HWaddr FA:16:3E:C2:EC:B5
>> inet addr:10.200.0.3 Bcast:10.200.255.255 Mask:255.255.0.0
>> inet6 addr: fe80::f816:3eff:fec2:ecb5/64 Scope:Link
>> UP BROADCAST RUNNING MTU:1500 Metric:1
>> RX packets:2369 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:2220 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:0
>> RX bytes:158390 (154.6 KiB) TX bytes:94804 (92.5 KiB)
>>
>> # ip netns exec qdhcp-e4448083-ec61-4293-ad0e-62239986965f tcpdump -i
>> tapc5195287-8f
>>
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> decode
>> listening on tapc5195287-8f, link-type EN10MB (Ethernet), capture size
>> 65535 bytes
>> 13:38:03.997934 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:03.997948 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:03.999280 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:04.998033 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:04.998044 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:04.999187 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:05.999197 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:06.001111 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:06.001121 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:06.998171 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:06.998181 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:07.998279 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:07.998289 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:09.001444 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:09.001454 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:09.004258 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:09.998436 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:09.998446 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:10.004161 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:10.998488 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:10.998498 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:11.004181 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:12.001742 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:12.001752 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:12.998647 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:12.998657 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:13.998762 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:13.998772 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:14.009308 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:15.000801 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:15.000812 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:24.019260 ARP, Request who-has 10.200.0.1 tell 10.200.0.3,
>> length 28
>> 13:38:24.998348 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:34.999130 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:34.999143 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:36.003134 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:36.003147 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:36.999278 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:36.999291 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:37.999362 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:37.999375 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:39.003456 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:39.003469 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:40.003526 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:40.003539 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:41.003609 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:41.003622 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:42.003783 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:42.003796 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>> 13:38:43.003781 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>> length 46
>> 13:38:43.003794 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>> Unknown), length 28
>>
>> I assume because of the fact I'm seeing the ARP requests and replies on
>> the dnsmasq interface that VLANs are working, as they are supposed to be
>> talking on VLAN 200, but ARP isn't working for some reason, or at least
>> the VM isn't getting the ARP reply.
>>
>> I tried setting my data interfaces on my network node and my compute
>> node to promiscuous mode (they weren't before), but it didn't seem to
>> help. Are those interfaces supposed to be in promiscuous mode?
>>
>>
>>
>> On 4/12/14, 1:02 PM, Erich Weiler wrote:
>>> Hi Y'all,
>>>
>>> I'm making some progress on my neutron VLAN deployment issues, but it's
>>> still not working as expected. I have my compute nodes data port
>>> connected to a switchport that is a trunk, allowing VLANs 200-209 to
>>> flow over the trunk. The neutron node also has its internal data port
>>> on the same switch, on a trunk port, also allowing VLANs 200-209 to flow
>>> over it.
>>>
>>> I *think* I have OVS configured correctly on the neutron network node
>>> and the compute nodes.
>>>
>>> To test I just created an internal network (on physnet1) callled
>>> test-net on segmentation-id 200 (network 10.200.0.0/16). So I assume it
>>> will use VLAN 200. I then created a VM and attached it to that network.
>>> But when the VM boots, it does not get an IP address from dnsmasq.
>>>
>>> Back on the network node, I see dnsmasq running on that VLAN, I think:
>>>
>>> # ps -afe | grep masq
>>> nobody 16674 1 0 Apr10 ? 00:00:00 dnsmasq --no-hosts
>>> --no-resolv --strict-order --bind-interfaces --interface=tapc5195287-8f
>>> --except-interface=lo
>>> --pid-file=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/pid
>>>
>>> --dhcp-hostsfile=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/host
>>>
>>>
>>> --addn-hosts=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/addn_hosts
>>>
>>>
>>> --dhcp-optsfile=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/opts
>>>
>>>
>>> --leasefile-ro --dhcp-range=tag0,10.200.0.0,static,86400s
>>> --dhcp-lease-max=65536 --conf-file= --domain=openstacklocal
>>>
>>> And I tried doing a tcpdump on eth2 (which is the data interface with
>>> the VLANs) while doing a DHCPDISCOVER from the VM:
>>>
>>> # tcpdump -i eth2
>>> tcpdump: WARNING: eth2: no IPv4 address assigned
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>> decode
>>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
>>> 12:43:46.440221 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.440563 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.440896 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.441414 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.441672 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.442001 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.442502 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.442830 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.443171 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:46.443470 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:47.718633 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP,
>>> Request from fa:16:3e:5d:11:5b (oui Unknown), length 280
>>> 12:43:47.719183 IP 10.200.0.3.bootps > 10.200.0.6.bootpc: BOOTP/DHCP,
>>> Reply, length 325
>>> 12:43:48.447761 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.448109 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.448593 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.448896 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.449266 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.449672 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.450010 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.450385 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.450678 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:48.451115 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:49.946348 VTPv1, Message Join message (0x04), length 166
>>> 12:43:50.459700 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.460184 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.460529 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.460867 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.461195 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.461625 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.461956 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.462335 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.462745 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:50.463072 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.455118 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.455591 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.455930 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.456258 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.456575 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.457053 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.457414 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.457702 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.458171 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.458503 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:52.718355 ARP, Request who-has 10.200.0.6 tell 10.200.0.3,
>>> length 28
>>> 12:43:53.718209 ARP, Request who-has 10.200.0.6 tell 10.200.0.3,
>>> length 28
>>> 12:43:54.460398 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.460871 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.461187 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.461588 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.461984 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.462361 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.462652 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.462981 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.463474 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:43:54.463751 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>>
>>> Where 10.200.0.3 is the IP address of the dnsmasq instance on the
>>> network node and 10.200.0.6 is the IP (supposedly) of the VM, from what
>>> nova tells me. It seems the traffic is getting to the network node, but
>>> the VM never gets the IP. Even if I configure the VM statically to be
>>> 10.200.0.6, I still can't ping the DHCP server on 10.200.0.3.
>>>
>>> It looks like ARP requests are getting sent out for the DHCP server hen
>>> I ping (as seen fro the network node):
>>>
>>> # tcpdump -i eth2
>>> tcpdump: WARNING: eth2: no IPv4 address assigned
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>> decode
>>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
>>> 12:57:08.871232 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.871717 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.872063 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.872421 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.872819 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.873212 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.873507 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.873839 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.874329 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:08.874621 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:09.041413 VTPv1, Message Join message (0x04), length 166
>>> 12:57:09.289413 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>> length 46
>>> 12:57:09.289435 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>> Unknown), length 28
>>> 12:57:10.289706 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>> length 46
>>> 12:57:10.289728 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>> Unknown), length 28
>>> 12:57:10.876689 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.877031 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.877439 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.877843 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.878190 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.878542 STP 802.1d, Config, Flags [none], bridge-id
>>> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.878824 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.879282 STP 802.1d, Config, Flags [none], bridge-id
>>> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.879612 STP 802.1d, Config, Flags [none], bridge-id
>>> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:10.879933 STP 802.1d, Config, Flags [none], bridge-id
>>> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>>> 12:57:11.289582 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>> length 46
>>> 12:57:11.289603 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>> Unknown), length 28
>>> 12:57:12.289668 ARP, Request who-has 10.200.0.3 tell 10.200.0.6,
>>> length 46
>>> 12:57:12.289689 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
>>> Unknown), length 28
>>>
>>> But the VM doesn't get any ping replies... Maybe ARP is having issues?
>>>
>>> Can anyone see anything wrong with this picture?
>>>
>>> Thanks!!
>>> -erich
More information about the Openstack
mailing list