[Openstack] Neutron Issues...
Erich Weiler
weiler at soe.ucsc.edu
Sat Apr 12 20:44:53 UTC 2014
I tried additionally checking the dnsmasq instance on the network node
while trying to ping it from the VM (I set the VMs IP statically to
10.200.0.6 and tried to ping dnsmasq which is 10.200.0.3). From the
network node, I did:
# ip netns
qdhcp-e4448083-ec61-4293-ad0e-62239986965f
[root at cloudnet-1 network-scripts]# ip netns exec
qdhcp-e4448083-ec61-4293-ad0e-62239986965f ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3184 (3.1 KiB) TX bytes:3184 (3.1 KiB)
tapc5195287-8f Link encap:Ethernet HWaddr FA:16:3E:C2:EC:B5
inet addr:10.200.0.3 Bcast:10.200.255.255 Mask:255.255.0.0
inet6 addr: fe80::f816:3eff:fec2:ecb5/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:2369 errors:0 dropped:0 overruns:0 frame:0
TX packets:2220 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:158390 (154.6 KiB) TX bytes:94804 (92.5 KiB)
# ip netns exec qdhcp-e4448083-ec61-4293-ad0e-62239986965f tcpdump -i
tapc5195287-8f
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tapc5195287-8f, link-type EN10MB (Ethernet), capture size
65535 bytes
13:38:03.997934 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:03.997948 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:03.999280 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:04.998033 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:04.998044 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:04.999187 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:05.999197 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:06.001111 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:06.001121 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:06.998171 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:06.998181 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:07.998279 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:07.998289 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:09.001444 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:09.001454 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:09.004258 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:09.998436 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:09.998446 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:10.004161 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:10.998488 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:10.998498 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:11.004181 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:12.001742 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:12.001752 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:12.998647 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:12.998657 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:13.998762 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:13.998772 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:14.009308 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:15.000801 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:15.000812 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:24.019260 ARP, Request who-has 10.200.0.1 tell 10.200.0.3, length 28
13:38:24.998348 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:34.999130 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:34.999143 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:36.003134 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:36.003147 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:36.999278 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:36.999291 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:37.999362 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:37.999375 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:39.003456 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:39.003469 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:40.003526 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:40.003539 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:41.003609 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:41.003622 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:42.003783 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:42.003796 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
13:38:43.003781 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
13:38:43.003794 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
Unknown), length 28
I assume because of the fact I'm seeing the ARP requests and replies on
the dnsmasq interface that VLANs are working, as they are supposed to be
talking on VLAN 200, but ARP isn't working for some reason, or at least
the VM isn't getting the ARP reply.
I tried setting my data interfaces on my network node and my compute
node to promiscuous mode (they weren't before), but it didn't seem to
help. Are those interfaces supposed to be in promiscuous mode?
On 4/12/14, 1:02 PM, Erich Weiler wrote:
> Hi Y'all,
>
> I'm making some progress on my neutron VLAN deployment issues, but it's
> still not working as expected. I have my compute nodes data port
> connected to a switchport that is a trunk, allowing VLANs 200-209 to
> flow over the trunk. The neutron node also has its internal data port
> on the same switch, on a trunk port, also allowing VLANs 200-209 to flow
> over it.
>
> I *think* I have OVS configured correctly on the neutron network node
> and the compute nodes.
>
> To test I just created an internal network (on physnet1) callled
> test-net on segmentation-id 200 (network 10.200.0.0/16). So I assume it
> will use VLAN 200. I then created a VM and attached it to that network.
> But when the VM boots, it does not get an IP address from dnsmasq.
>
> Back on the network node, I see dnsmasq running on that VLAN, I think:
>
> # ps -afe | grep masq
> nobody 16674 1 0 Apr10 ? 00:00:00 dnsmasq --no-hosts
> --no-resolv --strict-order --bind-interfaces --interface=tapc5195287-8f
> --except-interface=lo
> --pid-file=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/host
> --addn-hosts=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/addn_hosts
> --dhcp-optsfile=/var/lib/neutron/dhcp/e4448083-ec61-4293-ad0e-62239986965f/opts
> --leasefile-ro --dhcp-range=tag0,10.200.0.0,static,86400s
> --dhcp-lease-max=65536 --conf-file= --domain=openstacklocal
>
> And I tried doing a tcpdump on eth2 (which is the data interface with
> the VLANs) while doing a DHCPDISCOVER from the VM:
>
> # tcpdump -i eth2
> tcpdump: WARNING: eth2: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
> 12:43:46.440221 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.440563 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.440896 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.441414 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.441672 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.442001 STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.442502 STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.442830 STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.443171 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:46.443470 STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:47.718633 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP,
> Request from fa:16:3e:5d:11:5b (oui Unknown), length 280
> 12:43:47.719183 IP 10.200.0.3.bootps > 10.200.0.6.bootpc: BOOTP/DHCP,
> Reply, length 325
> 12:43:48.447761 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.448109 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.448593 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.448896 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.449266 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.449672 STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.450010 STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.450385 STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.450678 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:48.451115 STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:49.946348 VTPv1, Message Join message (0x04), length 166
> 12:43:50.459700 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.460184 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.460529 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.460867 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.461195 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.461625 STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.461956 STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.462335 STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.462745 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:50.463072 STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.455118 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.455591 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.455930 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.456258 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.456575 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.457053 STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.457414 STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.457702 STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.458171 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.458503 STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:52.718355 ARP, Request who-has 10.200.0.6 tell 10.200.0.3, length 28
> 12:43:53.718209 ARP, Request who-has 10.200.0.6 tell 10.200.0.3, length 28
> 12:43:54.460398 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.460871 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.461187 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.461588 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.461984 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.462361 STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.462652 STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.462981 STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.463474 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:43:54.463751 STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
>
> Where 10.200.0.3 is the IP address of the dnsmasq instance on the
> network node and 10.200.0.6 is the IP (supposedly) of the VM, from what
> nova tells me. It seems the traffic is getting to the network node, but
> the VM never gets the IP. Even if I configure the VM statically to be
> 10.200.0.6, I still can't ping the DHCP server on 10.200.0.3.
>
> It looks like ARP requests are getting sent out for the DHCP server hen
> I ping (as seen fro the network node):
>
> # tcpdump -i eth2
> tcpdump: WARNING: eth2: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
> 12:57:08.871232 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.871717 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.872063 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.872421 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.872819 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.873212 STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.873507 STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.873839 STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.874329 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:08.874621 STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:09.041413 VTPv1, Message Join message (0x04), length 166
> 12:57:09.289413 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 12:57:09.289435 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
> Unknown), length 28
> 12:57:10.289706 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 12:57:10.289728 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
> Unknown), length 28
> 12:57:10.876689 STP 802.1d, Config, Flags [none], bridge-id
> c0cd.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.877031 STP 802.1d, Config, Flags [none], bridge-id
> c0c9.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.877439 STP 802.1d, Config, Flags [none], bridge-id
> c0cc.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.877843 STP 802.1d, Config, Flags [none], bridge-id
> c0ca.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.878190 STP 802.1d, Config, Flags [none], bridge-id
> c0cb.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.878542 STP 802.1d, Config, Flags [none], bridge-id
> c0ce.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.878824 STP 802.1d, Config, Flags [none], bridge-id
> c0d0.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.879282 STP 802.1d, Config, Flags [none], bridge-id
> c0d1.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.879612 STP 802.1d, Config, Flags [none], bridge-id
> c0c8.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:10.879933 STP 802.1d, Config, Flags [none], bridge-id
> c0cf.c4:7d:4f:ec:d3:00.81a5, length 42
> 12:57:11.289582 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 12:57:11.289603 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
> Unknown), length 28
> 12:57:12.289668 ARP, Request who-has 10.200.0.3 tell 10.200.0.6, length 46
> 12:57:12.289689 ARP, Reply 10.200.0.3 is-at fa:16:3e:c2:ec:b5 (oui
> Unknown), length 28
>
> But the VM doesn't get any ping replies... Maybe ARP is having issues?
>
> Can anyone see anything wrong with this picture?
>
> Thanks!!
> -erich
More information about the Openstack
mailing list