Open Stack

try this

http://2014.texaslinuxfest.org/sites/default/files/HopkinsPPTdeck.pdf

On Tue, Sep 22, 2015 at 1:16 PM, Sam Stoelinga <sammiestoel at gmail.com>
wrote:

> Hi all,
>
> Would appreciate your networking expertise.
>
> I have a 7 node environment, 3 controllers which run also as network node
> and 4 compute nodes. There are 2 compute nodes which are behaving correctly
> both sending and receiving packets, but the other 2 compute nodes can sent
> broadcasts such as ARP request and DHCP requests but the responses is not
> received. Also note that previously this was working but out of nowhere
> this issue started happening.
>
> I've debugged this using tcpdump on both the compute node and the
> controller node. The VM's DHCP request gets successfully sent to the
> controller node and the controller node responds with a dhcp response. It
> also get's successfully sent out of eth3 (vlan trunk), but this dhcp
> response never arrives on the compute node. So then I tried to login via
> VNC and manually set the IP and then try pinging, the result was outgoing
> arp request seen by all nodes but the arp response was not received by the
> compute node.
>
> I've tried disabling hardware offloading as I thought this may be the nic
> discarding packets, but that didn't help. I have spent about a day
> debugging with tcpdump but running out of clueues. Anybody with a similar
> experience? It's weird that the node can sent out packets seen by other
> nodes but it can not receive the responses from other nodes.
>
> tcpdump on compute which isn't receiving packets
> tcpdump -i eth3 -nnNs 512
> tcpdump: WARNING: eth3: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth3, link-type EN10MB (Ethernet), capture size 512 bytes
> 08:02:02.232749 STP 802.1s, Rapid STP, CIST Flags [Learn, Forward,
> Agreement], length 102
> 08:02:03.990588 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request
> from fa:16:3e:0d:26:a5 (this is a VM), length 300
> 08:02:04.232797 STP 802.1s, Rapid STP, CIST Flags [Learn, Forward,
> Agreement], length 102
> 08:02:11.393997 LLDP, length 318: HP
> 08:02:12.232627 STP 802.1s, Rapid STP, CIST Flags [Learn, Forward,
> Agreement], length 102
> 08:02:13.118051 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request
> from fa:16:3e:0d:26:a5, length 300
> 08:02:14.232638 STP 802.1s, Rapid STP, CIST Flags [Learn, Forward,
> Agreement], length 102
>
> tcpdump on controller
> ip netns exec qdhcp-878e1f0a-abba-4637-8afd-2814a38136a5 tcpdump -i
> tape2f5fa09-19 -nnNs 512
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on tape2f5fa09-19, link-type EN10MB (Ethernet), capture size 512
> bytes
> 08:01:32.591906 ARP, Request who-has 192.168.111.122 tell 192.168.111.3,
> length 28
> 08:01:33.033029 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request
> from fa:16:3e:0d:26:a5, length 300
> 08:01:33.033327 IP 192.168.111.3.67 > 192.168.111.122.68: BOOTP/DHCP,
> Reply, length 328
> 08:01:38.048136 ARP, Request who-has 192.168.111.122 tell 192.168.111.3,
> length 28
> 08:01:39.051897 ARP, Request who-has 192.168.111.122 tell 192.168.111.3,
> length 28
> 08:01:40.051901 ARP, Request who-has 192.168.111.122 tell 192.168.111.3,
> length 28
> 08:01:45.168519 ARP, Request who-has 192.168.111.138 tell 192.168.111.138,
> length 46
> 08:01:45.168546 ARP, Request who-has 192.168.111.138 tell 192.168.111.138,
> length 46
> 08:01:46.962341 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request
> from fa:16:3e:0d:26:a5, length 300
> 08:01:46.962584 IP 192.168.111.3.67 > 192.168.111.122.68: BOOTP/DHCP,
> Reply, length 328
> 08:01:49.438756 ARP, Request who-has 192.168.111.138 tell 192.168.111.138,
> length 46
> 08:01:49.438780 ARP, Request who-has 192.168.111.138 tell 192.168.111.138,
> length 46
> 08:01:51.346561 ARP, Request who-has 192.168.111.112 tell 192.168.111.136,
> length 46
> 08:01:51.967905 ARP, Request who-has 192.168.111.122 tell 192.168.111.3,
> length 28
>
> Thanks,
> Sam Stoelinga
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>


-- 
Nasir Mahmood
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150922/9145b021/attachment.html>