[Openstack] [Openstack-operators] Recovering from full outage
Torin Woltjer
torin.woltjer at granddial.com
Mon Jul 16 12:41:16 UTC 2018
$ip netns exec qdhcp-87a5200d-057f-475d-953d-17e873a47454 curl http://169.254.169.254
<html>
<head>
<title>404 Not Found</title>
</head>
<body>
<h1>404 Not Found</h1>
The resource could not be found.<br /><br />
</body>
</html>
$ip netns exec qrouter-80c3bc40-b49c-446a-926f-99811adc0c5e curl http://169.254.169.254
curl: (7) Couldn't connect to server
Torin Woltjer
Grand Dial Communications - A ZK Tech Inc. Company
616.776.1066 ext. 2006
www.granddial.com
----------------------------------------
From: "Torin Woltjer" <torin.woltjer at granddial.com>
Sent: 7/12/18 11:16 AM
To: <haleyb.dev at gmail.com>, <thangam.arunx at gmail.com>, "jpetrini at coredial.com" <jpetrini at coredial.com>
Cc: openstack-operators at lists.openstack.org, openstack at lists.openstack.org
Subject: Re: [Openstack] [Openstack-operators] Recovering from full outage
Checking iptables for the metadata-proxy inside of qrouter provides the following:
$ip netns exec qrouter-80c3bc40-b49c-446a-926f-99811adc0c5e iptables-save -c | grep 169
[0:0] -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
[0:0] -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff
Packets:Bytes are both 0, so no traffic is touching this rule?
Interestingly the command:
$ip netns exec qrouter-80c3bc40-b49c-446a-926f-99811adc0c5e netstat -anep | grep 9697
returns nothing, so there isn't actually anything running on 9697 in the network namespace...
This is the output without grep:
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name
raw 0 0 0.0.0.0:112 0.0.0.0:* 7 0 76154 8404/keepalived
raw 0 0 0.0.0.0:112 0.0.0.0:* 7 0 76153 8404/keepalived
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node PID/Program name Path
unix 2 [ ] DGRAM 64501 7567/python2
unix 2 [ ] DGRAM 79953 8403/keepalived
Could the reason no traffic touching the rule be that nothing is listening on that port, or is there a second issue down the chain?
Curl fails even after restarting the neutron-dhcp-agent & neutron-metadata agent.
Thank you for this, and any future help.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20180716/57a32033/attachment.html>
More information about the Openstack
mailing list