[Openstack] [IceHouse][Trusty] iptables rules disappeared from within Tenant Namespace (qdhcp-XXX), all tenants affected! Metadata not working anymore...

Martinx - ジェームズ thiagocmartinsc at gmail.com
Sat Aug 16 03:48:37 UTC 2014


Fixed!

I just did on Neutron Node:

---
# stop neutron-* services
service neutron-ovs-cleanup start
reboot
---

---
ubuntu at linux-builder-1:~$ curl http://169.254.169.254/
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04
---

Knowledge base...    :-P


On 16 August 2014 00:40, Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote:

> Guys,
>
> I'm seeing the following error on Neutron Node:
>
> --
> 2014-08-15 20:36:59.596 12685 ERROR neutron.agent.dhcp_agent
> [req-17198f16-4149-47f8-8647-0b381df7d888 None] Unable to enable dhcp for
> f0076840-43f3-4b2e-aa15-d6b2422e3795.
> --
>
> I'm getting there...   :-)
>
>
> On 15 August 2014 21:10, Martinx - ジェームズ <thiagocmartinsc at gmail.com>
> wrote:
>
>> Guys,
>>
>> Today I'm facing a old/new problem... Metadata doesn't work anymore
>> (again)... From an already running instance, I'm seeing:
>>
>>
>> ---
>> ubuntu at linux-builder-1:~$ curl http://169.254.169.254/
>> curl: (7) Failed to connect to 169.254.169.254 port 80: Connection refused
>> ---
>>
>>
>> Then, I looked at my Neutron Node, at this tenant Namespace, there are no
>> iptables rules there, look:
>>
>> ---
>> root at neutron-node-1:/var/log/neutron# ip netns exec
>> qdhcp-f0076840-43f3-4b2e-aa15-d6b2422e3795 iptables -L -nv -t nat
>> Chain PREROUTING (policy ACCEPT 4 packets, 776 bytes)
>>  pkts bytes target     prot opt in     out     source
>> destination
>>
>> Chain INPUT (policy ACCEPT 4 packets, 776 bytes)
>>  pkts bytes target     prot opt in     out     source
>> destination
>>
>> Chain OUTPUT (policy ACCEPT 1 packets, 328 bytes)
>>  pkts bytes target     prot opt in     out     source
>> destination
>>
>> Chain POSTROUTING (policy ACCEPT 1 packets, 328 bytes)
>>  pkts bytes target     prot opt in     out     source
>> destination
>>
>>
>>  root at neutron-node-1:/var/log/neutron# ip netns exec
>> qdhcp-f0076840-43f3-4b2e-aa15-d6b2422e3795 ip -4 r
>> default via 10.192.0.1 dev tap216bf57d-e8
>> 10.192.0.0/20 dev tap216bf57d-e8  proto kernel  scope link  src
>> 10.192.0.3
>> 169.254.0.0/16 dev tap216bf57d-e8  proto kernel  scope link  src
>> 169.254.169.254
>> ---
>>
>>
>> I can see that "curl request" within the Namespace, with tcpdump:
>> ---
>> root at neutron-node-1:~# ip netns exec
>> qdhcp-f0076840-43f3-4b2e-aa15-d6b2422e3795 tcpdump -ni tap216bf57d-e8
>>
>>
>>
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>> listening on tap216bf57d-e8, link-type EN10MB (Ethernet), capture size
>> 65535 bytes
>> 21:01:29.582960 IP 10.192.0.90.55635 > 169.254.169.254.80: Flags [S], seq
>> 2521313833, win 29200, options [mss 1460,sackOK,TS val 85649833 ecr
>> 0,nop,wscale 7], length 0
>> 21:01:29.583140 IP 169.254.169.254.80 > 10.192.0.90.55635: Flags [R.],
>> seq 0, ack 2521313834, win 0, length 0
>> ---
>>
>>
>> If I'm not wrong, there was some iptables NAT rules there, to redirect
>> the Metadata traffic to the Nova API (controller-node-1) at TCP port 8775,
>> right?!
>>
>> I'm using `VLAN Provider Networks` (No L3 Router), my Instances have a
>> route to the 169.254.169.254 IP, via their Namespace IP (10.192.0.3), look:
>>
>> ---
>> ubuntu at linux-builder-1:~$ ip r
>> default via 10.192.0.1 dev eth0
>> 10.192.0.0/20 dev eth0  proto kernel  scope link  src 10.192.0.90
>> 169.254.169.254 via 10.192.0.3 dev eth0
>>
>> ubuntu at linux-builder-1:~$ ping -c 1 10.192.0.3
>> PING 10.192.0.3 (10.192.0.3) 56(84) bytes of data.
>> 64 bytes from 10.192.0.3: icmp_seq=1 ttl=64 time=4.55 ms
>>  ---
>>
>> It crashed today, it was okay yesterday... This time, I did nothing wrong
>> (I think)...   :-(
>>
>> BTW, I just upgraded all nodes (apt-get update / dist-upgrade), still
>> doesn't work... Proposed repo enabled.
>>
>> I really appreciate any help!
>>
>> Tks!
>> Thiago
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140816/b47e1ad1/attachment.html>


More information about the Openstack mailing list