[Openstack] [IceHouse][Trusty] iptables rules disappeared from within Tenant Namespace (qdhcp-XXX), all tenants affected! Metadata not working anymore...

Martinx - ジェームズ thiagocmartinsc at gmail.com
Sat Aug 16 03:40:53 UTC 2014


Guys,

I'm seeing the following error on Neutron Node:

--
2014-08-15 20:36:59.596 12685 ERROR neutron.agent.dhcp_agent
[req-17198f16-4149-47f8-8647-0b381df7d888 None] Unable to enable dhcp for
f0076840-43f3-4b2e-aa15-d6b2422e3795.
--

I'm getting there...   :-)


On 15 August 2014 21:10, Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote:

> Guys,
>
> Today I'm facing a old/new problem... Metadata doesn't work anymore
> (again)... From an already running instance, I'm seeing:
>
>
> ---
> ubuntu at linux-builder-1:~$ curl http://169.254.169.254/
> curl: (7) Failed to connect to 169.254.169.254 port 80: Connection refused
> ---
>
>
> Then, I looked at my Neutron Node, at this tenant Namespace, there are no
> iptables rules there, look:
>
> ---
> root at neutron-node-1:/var/log/neutron# ip netns exec
> qdhcp-f0076840-43f3-4b2e-aa15-d6b2422e3795 iptables -L -nv -t nat
> Chain PREROUTING (policy ACCEPT 4 packets, 776 bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> Chain INPUT (policy ACCEPT 4 packets, 776 bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> Chain OUTPUT (policy ACCEPT 1 packets, 328 bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> Chain POSTROUTING (policy ACCEPT 1 packets, 328 bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
>
>  root at neutron-node-1:/var/log/neutron# ip netns exec
> qdhcp-f0076840-43f3-4b2e-aa15-d6b2422e3795 ip -4 r
> default via 10.192.0.1 dev tap216bf57d-e8
> 10.192.0.0/20 dev tap216bf57d-e8  proto kernel  scope link  src
> 10.192.0.3
> 169.254.0.0/16 dev tap216bf57d-e8  proto kernel  scope link  src
> 169.254.169.254
> ---
>
>
> I can see that "curl request" within the Namespace, with tcpdump:
> ---
> root at neutron-node-1:~# ip netns exec
> qdhcp-f0076840-43f3-4b2e-aa15-d6b2422e3795 tcpdump -ni tap216bf57d-e8
>
>
>
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on tap216bf57d-e8, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 21:01:29.582960 IP 10.192.0.90.55635 > 169.254.169.254.80: Flags [S], seq
> 2521313833, win 29200, options [mss 1460,sackOK,TS val 85649833 ecr
> 0,nop,wscale 7], length 0
> 21:01:29.583140 IP 169.254.169.254.80 > 10.192.0.90.55635: Flags [R.], seq
> 0, ack 2521313834, win 0, length 0
> ---
>
>
> If I'm not wrong, there was some iptables NAT rules there, to redirect the
> Metadata traffic to the Nova API (controller-node-1) at TCP port 8775,
> right?!
>
> I'm using `VLAN Provider Networks` (No L3 Router), my Instances have a
> route to the 169.254.169.254 IP, via their Namespace IP (10.192.0.3), look:
>
> ---
> ubuntu at linux-builder-1:~$ ip r
> default via 10.192.0.1 dev eth0
> 10.192.0.0/20 dev eth0  proto kernel  scope link  src 10.192.0.90
> 169.254.169.254 via 10.192.0.3 dev eth0
>
> ubuntu at linux-builder-1:~$ ping -c 1 10.192.0.3
> PING 10.192.0.3 (10.192.0.3) 56(84) bytes of data.
> 64 bytes from 10.192.0.3: icmp_seq=1 ttl=64 time=4.55 ms
>  ---
>
> It crashed today, it was okay yesterday... This time, I did nothing wrong
> (I think)...   :-(
>
> BTW, I just upgraded all nodes (apt-get update / dist-upgrade), still
> doesn't work... Proposed repo enabled.
>
> I really appreciate any help!
>
> Tks!
> Thiago
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140816/687b2124/attachment.html>


More information about the Openstack mailing list