<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div>Hi,</div>
<div><br>
</div>
<div>We're using Openstack Ocata, deployed using Openstack Ansible v15.1.7. Neutron server is v10.0.3.</div>
<div>I can see enable_isolated_metadata and enable_metadata_network only used for isolated networks that don't have a router which is not our case.</div>
<div>Also, I checked all namespaces on all our novas and only affected 6 out of 66 ..and only 1 namespace / nova. Seems like isolated case that doesn't happen very often.</div>
<div><br>
</div>
<div>Can it be RabbitMQ? I'm not sure where to check.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Radu</div>
<div><br>
</div>
<div>On Fri, 2018-06-15 at 17:11 +0200, Saverio Proto wrote:</div>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<pre>Hello Radu,</pre>
<pre><br></pre>
<pre>yours look more or less like a bug report. This you check existing</pre>
<pre>open bugs for neutron ? Also what version of openstack are you running</pre>
<pre>?</pre>
<pre><br></pre>
<pre>how did you configure enable_isolated_metadata and</pre>
<pre>enable_metadata_network options ?</pre>
<pre><br></pre>
<pre>Saverio</pre>
<pre><br></pre>
<pre>2018-06-13 12:45 GMT+02:00 Radu Popescu | eMAG, Technology</pre>
<pre><<a href="mailto:radu.popescu@emag.ro">radu.popescu@emag.ro</a>>:</pre>
<pre></pre>
<pre>Hi all,</pre>
<pre><br></pre>
<pre>So, I'm having the following issue. I'm creating a VM with floating IP.</pre>
<pre>Everything is fine, namespace is there, postrouting and prerouting from the</pre>
<pre>internal IP to the floating IP are there. The only rules missing are the</pre>
<pre>rules to access metadata service:</pre>
<pre><br></pre>
<pre>-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp</pre>
<pre>--dport 80 -j REDIRECT --to-ports 9697</pre>
<pre>-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp</pre>
<pre>--dport 80 -j MARK --set-xmark 0x1/0xffff</pre>
<pre><br></pre>
<pre>(this is taken from another working namespace with iptables-save)</pre>
<pre><br></pre>
<pre>Forgot to mention, VM is booting ok, I have both the default route and the</pre>
<pre>one for the metadata service (cloud-init is running at boot time):</pre>
<pre>[ 57.150766] cloud-init[892]: ci-info:</pre>
<pre>+--------+------+--------------+---------------+-------+-------------------+</pre>
<pre>[ 57.150997] cloud-init[892]: ci-info: | Device | Up | Address |</pre>
<pre>Mask | Scope | Hw-Address |</pre>
<pre>[ 57.151219] cloud-init[892]: ci-info:</pre>
<pre>+--------+------+--------------+---------------+-------+-------------------+</pre>
<pre>[ 57.151431] cloud-init[892]: ci-info: | lo: | True | 127.0.0.1 |</pre>
<pre>255.0.0.0 | . | . |</pre>
<pre>[ 57.151627] cloud-init[892]: ci-info: | eth0: | True | 10.240.9.186 |</pre>
<pre>255.255.252.0 | . | fa:16:3e:43:d1:c2 |</pre>
<pre>[ 57.151815] cloud-init[892]: ci-info:</pre>
<pre>+--------+------+--------------+---------------+-------+-------------------+</pre>
<pre>[ 57.152018] cloud-init[892]: ci-info:</pre>
<pre>+++++++++++++++++++++++++++++++Route IPv4</pre>
<pre>info++++++++++++++++++++++++++++++++</pre>
<pre>[ 57.152225] cloud-init[892]: ci-info:</pre>
<pre>+-------+-----------------+------------+-----------------+-----------+-------+</pre>
<pre>[ 57.152426] cloud-init[892]: ci-info: | Route | Destination |</pre>
<pre>Gateway | Genmask | Interface | Flags |</pre>
<pre>[ 57.152621] cloud-init[892]: ci-info:</pre>
<pre>+-------+-----------------+------------+-----------------+-----------+-------+</pre>
<pre>[ 57.152813] cloud-init[892]: ci-info: | 0 | 0.0.0.0 |</pre>
<pre>10.240.8.1 | 0.0.0.0 | eth0 | UG |</pre>
<pre>[ 57.153013] cloud-init[892]: ci-info: | 1 | 10.240.1.0 |</pre>
<pre>0.0.0.0 | 255.255.255.0 | eth0 | U |</pre>
<pre>[ 57.153202] cloud-init[892]: ci-info: | 2 | 10.240.8.0 |</pre>
<pre>0.0.0.0 | 255.255.252.0 | eth0 | U |</pre>
<pre>[ 57.153397] cloud-init[892]: ci-info: | 3 | 169.254.169.254 |</pre>
<pre>10.240.8.1 | 255.255.255.255 | eth0 | UGH |</pre>
<pre>[ 57.153579] cloud-init[892]: ci-info:</pre>
<pre>+-------+-----------------+------------+-----------------+-----------+-------+</pre>
<pre><br></pre>
<pre>The extra route is there because the tenant has 2 subnets.</pre>
<pre><br></pre>
<pre>Before adding those 2 rules manually, I had this coming from cloud-init:</pre>
<pre><br></pre>
<pre>[ 192.451801] cloud-init[892]: 2018-06-13 12:29:26,179 -</pre>
<pre>url_helper.py[WARNING]: Calling</pre>
<pre>'<a href="http://169.254.169.254/2009-04-04/meta-data/instance-id'">http://169.254.169.254/2009-04-04/meta-data/instance-id'</a> failed [0/120s]:</pre>
<pre>request error [('Connection aborted.', error(113, 'No route to host'))]</pre>
<pre>[ 193.456805] cloud-init[892]: 2018-06-13 12:29:27,184 -</pre>
<pre>url_helper.py[WARNING]: Calling</pre>
<pre>'<a href="http://169.254.169.254/2009-04-04/meta-data/instance-id'">http://169.254.169.254/2009-04-04/meta-data/instance-id'</a> failed [1/120s]:</pre>
<pre>request error [('Connection aborted.', error(113, 'No route to host'))]</pre>
<pre>[ 194.461592] cloud-init[892]: 2018-06-13 12:29:28,189 -</pre>
<pre>url_helper.py[WARNING]: Calling</pre>
<pre>'<a href="http://169.254.169.254/2009-04-04/meta-data/instance-id'">http://169.254.169.254/2009-04-04/meta-data/instance-id'</a> failed [2/120s]:</pre>
<pre>request error [('Connection aborted.', error(113, 'No route to host'))]</pre>
<pre>[ 195.466441] cloud-init[892]: 2018-06-13 12:29:29,194 -</pre>
<pre>url_helper.py[WARNING]: Calling</pre>
<pre>'<a href="http://169.254.169.254/2009-04-04/meta-data/instance-id'">http://169.254.169.254/2009-04-04/meta-data/instance-id'</a> failed [3/120s]:</pre>
<pre>request error [('Connection aborted.', error(113, 'No route to host'))]</pre>
<pre><br></pre>
<pre>I can see no errors in neither nova or neutron services.</pre>
<pre>In the mean time, I've searched all our nova servers for this kind of</pre>
<pre>behavior and we have 1 random namespace missing those rules on 6 of our 66</pre>
<pre>novas.</pre>
<pre><br></pre>
<pre>Any ideas would be greatly appreciated.</pre>
<pre><br></pre>
<pre>Thanks,</pre>
<pre>Radu</pre>
<pre><br></pre>
<pre>_______________________________________________</pre>
<pre>OpenStack-operators mailing list</pre>
<pre><a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a></pre>
<pre><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a></pre>
<pre><br></pre>
</blockquote>
</body>
</html>