<div dir="ltr">Thanks, George.<div><br></div><div>I found this too: <a href="https://bugs.launchpad.net/neutron/+bug/1607398">https://bugs.launchpad.net/neutron/+bug/1607398</a></div><div><br></div><div>Chasing down that lead right now by trying to down-rev all my kernels, and re-test.</div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Aug 10, 2016 at 5:51 PM George Zhao <<a href="mailto:george@hd.net.nz">george@hd.net.nz</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I'm having the similar issue, only happens after live migrated instance<br>
to other host. In dashboard, it shows the port with floating IP is<br>
detached and status n/a, under instance tab, it shows the floating IP is<br>
associated with instance.<br>
<br>
nova compute showed successfully added bridge and veth, l3-agent showed<br>
successfully added namespace but failed doing arping.<br>
<br>
<br>
<br>
George Zhao<br>
<br>
<br>
<br>
On 08/11/2016 07:18 AM, Jonathan Mills wrote:<br>
> Hi all,<br>
><br>
> I’m running Mitaka on CentOS 7.2 with Neutron in dvr_snat mode.<br>
><br>
> # uname -msr<br>
><br>
> Linux 3.10.0-327.22.2.el7.x86_64 x86_64<br>
><br>
> I’m using vlans, not vxlans, but I don’t think that matters either way.<br>
> So basically, I have one NIC “eth2” which is in vlan trunk mode, and on<br>
> my switch side, I have every neutron-defined vlan trunked there.<br>
> Whether it’s a tenant network vlan, or an external vlan for floating<br>
> IPs, it all comes back to that same NIC.<br>
><br>
> So here’s a compute node “node1”. It has a successfully booted VM,<br>
> which has fixed IP 10.97.8.103 and floating IP 10.96.8.107. As seen<br>
> from the compute node:<br>
><br>
><br>
> # ip netns<br>
><br>
> fip-cbe55dc5-c4e4-4ec0-aa52-b4713f1279ee<br>
><br>
> qrouter-efc60192-97ad-49ef-bab7-cda42ca6bc29<br>
><br>
> snat-efc60192-97ad-49ef-bab7-cda42ca6bc29<br>
><br>
><br>
><br>
> # ip netns exec fip-cbe55dc5-c4e4-4ec0-aa52-b4713f1279ee ip addr<br>
><br>
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN<br>
><br>
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br>
><br>
> inet <a href="http://127.0.0.1/8" rel="noreferrer" target="_blank">127.0.0.1/8</a> <<a href="http://127.0.0.1/8" rel="noreferrer" target="_blank">http://127.0.0.1/8</a>> scope host lo<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> inet6 ::1/128 scope host<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> 2: fpr-efc60192-9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc<br>
> pfifo_fast state UP qlen 1000<br>
><br>
> link/ether 32:06:67:df:53:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0<br>
><br>
> inet <a href="http://169.254.109.47/31" rel="noreferrer" target="_blank">169.254.109.47/31</a> <<a href="http://169.254.109.47/31" rel="noreferrer" target="_blank">http://169.254.109.47/31</a>> scope global<br>
> fpr-efc60192-9<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> inet6 fe80::3006:67ff:fedf:53c6/64 scope link<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> 19: fg-152dc56a-c1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc<br>
> noqueue state UNKNOWN<br>
><br>
> link/ether fa:16:3e:40:9f:5b brd ff:ff:ff:ff:ff:ff<br>
><br>
> inet <a href="http://10.96.8.101/23" rel="noreferrer" target="_blank">10.96.8.101/23</a> <<a href="http://10.96.8.101/23" rel="noreferrer" target="_blank">http://10.96.8.101/23</a>> brd 10.96.9.255 scope<br>
> global fg-152dc56a-c1<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> inet6 fe80::f816:3eff:fe40:9f5b/64 scope link<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
><br>
><br>
> # ip netns exec qrouter-efc60192-97ad-49ef-bab7-cda42ca6bc29 ip addr<br>
><br>
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN<br>
><br>
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br>
><br>
> inet <a href="http://127.0.0.1/8" rel="noreferrer" target="_blank">127.0.0.1/8</a> <<a href="http://127.0.0.1/8" rel="noreferrer" target="_blank">http://127.0.0.1/8</a>> scope host lo<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> inet6 ::1/128 scope host<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> 2: rfp-efc60192-9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc<br>
> pfifo_fast state UP qlen 1000<br>
><br>
> link/ether 72:49:e7:78:48:5d brd ff:ff:ff:ff:ff:ff link-netnsid 0<br>
><br>
> inet <a href="http://169.254.109.46/31" rel="noreferrer" target="_blank">169.254.109.46/31</a> <<a href="http://169.254.109.46/31" rel="noreferrer" target="_blank">http://169.254.109.46/31</a>> scope global<br>
> rfp-efc60192-9<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> inet <a href="http://10.96.8.107/32" rel="noreferrer" target="_blank">10.96.8.107/32</a> <<a href="http://10.96.8.107/32" rel="noreferrer" target="_blank">http://10.96.8.107/32</a>> brd 10.96.8.107 scope<br>
> global rfp-efc60192-9<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> inet6 fe80::7049:e7ff:fe78:485d/64 scope link<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> 17: qr-ffc302ba-82: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc<br>
> noqueue state UNKNOWN<br>
><br>
> link/ether fa:16:3e:8d:7c:62 brd ff:ff:ff:ff:ff:ff<br>
><br>
> inet <a href="http://10.97.8.1/23" rel="noreferrer" target="_blank">10.97.8.1/23</a> <<a href="http://10.97.8.1/23" rel="noreferrer" target="_blank">http://10.97.8.1/23</a>> brd 10.97.9.255 scope global<br>
> qr-ffc302ba-82<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
> inet6 fe80::f816:3eff:fe8d:7c62/64 scope link<br>
><br>
> valid_lft forever preferred_lft forever<br>
><br>
><br>
><br>
><br>
> So you can see that I have both the ‘fpr’ and ‘rfp’ namespaces, which<br>
> is a good indicator I didn’t totally flub the dvr_snat neutron config.<br>
> From within either namespace, I can ping the floating IP 10.96.8.107,<br>
> which makes sense. However, for the floating IP to be useful, it would<br>
> need to be generally reachable by any other system in its designated<br>
> vlan, and that is not the case. In my real-world use case, I would be<br>
> running the vlan of this floating IP network back over to my bastion<br>
> host, to allow users to ssh into their VMs via the floating IP. I can’t<br>
> reach the floating IPs though from anywhere outside the namespace on the<br>
> compute node.<br>
><br>
><br>
> One more clue, in the l3-agent log on the compute node in question:<br>
><br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib [-] Failed<br>
> sending gratuitous ARP to 10.96.8.107 on fg-152dc56a-c1 in namespace<br>
> fip-cbe55dc5-c4e4-4ec0-aa52-b4713f1279ee<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib Traceback<br>
> (most recent call last):<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib File<br>
> "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line<br>
> 1040, in _arping<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib<br>
> ip_wrapper.netns.execute(arping_cmd, check_exit_code=True)<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib File<br>
> "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line<br>
> 927, in execute<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib<br>
> log_fail_as_error=log_fail_as_error, **kwargs)<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib File<br>
> "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line<br>
> 140, in execute<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib raise<br>
> RuntimeError(msg)<br>
><br>
> 2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib<br>
> RuntimeError: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot<br>
> assign requested address<br>
><br>
><br>
> After a little Googling, I think I may be seeing the same behavior as<br>
> this user:<br>
><br>
> <a href="https://bugs.centos.org/view.php?id=11238" rel="noreferrer" target="_blank">https://bugs.centos.org/view.php?id=11238</a><br>
><br>
> I’m reaching out to see if anyone else has witnessed this, or has any<br>
> sage advice for me.<br>
><br>
><br>
> Jonathan<br>
><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-operators mailing list<br>
> <a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
><br>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</blockquote></div>