<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Ignore that now all,</p>
<p>Managed to fix it by restarting the l3-agent. Looks like it must
have been cached in memory.</p>
<p>Thanks,<br>
</p>
<br>
<div class="moz-cite-prefix">On 08/06/17 18:07, Grant Morley wrote:<br>
</div>
<blockquote type="cite"
cite="mid:e3897c96-118b-6f9b-bef5-c5a7d27fd658@absolutedevops.io">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<p>Hi All,</p>
<p>We have noticed in our neutron-l3-agent logs that there are a
number of routers that neutron seems to think exist ( but they
no longer exist in the data base ) and it is constantly trying
to delete them. However because they don't appear to exist, so
neutron seems to get stuck in a loop of trying to delete
routers that are no longer there.</p>
<p>We originally noticed that neutron was trying to delete the
"qrouter-xxxx-xxx-xxxx" and that didn't exist. So we added the
router manually with "ip netns add qrouter-xxx-xxx-xxx-xxx"
which then deletes the router however we then get the following
error messages:</p>
<p>2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent [-]
Error while deleting router da7b633a-233b-46d1-ba3d-315b3bee6a61<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
Traceback (most recent call last):<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/agent.py",
line 359, in _safe_router_removed<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self._router_removed(router_id)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/agent.py",
line 377, in _router_removed<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
ri.delete(self)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py",
line 380, in delete<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
super(HaRouter, self).delete(agent)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
line 361, in delete<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self.process_delete(agent)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/common/utils.py",
line 385, in call<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self.logger(e)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/oslo_utils/excutils.py",
line 220, in __exit__<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self.force_reraise()<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/oslo_utils/excutils.py",
line 196, in force_reraise<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
six.reraise(self.type_, self.value, self.tb)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/common/utils.py",
line 382, in call<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
return func(*args, **kwargs)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
line 972, in process_delete<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self._process_external_on_delete(agent)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
line 794, in _process_external_on_delete<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self._process_external_gateway(ex_gw_port, agent.pd)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
line 693, in _process_external_gateway<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self.external_gateway_removed(self.ex_gw_port, interface_name)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py",
line 371, in external_gateway_removed<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
interface_name)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
line 668, in external_gateway_removed<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
ip_addr['prefixlen']))<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
line 286, in remove_external_gateway_ip<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
device.delete_addr_and_conntrack_state(ip_cidr)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py",
line 290, in delete_addr_and_conntrack_state<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
self.addr.delete(cidr)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py",
line 580, in delete<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
'dev', self.name))<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py",
line 361, in _as_root<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
use_root_namespace=use_root_namespace)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py",
line 94, in _as_root<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
log_fail_as_error=self.log_fail_as_error)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py",
line 103, in _execute<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
log_fail_as_error=log_fail_as_error)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent File
"/openstack/venvs/neutron-13.3.8/lib/python2.7/site-packages/neutron/agent/linux/utils.py",
line 140, in execute<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
raise RuntimeError(msg)<br>
2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot
find device "qg-c4080b5c-c9"</p>
<p>Has anyone come across this before?</p>
<p>We don't seem to have an entry for them anywhere from what we
can see.</p>
<p>Regards,<br>
</p>
<p><br>
</p>
<div class="moz-signature">-- <br>
<!--?xml version="1.0" encoding="UTF-8" standalone="no"?-->
<title></title>
<div style="height:13px"> </div>
<div> <font color="#A8D8FF" face="Lucida Grande, Verdana,
Arial, sans-serif"> <span style="font-size: 11px;
line-height: 15px; float: left; margin-right: 10px;"> <img
src="https://dl.dropboxusercontent.com/u/478290/email/ado_new.png"
style="color: #a8d8ff; font-family: 'Lucida Grande',
Verdana, Arial, sans-serif; font-size: 11px;
line-height: 15px; float: left; margin-top: 0px;
margin-right: 10px" moz-do-not-send="true"> </span> </font>
<div style="margin-left: 80px; line-height: 15px; color:
rgb(109, 190, 255); margin-bottom: 5px;"> <span
style="font-family: Hind, 'Lucida Grande', Verdana, Arial,
sans-serif; font-size: 18px; font-weight: 200; color:
rgb(35, 157, 255);">Grant Morley</span> <span
style="color: rgb(35, 157, 255); font-size: 15px;"> </span>
<font face="Hind, Lucida Grande, Verdana, Arial, sans-serif">
<span style="font-size: 15px;"> </span> </font> </div>
<div style="margin-left: 80px; line-height: 15px; color:
rgb(109, 190, 255); margin-bottom: 10px;"> <font
face="Hind, Lucida Grande, Verdana, Arial, sans-serif"> <span
style="font-size: 15px;">Cloud Lead</span> </font> </div>
<div style="font-family: Hind, 'Lucida Grande', Verdana,
Arial, sans-serif; font-size: 13px; margin-left: 80px;
line-height: 15px; color: rgb(153, 153, 153);"> <span
style="color: #616161">Absolute</span> <span
style="color: #98a5b0">DevOps Ltd</span> <br>
Units H, J & K, Gateway 1000, Whittle Way, Stevenage,
Herts, SG1 2FP </div>
<div style="font-family: Hind, helvetica, arial; font-size:
13px; margin-left: 80px;"> <a
href="http://www.absolutedevops.io/" style="color:
#239dff; font-family: Hind, 'Lucida Grande', Verdana,
Arial, sans-serif;line-height: 15px; margin-right: 6px;
text-decoration: none;" moz-do-not-send="true">www.absolutedevops.io</a>
<a href="mailto:grant@absolutedevops.i" style="color:
#239dff; font-family: Hind, 'Lucida Grande', Verdana,
Arial, sans-serif;line-height: 15px; margin-right: 6px;
text-decoration: none;" moz-do-not-send="true">grant@absolutedevops.io</a>
<a style="color: #239dff; font-family: Hind, 'Lucida
Grande', Verdana, Arial, sans-serif;line-height: 15px;
margin-right: 6px;" moz-do-not-send="true">0845 874 0580</a>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-operators mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<!--?xml version="1.0" encoding="UTF-8" standalone="no"?-->
<title></title>
<div style="height:13px"> </div>
<div> <font color="#A8D8FF" face="Lucida Grande, Verdana, Arial,
sans-serif"> <span style="font-size: 11px; line-height: 15px;
float: left; margin-right: 10px;"> <img
src="https://dl.dropboxusercontent.com/u/478290/email/ado_new.png"
style="color: #a8d8ff; font-family: 'Lucida Grande',
Verdana, Arial, sans-serif; font-size: 11px; line-height:
15px; float: left; margin-top: 0px; margin-right: 10px"> </span>
</font>
<div style="margin-left: 80px; line-height: 15px; color:
rgb(109, 190, 255); margin-bottom: 5px;"> <span
style="font-family: Hind, 'Lucida Grande', Verdana, Arial,
sans-serif; font-size: 18px; font-weight: 200; color:
rgb(35, 157, 255);">Grant Morley</span> <span style="color:
rgb(35, 157, 255); font-size: 15px;"> </span> <font
face="Hind, Lucida Grande, Verdana, Arial, sans-serif"> <span
style="font-size: 15px;"> </span> </font> </div>
<div style="margin-left: 80px; line-height: 15px; color:
rgb(109, 190, 255); margin-bottom: 10px;"> <font face="Hind,
Lucida Grande, Verdana, Arial, sans-serif"> <span
style="font-size: 15px;">Cloud Lead</span> </font> </div>
<div style="font-family: Hind, 'Lucida Grande', Verdana, Arial,
sans-serif; font-size: 13px; margin-left: 80px; line-height:
15px; color: rgb(153, 153, 153);"> <span style="color:
#616161">Absolute</span> <span style="color: #98a5b0">DevOps
Ltd</span> <br>
Units H, J & K, Gateway 1000, Whittle Way, Stevenage,
Herts, SG1 2FP
</div>
<div style="font-family: Hind, helvetica, arial; font-size:
13px; margin-left: 80px;"> <a
href="http://www.absolutedevops.io/" style="color: #239dff;
font-family: Hind, 'Lucida Grande', Verdana, Arial,
sans-serif;line-height: 15px; margin-right: 6px;
text-decoration: none;">www.absolutedevops.io</a> <a
href="mailto:grant@absolutedevops.i" style="color: #239dff;
font-family: Hind, 'Lucida Grande', Verdana, Arial,
sans-serif;line-height: 15px; margin-right: 6px;
text-decoration: none;">grant@absolutedevops.io</a> <a
style="color: #239dff; font-family: Hind, 'Lucida Grande',
Verdana, Arial, sans-serif;line-height: 15px; margin-right:
6px;">0845 874 0580</a> </div>
</div>
</div>
</body>
</html>