Open Stack

I patched this back in liberty.
What version of Openstack are you using ?

Saverio


2017-06-08 19:36 GMT+02:00 Grant Morley <grant at absolutedevops.io>:

> Ignore that now all,
>
> Managed to fix it by restarting the l3-agent. Looks like it must have been
> cached in memory.
>
> Thanks,
>
> On 08/06/17 18:07, Grant Morley wrote:
>
> Hi All,
>
> We have  noticed in our neutron-l3-agent logs that there are a number of
> routers that neutron seems to think exist ( but they no longer exist in the
> data base )  and it is constantly trying to delete them. However because
> they don't appear to exist,  so neutron seems to get stuck in a loop of
> trying to delete  routers that are no longer there.
>
> We originally noticed that neutron was trying to delete the
> "qrouter-xxxx-xxx-xxxx" and that didn't exist. So we added the router
> manually with "ip netns add qrouter-xxx-xxx-xxx-xxx" which then deletes the
> router however we then get the following error messages:
>
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent [-] Error while
> deleting router da7b633a-233b-46d1-ba3d-315b3bee6a61
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent Traceback (most
> recent call last):
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/agent.py", line 359, in _safe_router_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self._router_removed(router_id)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/agent.py", line 377, in _router_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> ri.delete(self)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/ha_router.py", line 380, in delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> super(HaRouter, self).delete(agent)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 361, in delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self.process_delete(agent)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/common/utils.py", line 385, in call
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent     self.logger(e)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/oslo_utils/excutils.py", line 220, in __exit__
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self.force_reraise()
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/oslo_utils/excutils.py", line 196, in force_reraise
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> six.reraise(self.type_, self.value, self.tb)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/common/utils.py", line 382, in call
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent     return
> func(*args, **kwargs)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 972, in process_delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self._process_external_on_delete(agent)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 794, in
> _process_external_on_delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self._process_external_gateway(ex_gw_port, agent.pd)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 693, in
> _process_external_gateway
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self.external_gateway_removed(self.ex_gw_port, interface_name)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/ha_router.py", line 371, in
> external_gateway_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> interface_name)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 668, in
> external_gateway_removed
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> ip_addr['prefixlen']))
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/l3/router_info.py", line 286, in
> remove_external_gateway_ip
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> device.delete_addr_and_conntrack_state(ip_cidr)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/linux/ip_lib.py", line 290, in
> delete_addr_and_conntrack_state
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> self.addr.delete(cidr)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/linux/ip_lib.py", line 580, in delete
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent     'dev',
> self.name))
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/linux/ip_lib.py", line 361, in _as_root
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> use_root_namespace=use_root_namespace)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/linux/ip_lib.py", line 94, in _as_root
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> log_fail_as_error=self.log_fail_as_error)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/linux/ip_lib.py", line 103, in _execute
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent
> log_fail_as_error=log_fail_as_error)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent   File
> "/openstack/venvs/neutron-13.3.8/lib/python2.7/site-
> packages/neutron/agent/linux/utils.py", line 140, in execute
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent     raise
> RuntimeError(msg)
> 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent RuntimeError:
> Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-c4080b5c-c9"
>
> Has anyone come across this before?
>
> We don't seem to have an entry for them anywhere from what we can see.
>
> Regards,
>
>
> --
> Grant Morley
> Cloud Lead
> Absolute DevOps Ltd
> Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
> www.absolutedevops.io grant at absolutedevops.io <grant at absolutedevops.i> 0845
> 874 0580
>
>
> _______________________________________________
> OpenStack-operators mailing listOpenStack-operators at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
> --
> Grant Morley
> Cloud Lead
> Absolute DevOps Ltd
> Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
> www.absolutedevops.io grant at absolutedevops.io <grant at absolutedevops.i> 0845
> 874 0580
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170613/8b6bf2b8/attachment.html>