neutron network namespaces not create
Hi, I can create router from horizon. But network namespaces not created. I check with # ip netns list command. Not found router ID, but showing in horizon. Here is some log I get from neutron: #cat /var/log/neutron/l3-agent.log 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for cad85ce0-6624-42ff-b42b-09480aea2613, action None: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:13.024 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup 2019-05-21 17:37:14.358 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup Thanks & B'Rgds, Rony
Hi, Error which You pasted looks that is related to moment when router is deleted. Do You have any errors in neutron-server and/or neutron-l3-agent logs during creation of the router? Can You check with command “neutron l3-agent-list-hosting-router <router_id>” by which l3 agent router should be hosted? What kind of router Your are creating? Is it HA, DVR, DVR-HA router? Or maybe Legacy?
On 21 May 2019, at 16:51, Md. Farhad Hasan Khan <rony.khan@novotel-bd.com> wrote:
Hi, I can create router from horizon. But network namespaces not created. I check with # ip netns list command. Not found router ID, but showing in horizon.
Here is some log I get from neutron:
#cat /var/log/neutron/l3-agent.log
2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for cad85ce0-6624-42ff-b42b-09480aea2613, action None: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:13.024 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup 2019-05-21 17:37:14.358 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup
Thanks & B’Rgds, Rony
— Slawek Kaplonski Senior software engineer Red Hat
Hi, We are using L3 agent HA. We are creating router in a project from horizon. We didn’t get any error in neutron -server log. Only getting log in l3-agent.log. [root@controller1 neutron]# cat /var/log/neutron/l3-agent.log |grep 2b39246b-5cd5-461b-babb-f37aae38c25f 2019-05-22 11:16:29.041 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:29.111 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:29.111 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:29.112 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:32.223 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:35.035 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:35.103 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:35.103 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:35.104 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:37.590 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:37.656 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:37.656 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:37.657 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:43.952 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:44.022 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:44.022 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:44.023 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:46.491 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:46.550 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:46.550 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:46.551 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:54.852 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:01.427 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:01.494 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:01.494 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:01.495 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:03.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:06.665 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:06.737 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:06.737 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:06.738 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:09.306 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:09.363 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:09.363 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:09.364 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:09.364 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for 2b39246b-5cd5-461b-babb-f37aae38c25f, action None: BridgeDoesNExist: Bridge br-int does not exist. 2019-05-22 11:17:09.365 52248 WARNING neutron.agent.l3.agent [-] Info for router 2b39246b-5cd5-461b-babb-f37aae38c25f was not found. Performing router cleanup: BridgeesNotExist: Bridge br-int does not exist. 2019-05-22 11:17:15.145 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:15.203 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:15.203 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:15.204 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:15.205 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for 2b39246b-5cd5-461b-babb-f37aae38c25f, action None: BridgeDoesNExist: Bridge br-int does not exist. 2019-05-22 11:17:15.205 52248 WARNING neutron.agent.l3.agent [-] Info for router 2b39246b-5cd5-461b-babb-f37aae38c25f was not found. Performing router cleanup: BridgeesNotExist: Bridge br-int does not exist. [root@controller1 neutron]# #cat /var/log/neutron/l3-agent.log |grep 2b39246b-5cd5-461b-babb-f37aae38c25f [root@controller1 neutron]# tailf /var/log/neutron/l3-agent.log 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exist 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent 2019-05-22 12:04:29.356 52248 WARNING neutron.agent.l3.agent [-] Info for router 2cb935c7-a007-4fcc-82ce-7f2a6108b12d was not found. Performing router cleanup 2019-05-22 12:04:33.125 52248 WARNING neutron.agent.l3.agent [-] Info for router 2cb935c7-a007-4fcc-82ce-7f2a6108b12d was not found. Performing router cleanup 2019-05-22 12:24:26.582 52248 WARNING neutron.agent.l3.agent [-] Info for router b9ecd487-336a-4c5f-94c1-2b4fe7079a93 was not found. Performing router cleanup ^Z [2]+ Stopped tailf /var/log/neutron/l3-agent.log -----Original Message----- From: Slawomir Kaplonski [mailto:skaplons@redhat.com] Sent: Wednesday, May 22, 2019 2:13 AM To: Md. Farhad Hasan Khan Cc: openstack-discuss@lists.openstack.org Subject: Re: neutron network namespaces not create Hi, Error which You pasted looks that is related to moment when router is deleted. Do You have any errors in neutron-server and/or neutron-l3-agent logs during creation of the router? Can You check with command “neutron l3-agent-list-hosting-router <router_id>” by which l3 agent router should be hosted? What kind of router Your are creating? Is it HA, DVR, DVR-HA router? Or maybe Legacy?
On 21 May 2019, at 16:51, Md. Farhad Hasan Khan <rony.khan@novotel-bd.com> wrote:
Hi, I can create router from horizon. But network namespaces not created. I check with # ip netns list command. Not found router ID, but showing in horizon.
Here is some log I get from neutron:
#cat /var/log/neutron/l3-agent.log
2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for cad85ce0-6624-42ff-b42b-09480aea2613, action None: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:13.024 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup 2019-05-21 17:37:14.358 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup
Thanks & B’Rgds, Rony
— Slawek Kaplonski Senior software engineer Red Hat
Hi, We are using L3 agent HA. We are creating router in a project from horizon. We didn’t get any error in neutron -server log. Only getting log in l3-agent.log.
From horizon we can see router interfaces HA port status down
[root@controller1 neutron]# cat /var/log/neutron/l3-agent.log |grep 2b39246b-5cd5-461b-babb-f37aae38c25f 2019-05-22 11:16:29.041 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:29.111 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:29.111 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:29.112 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:32.223 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:35.035 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:35.103 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:35.103 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:35.104 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:37.590 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:37.656 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:37.656 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:37.657 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:43.952 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:44.022 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:44.022 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:44.023 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:46.491 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:46.550 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:46.550 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:46.551 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:54.852 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:01.427 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:01.494 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:01.494 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:01.495 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:03.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:06.665 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:06.737 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:06.737 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:06.738 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:09.306 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:09.363 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:09.363 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:09.364 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:09.364 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for 2b39246b-5cd5-461b-babb-f37aae38c25f, action None: BridgeDoesNExist: Bridge br-int does not exist. 2019-05-22 11:17:09.365 52248 WARNING neutron.agent.l3.agent [-] Info for router 2b39246b-5cd5-461b-babb-f37aae38c25f was not found. Performing router cleanup: BridgeesNotExist: Bridge br-int does not exist. 2019-05-22 11:17:15.145 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:15.203 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:15.203 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:15.204 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:15.205 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for 2b39246b-5cd5-461b-babb-f37aae38c25f, action None: BridgeDoesNExist: Bridge br-int does not exist. 2019-05-22 11:17:15.205 52248 WARNING neutron.agent.l3.agent [-] Info for router 2b39246b-5cd5-461b-babb-f37aae38c25f was not found. Performing router cleanup: BridgeesNotExist: Bridge br-int does not exist. [root@controller1 neutron]# #cat /var/log/neutron/l3-agent.log |grep 2b39246b-5cd5-461b-babb-f37aae38c25f [root@controller1 neutron]# tailf /var/log/neutron/l3-agent.log 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exist 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent 2019-05-22 12:04:29.356 52248 WARNING neutron.agent.l3.agent [-] Info for router 2cb935c7-a007-4fcc-82ce-7f2a6108b12d was not found. Performing router cleanup 2019-05-22 12:04:33.125 52248 WARNING neutron.agent.l3.agent [-] Info for router 2cb935c7-a007-4fcc-82ce-7f2a6108b12d was not found. Performing router cleanup 2019-05-22 12:24:26.582 52248 WARNING neutron.agent.l3.agent [-] Info for router b9ecd487-336a-4c5f-94c1-2b4fe7079a93 was not found. Performing router cleanup ^Z [2]+ Stopped tailf /var/log/neutron/l3-agent.log -----Original Message----- From: Slawomir Kaplonski [mailto:skaplons@redhat.com] Sent: Wednesday, May 22, 2019 2:13 AM To: Md. Farhad Hasan Khan Cc: openstack-discuss@lists.openstack.org Subject: Re: neutron network namespaces not create Hi, Error which You pasted looks that is related to moment when router is deleted. Do You have any errors in neutron-server and/or neutron-l3-agent logs during creation of the router? Can You check with command “neutron l3-agent-list-hosting-router <router_id>” by which l3 agent router should be hosted? What kind of router Your are creating? Is it HA, DVR, DVR-HA router? Or maybe Legacy?
On 21 May 2019, at 16:51, Md. Farhad Hasan Khan <rony.khan@novotel-bd.com> wrote:
Hi, I can create router from horizon. But network namespaces not created. I check with # ip netns list command. Not found router ID, but showing in horizon.
Here is some log I get from neutron:
#cat /var/log/neutron/l3-agent.log
2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for cad85ce0-6624-42ff-b42b-09480aea2613, action None: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:13.024 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup 2019-05-21 17:37:14.358 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup
Thanks & B’Rgds, Rony
— Slawek Kaplonski Senior software engineer Red Hat
Hi, According to Michael's email, can You also check if neutron-openvswitch-agent running on Your node. Because br-int should be created on node when neutron-ovs-agent is starting. It’s in [1]. Or maybe if You are using some other solution rather than ML2/OVS, maybe You should change to other than “openvswitch” interface driver in neutron-l3-agent’s config? [1] https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers...
On 22 May 2019, at 08:45, Md. Farhad Hasan Khan <rony.khan@brilliant.com.bd> wrote:
Hi, We are using L3 agent HA. We are creating router in a project from horizon. We didn’t get any error in neutron -server log. Only getting log in l3-agent.log. From horizon we can see router interfaces HA port status down
[root@controller1 neutron]# cat /var/log/neutron/l3-agent.log |grep 2b39246b-5cd5-461b-babb-f37aae38c25f 2019-05-22 11:16:29.041 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:29.111 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:29.111 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:29.112 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:32.223 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:32.280 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:35.035 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:35.103 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:35.103 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:35.104 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:37.590 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:37.656 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:37.656 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:37.657 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:43.952 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:44.022 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:44.022 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:44.023 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:46.491 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:46.550 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:46.550 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:46.551 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:16:54.852 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:16:54.912 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:01.427 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:01.494 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:01.494 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:01.495 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:03.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:04.037 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:06.665 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:06.737 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:06.737 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:06.738 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:09.306 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:09.363 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:09.363 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:09.364 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:09.364 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for 2b39246b-5cd5-461b-babb-f37aae38c25f, action None: BridgeDoesNExist: Bridge br-int does not exist. 2019-05-22 11:17:09.365 52248 WARNING neutron.agent.l3.agent [-] Info for router 2b39246b-5cd5-461b-babb-f37aae38c25f was not found. Performing router cleanup: BridgeesNotExist: Bridge br-int does not exist. 2019-05-22 11:17:15.145 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge br-intoes not exist. 2019-05-22 11:17:15.203 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router 2b39246b-5cd5-461b-babb-f37aae38c25f: OSError: [Errno 2] No such file or dictory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c25f' 2019-05-22 11:17:15.203 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/2b39246b-5cd5-461b-babb-f37aae38c2' 2019-05-22 11:17:15.204 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 2b39246b-5cd5-461b-babb-f37aae38c25f: BridgeDoesNotExist: Bridge bint does not exist. 2019-05-22 11:17:15.205 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for 2b39246b-5cd5-461b-babb-f37aae38c25f, action None: BridgeDoesNExist: Bridge br-int does not exist. 2019-05-22 11:17:15.205 52248 WARNING neutron.agent.l3.agent [-] Info for router 2b39246b-5cd5-461b-babb-f37aae38c25f was not found. Performing router cleanup: BridgeesNotExist: Bridge br-int does not exist. [root@controller1 neutron]# #cat /var/log/neutron/l3-agent.log |grep 2b39246b-5cd5-461b-babb-f37aae38c25f [root@controller1 neutron]# tailf /var/log/neutron/l3-agent.log 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exist 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-22 12:04:28.315 52248 ERROR neutron.agent.l3.agent 2019-05-22 12:04:29.356 52248 WARNING neutron.agent.l3.agent [-] Info for router 2cb935c7-a007-4fcc-82ce-7f2a6108b12d was not found. Performing router cleanup 2019-05-22 12:04:33.125 52248 WARNING neutron.agent.l3.agent [-] Info for router 2cb935c7-a007-4fcc-82ce-7f2a6108b12d was not found. Performing router cleanup 2019-05-22 12:24:26.582 52248 WARNING neutron.agent.l3.agent [-] Info for router b9ecd487-336a-4c5f-94c1-2b4fe7079a93 was not found. Performing router cleanup ^Z [2]+ Stopped tailf /var/log/neutron/l3-agent.log
-----Original Message----- From: Slawomir Kaplonski [mailto:skaplons@redhat.com] Sent: Wednesday, May 22, 2019 2:13 AM To: Md. Farhad Hasan Khan Cc: openstack-discuss@lists.openstack.org Subject: Re: neutron network namespaces not create
Hi,
Error which You pasted looks that is related to moment when router is deleted. Do You have any errors in neutron-server and/or neutron-l3-agent logs during creation of the router? Can You check with command “neutron l3-agent-list-hosting-router <router_id>” by which l3 agent router should be hosted? What kind of router Your are creating? Is it HA, DVR, DVR-HA router? Or maybe Legacy?
On 21 May 2019, at 16:51, Md. Farhad Hasan Khan <rony.khan@novotel-bd.com> wrote:
Hi, I can create router from horizon. But network namespaces not created. I check with # ip netns list command. Not found router ID, but showing in horizon.
Here is some log I get from neutron:
#cat /var/log/neutron/l3-agent.log
2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.646 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:35:44.709 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.710 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for cad85ce0-6624-42ff-b42b-09480aea2613, action None: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:35:44.711 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.705 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:05.761 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:05.762 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:07.975 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:08.029 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.277 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:10.351 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:10.352 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent [-] Error while initializing router cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.575 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Error while deleting router cad85ce0-6624-42ff-b42b-09480aea2613: OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 372, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.delete() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 452, in delete 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.disable_keepalived() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 178, in disable_keepalived 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent shutil.rmtree(conf_dir) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 239, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent onerror(os.listdir, path, sys.exc_info()) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/shutil.py", line 237, in rmtree 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent names = os.listdir(path) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent OSError: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/cad85ce0-6624-42ff-b42b-09480aea2613' 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: cad85ce0-6624-42ff-b42b-09480aea2613: BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 562, in _process_router_update 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 481, in _process_router_if_compatible 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._process_added_router(router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 486, in _process_added_router 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self._router_added(router['id'], router) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 375, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent router_id) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.force_reraise() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 364, in _router_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 128, in initialize 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.ha_network_added() 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 198, in ha_network_added 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent mtu=self.ha_port.get('mtu')) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 263, in plug 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent bridge, namespace, prefix, mtu) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 346, in plug_new 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent self.check_bridge_exists(bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/interface.py", line 221, in check_bridge_exists 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent raise exceptions.BridgeDoesNotExist(bridge=bridge) 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent BridgeDoesNotExist: Bridge br-int does not exist. 2019-05-21 17:37:12.640 52248 ERROR neutron.agent.l3.agent 2019-05-21 17:37:13.024 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup 2019-05-21 17:37:14.358 52248 WARNING neutron.agent.l3.agent [-] Info for router cad85ce0-6624-42ff-b42b-09480aea2613 was not found. Performing router cleanup
Thanks & B’Rgds, Rony
— Slawek Kaplonski Senior software engineer Red Hat
— Slawek Kaplonski Senior software engineer Red Hat
participants (3)
-
Md. Farhad Hasan Khan
-
Md. Farhad Hasan Khan
-
Slawomir Kaplonski