Open Stack

Please, file a bug about that on https://bugs.launchpad.net/neutron/+bugs
with tag "l3-ha".

On Thu, Nov 30, 2017 at 6:00 AM Ajay Kalambur (akalambu) <akalambu at cisco.com>
wrote:

> I noticed that this happens when the router HA interface shows status:
> Down what can cause the ha interface to do down
>
> Ajay
>
>
> From: Ajay Kalambur <akalambu at cisco.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Date: Wednesday, November 29, 2017 at 4:14 PM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Subject: [openstack-dev] [neutron] Neutron L3 agent/Keepalived
>
> Hi
> I have a case where after running the system for a while I see that
> floating ip association API gets accepted and no Errors in neutron l3 logs
> But when I go into the qrouter namespace and check the qg- interface the
> floating ip is not added
> Also the keepalived.conf is not updated and SIGUP of the keepalived
> process is not done
>
> So whats the place to look in this case.  What is the flow in neutron l3
> agent who adds the floating ip to the qg- interface
> I see the router update notification received
> Key log snippet attached. As you can see SIGUP of keepalived is missing
> and also I confirmed keepalived configs are not updated
>
>
> 2017-11-29 14:32:08.883 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
> received message with unique_id: b3886b998c3049e799170bb5351300d1 __call__
> /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:269
> 2017-11-29 14:32:08.885 78 DEBUG neutron.agent.l3.agent
> [req-6c505d32-2d1a-4392-8ea3-0bb888397b9e e759eebecc4648b4964d4ecf439dc0ff
> 13b4f6fb188048ba9bbf211344e3342f - - -] Got routers updated notification
> :[u'a1a59e5f-d74e-404d-a3aa-1667c4442aed'] routers_updated
> /usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py:413
> 2017-11-29 14:32:08.886 78 DEBUG neutron.agent.l3.agent [-] Starting
> router update for a1a59e5f-d74e-404d-a3aa-1667c4442aed, action None,
> priority 0 _process_router_update
> /usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py:487
> 2017-11-29 14:32:08.886 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
> CALL msg_id: 92eb015c8c84489681b1efbe949fab8b exchange 'neutron' topic
> 'q-l3-plugin' _send
> /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:568
>
> 2017-11-29 14:32:09.571 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
> received reply msg_id: 92eb015c8c84489681b1efbe949fab8b __call__
> /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:416
> 2017-11-29 14:32:09.571 78 DEBUG neutron.callbacks.manager [-] Notify
> callbacks [] for router, before_update _notify_loop
> /usr/lib/python2.7/site-packages/neutron/callbacks/manager.py:142
> 2017-11-29 14:32:09.572 78 DEBUG neutron.agent.l3.router_info [-] process
> router updates process
> /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:1105
> 2017-11-29 14:32:09.572 78 DEBUG neutron.agent.linux.utils [-] Running
> command (rootwrap daemon): ['ip', 'netns', 'exec',
> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'find', '/sys/class/net',
> '-maxdepth', '1', '-type', 'l', '-printf', '%f '] execute_rootwrap_daemon
> /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105
> 2017-11-29 14:32:09.709 78 DEBUG neutron.agent.linux.utils [-] Exit code:
> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150
> 2017-11-29 14:32:09.710 78 DEBUG oslo_concurrency.lockutils [-] Lock
> "l3-agent-pd" acquired by "neutron.agent.linux.pd.sync_router" :: waited
> 0.000s inner
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:270
> 2017-11-29 14:32:09.710 78 DEBUG oslo_concurrency.lockutils [-] Lock
> "l3-agent-pd" released by "neutron.agent.linux.pd.sync_router" :: held
> 0.000s inner
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282
> 2017-11-29 14:32:09.710 78 DEBUG neutron.agent.linux.utils [-] Running
> command (rootwrap daemon): ['ip', 'netns', 'exec',
> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'find', '/sys/class/net',
> '-maxdepth', '1', '-type', 'l', '-printf', '%f '] execute_rootwrap_daemon
> /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105
> 2017-11-29 14:32:09.845 78 DEBUG neutron.agent.linux.utils [-] Exit code:
> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150
> 2017-11-29 14:32:09.846 78 DEBUG oslo_concurrency.lockutils [-] Acquired
> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
> 2017-11-29 14:32:09.846 78 DEBUG neutron.agent.linux.utils [-] Running
> command (rootwrap daemon): ['ip', 'netns', 'exec',
> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-save']
> execute_rootwrap_daemon
> /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105
> 2017-11-29 14:32:09.983 78 DEBUG neutron.agent.linux.utils [-] Exit code:
> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150
> 2017-11-29 14:32:09.986 78 DEBUG neutron.agent.linux.utils [-] Running
> command (rootwrap daemon): ['ip', 'netns', 'exec',
> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-restore', '-n']
> execute_rootwrap_daemon
> /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105
> 2017-11-29 14:32:10.120 78 DEBUG neutron.agent.linux.utils [-] Exit code:
> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150
> 2017-11-29 14:32:10.120 78 DEBUG neutron.agent.linux.iptables_manager [-]
> IPTablesManager.apply completed with success. 7 iptables commands were
> issued _apply_synchronized
> /usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py:576
> 2017-11-29 14:32:10.121 78 DEBUG oslo_concurrency.lockutils [-] Releasing
> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
> *2017-11-29 14:32:10.121 78 DEBUG neutron.agent.l3.router_info [-]
> Floating ip 5966c8b3-ff5c-4f01-91a6-f81d215bdeca added, status ACTIVE
> process_floating_ip_addresses
> /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:347*
> *2017-11-29 14:32:10.122 78 DEBUG neutron.agent.l3.router_info [-] Sending
> floating ip statuses: {u'5966c8b3-ff5c-4f01-91a6-f81d215bdeca': 'ACTIVE'}
> update_fip_statuses
> /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:915*
> 2017-11-29 14:32:10.122 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
> CALL msg_id: 18e6ceffec9549a5bd51f65551b65596 exchange 'neutron' topic
> 'q-l3-plugin' _send
> /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:568
> 2017-11-29 14:32:10.150 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
> received reply msg_id: 18e6ceffec9549a5bd51f65551b65596 __call__
> /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:416
> 2017-11-29 14:32:10.151 78 DEBUG oslo_concurrency.lockutils [-] Acquired
> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
> 2017-11-29 14:32:10.151 78 DEBUG neutron.agent.linux.utils [-] Running
> command (rootwrap daemon): ['ip', 'netns', 'exec',
> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-save']
> execute_rootwrap_daemon
> /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105
> 2017-11-29 14:32:10.301 78 DEBUG neutron.agent.linux.utils [-] Exit code:
> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150
> 2017-11-29 14:32:10.304 78 DEBUG neutron.agent.linux.iptables_manager [-]
> IPTablesManager.apply completed with success. 0 iptables commands were
> issued _apply_synchronized
> /usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py:576
> 2017-11-29 14:32:10.304 78 DEBUG oslo_concurrency.lockutils [-] Releasing
> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
> 2017-11-29 14:32:10.304 78 DEBUG neutron.callbacks.manager [-] Notify
> callbacks [('neutron.agent.linux.pd.update_router--9223372036851355335',
> <function update_router at 0x3431398>),
> ('neutron.agent.metadata.driver.after_router_updated-3681661', <function
> after_router_updated at 0x382d7d0>)] for router, after_update _notify_loop
> /usr/lib/python2.7/site-packages/neutron/callbacks/manager.py:142
> 2017-11-29 14:32:10.305 78 DEBUG oslo_concurrency.lockutils [-] Lock
> "l3-agent-pd" acquired by "neutron.agent.linux.pd.update_router" :: waited
> 0.000s inner
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:270
> 2017-11-29 14:32:10.305 78 DEBUG oslo_concurrency.lockutils [-] Lock
> "l3-agent-pd" released by "neutron.agent.linux.pd.update_router" :: held
> 0.000s inner
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282
> 2017-11-29 14:32:10.305 78 DEBUG neutron.agent.l3.agent [-] Finished a
> router update for a1a59e5f-d74e-404d-a3aa-1667c4442aed
> _process_router_update
> /usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py:536
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 
Regards,
Ann Taraday
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171204/a72c0253/attachment.html>