[openstack-dev] [neutron] Neutron L3 agent/Keepalived

Anil Venkata anilvenkata at redhat.com
Tue Dec 5 16:18:57 UTC 2017


Which neutron branch you are using?
May be this bug https://bugs.launchpad.net/neutron/+bug/1723848 is similar
and
corresponding https://review.openstack.org/#/c/512179/ patch was merged in
u/s
and backported to Ocata and pike branches.

Please also check these [1] relevant patches
[1]
https://review.openstack.org/#/q/topic:bug/1597461+(status:open+OR+status:merged)

Thanks
Anilvenkata

On Mon, Dec 4, 2017 at 6:38 PM, Anna Taraday <akamyshnikova at mirantis.com>
wrote:

> Please, file a bug about that on https://bugs.launchpad.net/neutron/+bugs
> with tag "l3-ha".
>
> On Thu, Nov 30, 2017 at 6:00 AM Ajay Kalambur (akalambu) <
> akalambu at cisco.com> wrote:
>
>> I noticed that this happens when the router HA interface shows status:
>> Down what can cause the ha interface to do down
>>
>> Ajay
>>
>>
>> From: Ajay Kalambur <akalambu at cisco.com>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Date: Wednesday, November 29, 2017 at 4:14 PM
>> To: "OpenStack Development Mailing List (not for usage questions)" <
>> openstack-dev at lists.openstack.org>
>> Subject: [openstack-dev] [neutron] Neutron L3 agent/Keepalived
>>
>> Hi
>> I have a case where after running the system for a while I see that
>> floating ip association API gets accepted and no Errors in neutron l3 logs
>> But when I go into the qrouter namespace and check the qg- interface the
>> floating ip is not added
>> Also the keepalived.conf is not updated and SIGUP of the keepalived
>> process is not done
>>
>> So whats the place to look in this case.  What is the flow in neutron l3
>> agent who adds the floating ip to the qg- interface
>> I see the router update notification received
>> Key log snippet attached. As you can see SIGUP of keepalived is missing
>> and also I confirmed keepalived configs are not updated
>>
>>
>> 2017-11-29 14:32:08.883 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
>> received message with unique_id: b3886b998c3049e799170bb5351300d1
>> __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_
>> drivers/amqpdriver.py:269
>> 2017-11-29 14:32:08.885 78 DEBUG neutron.agent.l3.agent
>> [req-6c505d32-2d1a-4392-8ea3-0bb888397b9e e759eebecc4648b4964d4ecf439dc0ff
>> 13b4f6fb188048ba9bbf211344e3342f - - -] Got routers updated notification
>> :[u'a1a59e5f-d74e-404d-a3aa-1667c4442aed'] routers_updated
>> /usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py:413
>> 2017-11-29 14:32:08.886 78 DEBUG neutron.agent.l3.agent [-] Starting
>> router update for a1a59e5f-d74e-404d-a3aa-1667c4442aed, action None,
>> priority 0 _process_router_update /usr/lib/python2.7/site-
>> packages/neutron/agent/l3/agent.py:487
>> 2017-11-29 14:32:08.886 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
>> CALL msg_id: 92eb015c8c84489681b1efbe949fab8b exchange 'neutron' topic
>> 'q-l3-plugin' _send /usr/lib/python2.7/site-packages/oslo_messaging/_
>> drivers/amqpdriver.py:568
>>
>> 2017-11-29 14:32:09.571 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
>> received reply msg_id: 92eb015c8c84489681b1efbe949fab8b __call__
>> /usr/lib/python2.7/site-packages/oslo_messaging/_
>> drivers/amqpdriver.py:416
>> 2017-11-29 14:32:09.571 78 DEBUG neutron.callbacks.manager [-] Notify
>> callbacks [] for router, before_update _notify_loop /usr/lib/python2.7/site-
>> packages/neutron/callbacks/manager.py:142
>> 2017-11-29 14:32:09.572 78 DEBUG neutron.agent.l3.router_info [-] process
>> router updates process /usr/lib/python2.7/site-packages/neutron/agent/l3/
>> router_info.py:1105
>> 2017-11-29 14:32:09.572 78 DEBUG neutron.agent.linux.utils [-] Running
>> command (rootwrap daemon): ['ip', 'netns', 'exec',
>> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'find',
>> '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f ']
>> execute_rootwrap_daemon /usr/lib/python2.7/site-
>> packages/neutron/agent/linux/utils.py:105
>> 2017-11-29 14:32:09.709 78 DEBUG neutron.agent.linux.utils [-] Exit code:
>> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/
>> utils.py:150
>> 2017-11-29 14:32:09.710 78 DEBUG oslo_concurrency.lockutils [-] Lock
>> "l3-agent-pd" acquired by "neutron.agent.linux.pd.sync_router" :: waited
>> 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/
>> lockutils.py:270
>> 2017-11-29 14:32:09.710 78 DEBUG oslo_concurrency.lockutils [-] Lock
>> "l3-agent-pd" released by "neutron.agent.linux.pd.sync_router" :: held
>> 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/
>> lockutils.py:282
>> 2017-11-29 14:32:09.710 78 DEBUG neutron.agent.linux.utils [-] Running
>> command (rootwrap daemon): ['ip', 'netns', 'exec',
>> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'find',
>> '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f ']
>> execute_rootwrap_daemon /usr/lib/python2.7/site-
>> packages/neutron/agent/linux/utils.py:105
>> 2017-11-29 14:32:09.845 78 DEBUG neutron.agent.linux.utils [-] Exit code:
>> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/
>> utils.py:150
>> 2017-11-29 14:32:09.846 78 DEBUG oslo_concurrency.lockutils [-] Acquired
>> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
>> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>> 2017-11-29 14:32:09.846 78 DEBUG neutron.agent.linux.utils [-] Running
>> command (rootwrap daemon): ['ip', 'netns', 'exec',
>> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-save']
>> execute_rootwrap_daemon /usr/lib/python2.7/site-
>> packages/neutron/agent/linux/utils.py:105
>> 2017-11-29 14:32:09.983 78 DEBUG neutron.agent.linux.utils [-] Exit code:
>> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/
>> utils.py:150
>> 2017-11-29 14:32:09.986 78 DEBUG neutron.agent.linux.utils [-] Running
>> command (rootwrap daemon): ['ip', 'netns', 'exec',
>> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-restore',
>> '-n'] execute_rootwrap_daemon /usr/lib/python2.7/site-
>> packages/neutron/agent/linux/utils.py:105
>> 2017-11-29 14:32:10.120 78 DEBUG neutron.agent.linux.utils [-] Exit code:
>> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/
>> utils.py:150
>> 2017-11-29 14:32:10.120 78 DEBUG neutron.agent.linux.iptables_manager
>> [-] IPTablesManager.apply completed with success. 7 iptables commands were
>> issued _apply_synchronized /usr/lib/python2.7/site-
>> packages/neutron/agent/linux/iptables_manager.py:576
>> 2017-11-29 14:32:10.121 78 DEBUG oslo_concurrency.lockutils [-] Releasing
>> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
>> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>> *2017-11-29 14:32:10.121 78 DEBUG neutron.agent.l3.router_info [-]
>> Floating ip 5966c8b3-ff5c-4f01-91a6-f81d215bdeca added, status ACTIVE
>> process_floating_ip_addresses
>> /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:347*
>> *2017-11-29 14:32:10.122 78 DEBUG neutron.agent.l3.router_info [-]
>> Sending floating ip statuses: {u'5966c8b3-ff5c-4f01-91a6-f81d215bdeca':
>> 'ACTIVE'} update_fip_statuses
>> /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:915*
>> 2017-11-29 14:32:10.122 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
>> CALL msg_id: 18e6ceffec9549a5bd51f65551b65596 exchange 'neutron' topic
>> 'q-l3-plugin' _send /usr/lib/python2.7/site-packages/oslo_messaging/_
>> drivers/amqpdriver.py:568
>> 2017-11-29 14:32:10.150 78 DEBUG oslo_messaging._drivers.amqpdriver [-]
>> received reply msg_id: 18e6ceffec9549a5bd51f65551b65596 __call__
>> /usr/lib/python2.7/site-packages/oslo_messaging/_
>> drivers/amqpdriver.py:416
>> 2017-11-29 14:32:10.151 78 DEBUG oslo_concurrency.lockutils [-] Acquired
>> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
>> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>> 2017-11-29 14:32:10.151 78 DEBUG neutron.agent.linux.utils [-] Running
>> command (rootwrap daemon): ['ip', 'netns', 'exec',
>> 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-save']
>> execute_rootwrap_daemon /usr/lib/python2.7/site-
>> packages/neutron/agent/linux/utils.py:105
>> 2017-11-29 14:32:10.301 78 DEBUG neutron.agent.linux.utils [-] Exit code:
>> 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/
>> utils.py:150
>> 2017-11-29 14:32:10.304 78 DEBUG neutron.agent.linux.iptables_manager
>> [-] IPTablesManager.apply completed with success. 0 iptables commands were
>> issued _apply_synchronized /usr/lib/python2.7/site-
>> packages/neutron/agent/linux/iptables_manager.py:576
>> 2017-11-29 14:32:10.304 78 DEBUG oslo_concurrency.lockutils [-] Releasing
>> semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock
>> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>> 2017-11-29 14:32:10.304 78 DEBUG neutron.callbacks.manager [-] Notify
>> callbacks [('neutron.agent.linux.pd.update_router--9223372036851355335',
>> <function update_router at 0x3431398>), ('neutron.agent.metadata.
>> driver.after_router_updated-3681661', <function after_router_updated at
>> 0x382d7d0>)] for router, after_update _notify_loop /usr/lib/python2.7/site-
>> packages/neutron/callbacks/manager.py:142
>> 2017-11-29 14:32:10.305 78 DEBUG oslo_concurrency.lockutils [-] Lock
>> "l3-agent-pd" acquired by "neutron.agent.linux.pd.update_router" ::
>> waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/
>> lockutils.py:270
>> 2017-11-29 14:32:10.305 78 DEBUG oslo_concurrency.lockutils [-] Lock
>> "l3-agent-pd" released by "neutron.agent.linux.pd.update_router" :: held
>> 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/
>> lockutils.py:282
>> 2017-11-29 14:32:10.305 78 DEBUG neutron.agent.l3.agent [-] Finished a
>> router update for a1a59e5f-d74e-404d-a3aa-1667c4442aed
>> _process_router_update /usr/lib/python2.7/site-packages/neutron/agent/l3/
>> agent.py:536
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
>> unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> --
> Regards,
> Ann Taraday
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171205/027aeb54/attachment.html>


More information about the OpenStack-dev mailing list