Open Stack

On Thu, Aug 6, 2015 at 5:23 PM, Korzeniewski, Artur <
artur.korzeniewski at intel.com> wrote:

> Thanks Kevin for that hint.
>
> But it does not resolve the connectivity problem, it is just not removing
> the namespace when it is asked to.
>
> The real question is, why do we invoke the /neutron/neutron/agent/l3/dvr_fip_ns.py
> FipNamespace.delete() method in the first place?
>
>
>
> I’ve captured the traceback for this situation:
>
> 2015-08-06 06:35:28.469 DEBUG neutron.agent.linux.utils [-] Unable to
> access
> /opt/openstack/data/neutron/external/pids/8223e12e-837b-49d4-9793-63603fccbc9f.pid
> from (pid=70216) get_value_from_file
> /opt/openstack/neutron/neutron/agent/linux/utils.py:222
>
> 2015-08-06 06:35:28.469 DEBUG neutron.agent.linux.utils [-] Unable to
> access
> /opt/openstack/data/neutron/external/pids/8223e12e-837b-49d4-9793-63603fccbc9f.pid
> from (pid=70216) get_value_from_file
> /opt/openstack/neutron/neutron/agent/linux/utils.py:222
>
> 2015-08-06 06:35:28.469 DEBUG neutron.agent.linux.external_process [-] No
> process started for 8223e12e-837b-49d4-9793-63603fccbc9f from (pid=70216)
> disable /opt/openstack/neutron/neutron/agent/linux/external_process.py:113
>
> Traceback (most recent call last):
>
>  File "/usr/local/lib/python2.7/dist-packages/eventlet/queue.py", line
> 117, in switch
>
>     self.greenlet.switch(value)
>
>   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py",
> line 214, in main
>
>     result = function(*args, **kwargs)
>
>   File "/usr/local/lib/python2.7/dist-packages/oslo_service/service.py",
> line 612, in run_service
>
>     service.start()
>
>   File "/opt/openstack/neutron/neutron/service.py", line 233, in start
>
>     self.manager.after_start()
>
>   File "/opt/openstack/neutron/neutron/agent/l3/agent.py", line 641, in
> after_start
>
>     self.periodic_sync_routers_task(self.context)
>
>   File "/opt/openstack/neutron/neutron/agent/l3/agent.py", line 519, in
> periodic_sync_routers_task
>
>     self.fetch_and_sync_all_routers(context, ns_manager)
>
>   File "/opt/openstack/neutron/neutron/agent/l3/namespace_manager.py",
> line 91, in __exit__
>
>     self._cleanup(_ns_prefix, ns_id)
>
>   File "/opt/openstack/neutron/neutron/agent/l3/namespace_manager.py",
> line 140, in _cleanup
>
>     ns.delete()
>
>   File "/opt/openstack/neutron/neutron/agent/l3/dvr_fip_ns.py", line 147,
> in delete
>
>     raise TypeError("ss")
>
> TypeError: ss
>
>
>
> It seems that the fip namespace is not processed at startup of L3 agent,
> and the cleanup is removing the namespace…
>
> It is also removing the interface to local dvr router connection so… VM
> has no internet access with floating IP:
>
> Command: ['ip', 'netns', 'exec',
> 'fip-8223e12e-837b-49d4-9793-63603fccbc9f', 'ip', 'link', 'del',
> u'fpr-fe517b4b-d']
>
>
>
> If the interface inside the fip namespace is not deleted, the VM has full
> internet access without any downtime.
>
>
>
> Ca we consider it a bug? I guess it is something in startup/full-sync
> logic since the log is saying:
>
>
> /opt/openstack/data/neutron/external/pids/8223e12e-837b-49d4-9793-63603fccbc9f.pid
>

I think yes, we can consider it a bug. Can you please file one? I can take
and probably fix it.


>
>
> And after finishing the sync loop, the fip namespace is deleted…
>
>
>
> Regards,
>
> Artur
>
>
>
> *From:* Kevin Benton [mailto:blak111 at gmail.com]
> *Sent:* Thursday, August 6, 2015 7:40 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [neutron][dvr] Removing fip namespace when
> restarting L3 agent.
>
>
>
> Can you try setting the following to False:
>
>
> https://github.com/openstack/neutron/blob/dc0944f2d4e347922054bba679ba7f5d1ae6ffe2/etc/l3_agent.ini#L97
>
>
>
> On Wed, Aug 5, 2015 at 3:36 PM, Korzeniewski, Artur <
> artur.korzeniewski at intel.com> wrote:
>
> Hi all,
>
> During testing of Neutron upgrades, I have found that restarting the L3
> agent in DVR mode is causing the VM network downtime for configured
> floating IP.
>
> The lockdown is visible when pinging the VM from external network, 2-3
> pings are lost.
>
> The responsible place in code is:
>
> DVR: destroy fip ns: fip-8223e12e-837b-49d4-9793-63603fccbc9f from
> (pid=156888) delete
> /opt/openstack/neutron/neutron/agent/l3/dvr_fip_ns.py:164
>
>
>
> Can someone explain why the fip namespace is deleted? Can we workout the
> situation, when there is no downtime of VM access?
>
>
>
> Artur Korzeniewski
>
> --------------------------------------------
>
> Intel Technology Poland sp. z o.o.
>
> KRS 101882
>
> ul. Slowackiego 173, 80-298 Gdansk
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
> --
>
> Kevin Benton
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150806/91f307e8/attachment.html>