Open Stack

Hi,

It will be very helpful if any developers involved in Neutron can throw
some light about cases when a router's internal interface can be deleted
after being added. Please refer to the issue below.

Thanks,
Balu


---------- Forwarded message ----------
From: Balamurugan V G <balamuruganvg at gmail.com>
Date: Tue, Oct 22, 2013 at 7:51 AM
Subject: [OpenStack:Networking] Router Internal interface goes down
sometimes
To: openstack at lists.openstack.org


   Hi,

>
> I have a Grizzly 2013.1.3 4 nodes setup(Controller, Network, Compute1,
> Compute2) on Ubuntu 12.04 LTS.
>
> I launch an instance which is attached to two networks(subnets) as shown
> in the diagram linked below. The subnets are in turn attached to two
> separate routers. Some times I see that the internal interface on one of
> the router remains DOWN (shown as red dot in the diagram below). Can
> someone help me with why this is happening. As said this only happens once
> in a while making the deployment unreliable.
>
>
>
> https://dl.dropboxusercontent.com/u/5145369/OpenStack/NetworkTopology_RouterInternalPortIssue.png
> (Topology Diagram)
>

    Please note that the provisioning of the networks, routers, instances
are automated and happen successively with no delay between operations.

>
> When i look at the logs, the tap interfaces corresponding to the port
> which is DOWN is removed soon after its added. The ones highlighted in
> green correspond the internal interface which is good and the ones in
> yellow correspond to the problem port.
>
> /var/log/syslog
>

    8bc4ac6b - Interface which is fine
    09d96215  - Interface which is DOWN

>
>
> Oct 21 10:07:13 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qr-8bc4ac6b-90 --
> set Interface qr-8bc4ac6b-90 type=internal -- set Interface qr-8bc4ac6b-90
> external-ids:iface-id=8bc4ac6b-9027-4eee-9e1e-7af7b9a9c0df -- set Interface
> qr-8bc4ac6b-90 external-ids:iface-status=active -- set Interface
> qr-8bc4ac6b-90 external-ids:attached-mac=fa:16:3e:02:3c:1e
>
> Oct 21 10:07:13 openstack-blr-network kernel: [  401.933281] device
> qr-8bc4ac6b-90 entered promiscuous mode
>
> Oct 21 10:07:15 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl --timeout=2 set Port qr-8bc4ac6b-90 tag=5
>
> Oct 21 10:07:20 openstack-blr-network kernel: [  408.144038]
> qg-f1b0e315-df: no IPv6 routers present
>
> Oct 21 10:07:20 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qr-09d96215-c4 --
> set Interface qr-09d96215-c4 type=internal -- set Interface qr-09d96215-c4
> external-ids:iface-id=09d96215-c4e8-40a1-afc0-9c3904474b73 -- set Interface
> qr-09d96215-c4 external-ids:iface-status=active -- set Interface
> qr-09d96215-c4 external-ids:attached-mac=fa:16:3e:56:9d:29
>
> Oct 21 10:07:20 openstack-blr-network kernel: [  408.329781] device qr-
> 09d96215-c4 entered promiscuous mode
>
> Oct 21 10:07:20 openstack-blr-network kernel: [  408.640042]
> tap5507f5e7-ea: no IPv6 routers present
>
> Oct 21 10:07:21 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl --timeout=2 set Port qr-09d96215-c4 tag=4
>
> Oct 21 10:07:22 openstack-blr-network kernel: [  410.280038]
> tap44bede85-d8: no IPv6 routers present
>
> Oct 21 10:07:24 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl -- --may-exist add-port br-ex qg-4414a87a-f1 -- set
> Interface qg-4414a87a-f1 type=internal -- set Interface qg-4414a87a-f1
> external-ids:iface-id=4414a87a-f166-426c-93ab-b51d9e8253c1 -- set Interface
> qg-4414a87a-f1 external-ids:iface-status=active -- set Interface
> qg-4414a87a-f1 external-ids:attached-mac=fa:16:3e:94:c0:dd
>
> Oct 21 10:07:24 openstack-blr-network kernel: [  412.883270] device
> qg-4414a87a-f1 entered promiscuous mode
>
> Oct 21 10:07:25 openstack-blr-network kernel: [  413.296045] qr-8bc4ac6b-90:
> no IPv6 routers present
>
> Oct 21 10:07:29 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-int qr-
> 09d96215-c4
>
> Oct 21 10:07:29 openstack-blr-network kernel: [  417.399603] device qr-
> 09d96215-c4 left promiscuous mode
>
> Oct 21 10:07:35 openstack-blr-network kernel: [  423.800021]
> qg-4414a87a-f1: no IPv6 routers present
>
> Oct 21 10:07:45 openstack-blr-network dnsmasq-dhcp[3861]:
> DHCPREQUEST(tap3faa7b42-37) 192.168.2.2 fa:16:3e:81:7a:58
>
> Oct 21 10:07:45 openstack-blr-network dnsmasq-dhcp[3861]:
> DHCPACK(tap3faa7b42-37) 192.168.2.2 fa:16:3e:81:7a:58 host-192-168-2-2
>
> Oct 21 10:08:16 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl -- --may-exist add-port br-ex qg-4f64cb40-a6 -- set
> Interface qg-4f64cb40-a6 type=internal -- set Interface qg-4f64cb40-a6
> external-ids:iface-id=4f64cb40-a63a-47cc-935d-950e99212304 -- set Interface
> qg-4f64cb40-a6 external-ids:iface-status=active -- set Interface
> qg-4f64cb40-a6 external-ids:attached-mac=fa:16:3e:4d:44:75
>
> Oct 21 10:08:16 openstack-blr-network kernel: [  464.342581] device
> qg-4f64cb40-a6 entered promiscuous mode
>
> Oct 21 10:08:17 openstack-blr-network ovs-vsctl: 00001|vsctl|INFO|Called
> as /usr/bin/ovs-vsctl -- --may-exist add-port br-int tapacf9db3a-2c -- set
> Interface tapacf9db3a-2c type=internal -- set Interface tapacf9db3a-2c
> external-ids:iface-id=acf9db3a-2cae-45d8-97c2-02dde4ddeded -- set Interface
> tapacf9db3a-2c external-ids:iface-status=active -- set Interface
> tapacf9db3a-2c external-ids:attached-mac=fa:16:3e:1b:f7:f5
>
> Oct 21 10:08:17 openstack-blr-network kernel: [  465.064522] device
> tapacf9db3a-2c entered promiscuous mode
>
>
>
>
>
>
>
>
>
> /var/log/quantum/openvswitch-agent.log
>
>
>
    8bc4ac6b-9027-4eee-9e1e-7af7b9a9c0df   - Port which is fine
    09d96215-c4e8-40a1-afc0-9c3904474b73  - Port which is DOWN

>
>
> 2013-10-21 10:07:11     INFO [quantum.agent.securitygroups_rpc] Refresh
> firewall rules
>
> 2013-10-21 10:07:11     INFO [quantum.agent.securitygroups_rpc] Preparing
> filters for devices set([u'44bede85-d873-493b-a9a3-79a5d7f55174',
> u'5507f5e7-ea9f-42b6-96eb-6d1984619c27'])
>
> 2013-10-21 10:07:11     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 44bede85-d873-493b-a9a3-79a5d7f55174 added
>
> 2013-10-21 10:07:11     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 44bede85-d873-493b-a9a3-79a5d7f55174 updated. Details: {u'admin_state_up':
> True, u'network_id': u'c5c34bc4-f995-412d-923f-678d62f49361',
> u'segmentation_id': 10, u'physical_network': u'physnet1', u'device':
> u'44bede85-d873-493b-a9a3-79a5d7f55174', u'port_id':
> u'44bede85-d873-493b-a9a3-79a5d7f55174', u'network_type': u'vlan'}
>
> 2013-10-21 10:07:11     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Assigning 4 as local
> vlan for net-id=c5c34bc4-f995-412d-923f-678d62f49361
>
> 2013-10-21 10:07:12     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 5507f5e7-ea9f-42b6-96eb-6d1984619c27 added
>
> 2013-10-21 10:07:12     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 5507f5e7-ea9f-42b6-96eb-6d1984619c27 updated. Details: {u'admin_state_up':
> True, u'network_id': u'956f3bfc-9cc2-4af6-8328-f5a359551671',
> u'segmentation_id': 11, u'physical_network': u'physnet1', u'device':
> u'5507f5e7-ea9f-42b6-96eb-6d1984619c27', u'port_id':
> u'5507f5e7-ea9f-42b6-96eb-6d1984619c27', u'network_type': u'vlan'}
>
> 2013-10-21 10:07:12     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Assigning 5 as local
> vlan for net-id=956f3bfc-9cc2-4af6-8328-f5a359551671
>
> 2013-10-21 10:07:15     INFO [quantum.agent.securitygroups_rpc] Preparing
> filters for devices set([u'8bc4ac6b-9027-4eee-9e1e-7af7b9a9c0df'])
>
> 2013-10-21 10:07:15     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 8bc4ac6b-9027-4eee-9e1e-7af7b9a9c0df added
>
> 2013-10-21 10:07:15     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 8bc4ac6b-9027-4eee-9e1e-7af7b9a9c0df updated. Details:
> {u'admin_state_up': True, u'network_id':
> u'956f3bfc-9cc2-4af6-8328-f5a359551671', u'segmentation_id': 11,
> u'physical_network': u'physnet1', u'device':
> u'8bc4ac6b-9027-4eee-9e1e-7af7b9a9c0df', u'port_id':
> u'8bc4ac6b-9027-4eee-9e1e-7af7b9a9c0df', u'network_type': u'vlan'}
>
> 2013-10-21 10:07:21     INFO [quantum.agent.securitygroups_rpc] Preparing
> filters for devices set([u'09d96215-c4e8-40a1-afc0-9c3904474b73'])
>
> 2013-10-21 10:07:21     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 09d96215-c4e8-40a1-afc0-9c3904474b73 added
>
> 2013-10-21 10:07:21     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 09d96215-c4e8-40a1-afc0-9c3904474b73 updated. Details:
> {u'admin_state_up': True, u'network_id':
> u'c5c34bc4-f995-412d-923f-678d62f49361', u'segmentation_id': 10,
> u'physical_network': u'physnet1', u'device':
> u'09d96215-c4e8-40a1-afc0-9c3904474b73', u'port_id':
> u'09d96215-c4e8-40a1-afc0-9c3904474b73', u'network_type': u'vlan'}
>
> 2013-10-21 10:07:29     INFO [quantum.agent.securitygroups_rpc] Remove
> device filter for set([u'09d96215-c4e8-40a1-afc0-9c3904474b73'])
>
> 2013-10-21 10:07:29     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Attachment
> 09d96215-c4e8-40a1-afc0-9c3904474b73 removed
>
> 2013-10-21 10:07:29     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 09d96215-c4e8-40a1-afc0-9c3904474b73 updated.
>
> 2013-10-21 10:08:16     INFO [quantum.agent.securitygroups_rpc] Security
> group member updated [u'7a952c14-45a6-4f5f-b8f1-551c3cfbb8ba']
>
> 2013-10-21 10:08:16     INFO [quantum.agent.securitygroups_rpc] Security
> group member updated [u'7a952c14-45a6-4f5f-b8f1-551c3cfbb8ba']
>
> 2013-10-21 10:08:16     INFO [quantum.agent.securitygroups_rpc] Provider
> rule updated
>
> 2013-10-21 10:08:16     INFO [quantum.agent.securitygroups_rpc] Refresh
> firewall rules
>
> 2013-10-21 10:08:17     INFO [quantum.agent.securitygroups_rpc] Preparing
> filters for devices set([u'acf9db3a-2cae-45d8-97c2-02dde4ddeded'])
>
> 2013-10-21 10:08:17     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> acf9db3a-2cae-45d8-97c2-02dde4ddeded added
>
> 2013-10-21 10:08:17     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> acf9db3a-2cae-45d8-97c2-02dde4ddeded updated. Details: {u'admin_state_up':
> True, u'network_id': u'5da5b3cd-81e0-45e0-95d6-4f29954bd4b1',
> u'segmentation_id': 13, u'physical_network': u'physnet1', u'device':
> u'acf9db3a-2cae-45d8-97c2-02dde4ddeded', u'port_id':
> u'acf9db3a-2cae-45d8-97c2-02dde4ddeded', u'network_type': u'vlan'}
>
> 2013-10-21 10:08:17     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Assigning 6 as local
> vlan for net-id=5da5b3cd-81e0-45e0-95d6-4f29954bd4b1
>
> 2013-10-21 10:08:18     INFO [quantum.agent.securitygroups_rpc] Provider
> rule updated
>
> 2013-10-21 10:08:18     INFO [quantum.agent.securitygroups_rpc] Refresh
> firewall rules
>
> 2013-10-21 10:08:19     INFO [quantum.agent.securitygroups_rpc] Preparing
> filters for devices set([u'3cc5f73d-eaff-49b4-9ce9-f482476557e8'])
>
> 2013-10-21 10:08:19     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 3cc5f73d-eaff-49b4-9ce9-f482476557e8 added
>
> 2013-10-21 10:08:19     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 3cc5f73d-eaff-49b4-9ce9-f482476557e8 updated. Details: {u'admin_state_up':
> True, u'network_id': u'1c37a512-1394-40f9-ab26-867d5347ff63',
> u'segmentation_id': 12, u'physical_network': u'physnet1', u'device':
> u'3cc5f73d-eaff-49b4-9ce9-f482476557e8', u'port_id':
> u'3cc5f73d-eaff-49b4-9ce9-f482476557e8', u'network_type': u'vlan'}
>
> 2013-10-21 10:08:19     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Assigning 7 as local
> vlan for net-id=1c37a512-1394-40f9-ab26-867d5347ff63
>
> 2013-10-21 10:08:23     INFO [quantum.agent.securitygroups_rpc] Preparing
> filters for devices set([u'62d43349-b25c-45fb-ac67-e756d889451e'])
>
> 2013-10-21 10:08:23     INFO
> [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Port
> 62d43349-b25c-45fb-ac67-e756d889451e added
>
>
>
> I would like to know why one port is deleted soon after being added. Any
> help will be greatly appreciated.
>
> Regards,
> Balu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131025/07d716bb/attachment.html>