[Openstack] Error Neutron: RTNETLINK answers: File exists
Marcio Prado
marcioprado at marcioprado.eti.br
Thu Aug 9 16:00:20 UTC 2018
Thanks Eugen.
Em 09-08-2018 12:16, Eugen Block escreveu:
> Sorry, somehow I didn't notice your answer and forgot the thread.
>
>> The problem is a wireless TP-link router with the OpenWRT firmware
>> configured with bridge.
>> When I connect this wireless router to the switch with the OpenStack
>> cloud servers, the Linux bridge agent starts to make an error and I
>> lose access to the VMs.
>
> It's good you have a hint to the cause, but I'm afraid I can't help
> you with this. Hopefully someone with more expertise will be able to
> point you to the right direction.
>
> Regards
>
>
> Zitat von Marcio Prado <marcioprado at marcioprado.eti.br>:
>
>> Guys, I figured out part of the problem.
>>
>> The problem is a wireless TP-link router with the OpenWRT firmware
>> configured with bridge.
>>
>> When I connect this wireless router to the switch with the OpenStack
>> cloud servers, the Linux bridge agent starts to make an error and I
>> lose access to the VMs.
>>
>> It is not duplicate IP or DHCP.
>>
>> Does anyone have any idea what it is?
>>
>>
>>
>>
>> Em 27-07-2018 08:32, Marcio Prado escreveu:
>>> Thanks for the help Eugen,
>>>
>>> This log is from the linuxbridge of the controller node. Compute
>>> nodes
>>> are not logging errors.
>>>
>>> Follows the output of the "openstack network agent list"
>>>
>>> +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
>>> | ID | Agent Type | Host
>>> | Availability Zone | Alive | State | Binary |
>>> +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
>>> | 590f5a6d-379b-4e8d-87ec-f1060cecf230 | Linux bridge agent |
>>> controller | None | True | UP |
>>> neutron-linuxbridge-agent |
>>> | 88fb87c9-4c03-4faa-8286-95be3586fc94 | DHCP agent |
>>> controller | nova | True | UP | neutron-dhcp-agent
>>> |
>>> | b982382e-438c-46a9-8d4e-d58d554150fd | Linux bridge agent |
>>> compute1
>>> | None | True | UP | neutron-linuxbridge-agent |
>>> | c7a9ba41-1fae-46cd-b61f-30bcacb0a4e8 | L3 agent |
>>> controller | nova | True | UP | neutron-l3-agent
>>> |
>>> | c9a1ea4b-2d5d-4bda-9849-cd6e302a2917 | Metadata agent |
>>> controller | None | True | UP |
>>> neutron-metadata-agent |
>>> | e690d4b9-9285-4ddd-a87a-f28ea99d9a73 | Linux bridge agent |
>>> compute3
>>> | None | False | UP | neutron-linuxbridge-agent |
>>> | fdd8f615-f5d6-4100-826e-59f8270df715 | Linux bridge agent |
>>> compute2
>>> | None | False | UP | neutron-linuxbridge-agent |
>>> +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
>>>
>>> compute2 and compute3 are turned off intentionally.
>>>
>>> Log compute1
>>>
>>> /var/log/neutron/neutron-linuxbridge-agent.log
>>>
>>> 2018-07-27 07:43:57.242 1895 INFO neutron.common.config [-]
>>> /usr/bin/neutron-linuxbridge-agent version 10.0.0
>>> 2018-07-27 07:43:57.243 1895 INFO
>>> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent
>>> [-] Interface mappings: {'provider': 'eno3'}
>>> 2018-07-27 07:43:57.243 1895 INFO
>>> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent
>>> [-] Bridge mappings: {}
>>> 2018-07-27 07:44:00.954 1895 INFO
>>> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent
>>> [-] Agent initialized successfully, now running...
>>> 2018-07-27 07:44:01.582 1895 INFO
>>> neutron.plugins.ml2.drivers.agent._common_agent
>>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] RPC agent_id:
>>> lb525400d52f59
>>> 2018-07-27 07:44:01.589 1895 INFO
>>> neutron.agent.agent_extensions_manager
>>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Loaded agent
>>> extensions: []
>>> 2018-07-27 07:44:01.716 1895 INFO
>>> neutron.plugins.ml2.drivers.agent._common_agent [-] Linux bridge
>>> agent
>>> Agent has just been revived. Doing a full sync.
>>> 2018-07-27 07:44:01.778 1895 INFO
>>> neutron.plugins.ml2.drivers.agent._common_agent
>>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Linux bridge
>>> agent Agent RPC Daemon Started!
>>> 2018-07-27 07:44:01.779 1895 INFO
>>> neutron.plugins.ml2.drivers.agent._common_agent
>>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Linux bridge
>>> agent Agent out of sync with plugin!
>>> 2018-07-27 07:44:02.418 1895 INFO
>>> neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect
>>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Clearing
>>> orphaned
>>> ARP spoofing entries for devices []
>>>
>>>
>>> I'm using this OpenStack cloud to run my master's experiment. I
>>> turned
>>> off all nodes, and after a few days I called again and from that the
>>> VMs were not remotely accessible.
>>>
>>> So I delete existing networks and re-create. It was in an attempt to
>>> solve the problem.
>>>
>>> Here is an attached image. Neutron is creating multiple interfaces on
>>> the 10.0.0.0 network on the router.
>>>
>>>
>>> Em 27-07-2018 05:05, Eugen Block escreveu:
>>>> Hi,
>>>>
>>>> is there anything in the linuxbridge-agent logs on control and/or
>>>> compute node(s)?
>>>> Which neutron services don't start? Can you paste "openstack network
>>>> agent list" output?
>>>>
>>>> The important question is: what was the cause of "neutron stopped
>>>> working" and why did you delete the existing networks? It probably
>>>> would be helpful knowing the reaseon to be able to prevent such
>>>> problemes in the future. Or are the provided logs from before?
>>>>
>>>> We experience network/neutron troubles from time to time, and
>>>> sometimes the only way to fix it is a reboot.
>>>>
>>>> Regards,
>>>> Eugen
>>>>
>>>>
>>>> Zitat von Marcio Prado <marcioprado at marcioprado.eti.br>:
>>>>
>>>>> Good afternoon,
>>>>>
>>>>> For no apparent reason my Neutron stopped working.
>>>>>
>>>>> I deleted the networks, subnets and routers, created everything
>>>>> again.
>>>>>
>>>>> But it does not work. The logs are:
>>>>>
>>>>>
>>>>> 2018-07-26 11:29:16.101 3272 INFO
>>>>> neutron.plugins.ml2.drivers.agent._common_agent
>>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Linux bridge
>>>>> agent Agent out of sync with plugin!
>>>>> 2018-07-26 11:29:16.101 3272 INFO neutron.agent.securitygroups_rpc
>>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Preparing
>>>>> filters for devices set(['tap69feb7be-2b', 'tap0efd5228-b0',
>>>>> 'tap83a57ce5-a8', 'tapd50d137f-f6'])
>>>>> 2018-07-26 11:29:18.218 3272 INFO
>>>>> neutron.plugins.ml2.drivers.agent._common_agent
>>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Port
>>>>> tap69feb7be-2b updated. Details: {u'profile': {},
>>>>> u'network_qos_policy_id': None, u'qos_policy_id': None,
>>>>> u'allowed_address_pairs': [], u'admin_state_up': True,
>>>>> u'network_id': u'0f293447-ad01-465e-a034-fdaa136a4488',
>>>>> u'segmentation_id': None, u'device_owner':
>>>>> u'network:router_gateway', u'physical_network': u'provider',
>>>>> u'mac_address': u'fa:16:3e:a3:be:5c', u'device':
>>>>> u'tap69feb7be-2b', u'port_security_enabled': False, u'port_id':
>>>>> u'69feb7be-2b9c-4604-a078-32c984d7075a', u'fixed_ips':
>>>>> [{u'subnet_id': u'5ef3df97-d88a-4c60-969c-5a862f04c1e0',
>>>>> u'ip_address': u'192.168.0.14'}], u'network_type': u'flat'}
>>>>> 2018-07-26 11:29:18.871 3272 INFO
>>>>> neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect
>>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Skipping ARP
>>>>> spoofing rules for port 'tap69feb7be-2b' because it has port
>>>>> security disabled
>>>>> 2018-07-26 11:29:20.208 3272 ERROR neutron.agent.linux.utils
>>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Exit code: 2;
>>>>> Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists
>>>>>
>>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>>> neutron.plugins.ml2.drivers.agent._common_agent
>>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Error in
>>>>> agent loop. Devices info: {'current': set(['tap69feb7be-2b',
>>>>> 'tap0efd5228-b0', 'tap83a57ce5-a8', 'tapd50d137f-f6']),
>>>>> 'timestamps': {'tap0efd5228-b0': 9, 'tap69feb7be-2b': 13,
>>>>> 'tap83a57ce5-a8': 10, 'tapd50d137f-f6': 8}, 'removed': set([]),
>>>>> 'added': set(['tap69feb7be-2b', 'tap0efd5228-b0',
>>>>> 'tap83a57ce5-a8', 'tapd50d137f-f6']), 'updated': set([])}
>>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>>> neutron.plugins.ml2.drivers.agent._common_agent Traceback (most
>>>>> recent call last):
>>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>>> neutron.plugins.ml2.drivers.agent._common_agent File
>>>>> "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",
>>>>> line 453, in daemon_loop
>>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>>> neutron.plugins.ml2.drivers.agent._common_agent sync =
>>>>> self.process_network_devices(device_info)
>>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>>> neutron.plugins.ml2.drivers.agent._common_agent File
>>>>> "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line
>>>>> 153, in wrapper
>>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>>> neutron.plugins.ml2.drivers.agent._common_agent return
>>>>> f(*args, **kwargs)
>>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>>> neutron.plugins.ml2.drivers.agent._common_agent File
>>>>> "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",
>>>>> line 210, in process_network_devices
>>>>>
>>>>> Has anyone had similar experience?
>>>>>
>>>>> -- Marcio Prado
>>>>> Analista de TI - Infraestrutura e Redes
>>>>> Fone: (35) 9.9821-3561
>>>>> www.marcioprado.eti.br
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list:
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>> Post to : openstack at lists.openstack.org
>>>>> Unsubscribe :
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to : openstack at lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>> -- Marcio Prado
>> Analista de TI - Infraestrutura e Redes
>> Fone: (35) 9.9821-3561
>> www.marcioprado.eti.br
--
Marcio Prado
Analista de TI - Infraestrutura e Redes
Fone: (35) 9.9821-3561
www.marcioprado.eti.br
More information about the Openstack
mailing list