[Openstack] Error Neutron: RTNETLINK answers: File exists
Eugen Block
eblock at nde.ag
Thu Aug 9 15:16:45 UTC 2018
Sorry, somehow I didn't notice your answer and forgot the thread.
> The problem is a wireless TP-link router with the OpenWRT firmware
> configured with bridge.
> When I connect this wireless router to the switch with the OpenStack
> cloud servers, the Linux bridge agent starts to make an error and I
> lose access to the VMs.
It's good you have a hint to the cause, but I'm afraid I can't help
you with this. Hopefully someone with more expertise will be able to
point you to the right direction.
Regards
Zitat von Marcio Prado <marcioprado at marcioprado.eti.br>:
> Guys, I figured out part of the problem.
>
> The problem is a wireless TP-link router with the OpenWRT firmware
> configured with bridge.
>
> When I connect this wireless router to the switch with the OpenStack
> cloud servers, the Linux bridge agent starts to make an error and I
> lose access to the VMs.
>
> It is not duplicate IP or DHCP.
>
> Does anyone have any idea what it is?
>
>
>
>
> Em 27-07-2018 08:32, Marcio Prado escreveu:
>> Thanks for the help Eugen,
>>
>> This log is from the linuxbridge of the controller node. Compute nodes
>> are not logging errors.
>>
>> Follows the output of the "openstack network agent list"
>>
>> +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
>> | ID | Agent Type | Host
>> | Availability Zone | Alive | State | Binary |
>> +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
>> | 590f5a6d-379b-4e8d-87ec-f1060cecf230 | Linux bridge agent |
>> controller | None | True | UP |
>> neutron-linuxbridge-agent |
>> | 88fb87c9-4c03-4faa-8286-95be3586fc94 | DHCP agent |
>> controller | nova | True | UP | neutron-dhcp-agent
>> |
>> | b982382e-438c-46a9-8d4e-d58d554150fd | Linux bridge agent | compute1
>> | None | True | UP | neutron-linuxbridge-agent |
>> | c7a9ba41-1fae-46cd-b61f-30bcacb0a4e8 | L3 agent |
>> controller | nova | True | UP | neutron-l3-agent
>> |
>> | c9a1ea4b-2d5d-4bda-9849-cd6e302a2917 | Metadata agent |
>> controller | None | True | UP |
>> neutron-metadata-agent |
>> | e690d4b9-9285-4ddd-a87a-f28ea99d9a73 | Linux bridge agent | compute3
>> | None | False | UP | neutron-linuxbridge-agent |
>> | fdd8f615-f5d6-4100-826e-59f8270df715 | Linux bridge agent | compute2
>> | None | False | UP | neutron-linuxbridge-agent |
>> +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
>>
>> compute2 and compute3 are turned off intentionally.
>>
>> Log compute1
>>
>> /var/log/neutron/neutron-linuxbridge-agent.log
>>
>> 2018-07-27 07:43:57.242 1895 INFO neutron.common.config [-]
>> /usr/bin/neutron-linuxbridge-agent version 10.0.0
>> 2018-07-27 07:43:57.243 1895 INFO
>> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent
>> [-] Interface mappings: {'provider': 'eno3'}
>> 2018-07-27 07:43:57.243 1895 INFO
>> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent
>> [-] Bridge mappings: {}
>> 2018-07-27 07:44:00.954 1895 INFO
>> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent
>> [-] Agent initialized successfully, now running...
>> 2018-07-27 07:44:01.582 1895 INFO
>> neutron.plugins.ml2.drivers.agent._common_agent
>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] RPC agent_id:
>> lb525400d52f59
>> 2018-07-27 07:44:01.589 1895 INFO
>> neutron.agent.agent_extensions_manager
>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Loaded agent
>> extensions: []
>> 2018-07-27 07:44:01.716 1895 INFO
>> neutron.plugins.ml2.drivers.agent._common_agent [-] Linux bridge agent
>> Agent has just been revived. Doing a full sync.
>> 2018-07-27 07:44:01.778 1895 INFO
>> neutron.plugins.ml2.drivers.agent._common_agent
>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Linux bridge
>> agent Agent RPC Daemon Started!
>> 2018-07-27 07:44:01.779 1895 INFO
>> neutron.plugins.ml2.drivers.agent._common_agent
>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Linux bridge
>> agent Agent out of sync with plugin!
>> 2018-07-27 07:44:02.418 1895 INFO
>> neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect
>> [req-3a8a42dc-32fc-40fc-8a4f-ddbb4d8c5f5b - - - - -] Clearing orphaned
>> ARP spoofing entries for devices []
>>
>>
>> I'm using this OpenStack cloud to run my master's experiment. I turned
>> off all nodes, and after a few days I called again and from that the
>> VMs were not remotely accessible.
>>
>> So I delete existing networks and re-create. It was in an attempt to
>> solve the problem.
>>
>> Here is an attached image. Neutron is creating multiple interfaces on
>> the 10.0.0.0 network on the router.
>>
>>
>> Em 27-07-2018 05:05, Eugen Block escreveu:
>>> Hi,
>>>
>>> is there anything in the linuxbridge-agent logs on control and/or
>>> compute node(s)?
>>> Which neutron services don't start? Can you paste "openstack network
>>> agent list" output?
>>>
>>> The important question is: what was the cause of "neutron stopped
>>> working" and why did you delete the existing networks? It probably
>>> would be helpful knowing the reaseon to be able to prevent such
>>> problemes in the future. Or are the provided logs from before?
>>>
>>> We experience network/neutron troubles from time to time, and
>>> sometimes the only way to fix it is a reboot.
>>>
>>> Regards,
>>> Eugen
>>>
>>>
>>> Zitat von Marcio Prado <marcioprado at marcioprado.eti.br>:
>>>
>>>> Good afternoon,
>>>>
>>>> For no apparent reason my Neutron stopped working.
>>>>
>>>> I deleted the networks, subnets and routers, created everything again.
>>>>
>>>> But it does not work. The logs are:
>>>>
>>>>
>>>> 2018-07-26 11:29:16.101 3272 INFO
>>>> neutron.plugins.ml2.drivers.agent._common_agent
>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Linux bridge
>>>> agent Agent out of sync with plugin!
>>>> 2018-07-26 11:29:16.101 3272 INFO
>>>> neutron.agent.securitygroups_rpc
>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Preparing
>>>> filters for devices set(['tap69feb7be-2b', 'tap0efd5228-b0',
>>>> 'tap83a57ce5-a8', 'tapd50d137f-f6'])
>>>> 2018-07-26 11:29:18.218 3272 INFO
>>>> neutron.plugins.ml2.drivers.agent._common_agent
>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Port
>>>> tap69feb7be-2b updated. Details: {u'profile': {},
>>>> u'network_qos_policy_id': None, u'qos_policy_id': None,
>>>> u'allowed_address_pairs': [], u'admin_state_up': True,
>>>> u'network_id': u'0f293447-ad01-465e-a034-fdaa136a4488',
>>>> u'segmentation_id': None, u'device_owner':
>>>> u'network:router_gateway', u'physical_network': u'provider',
>>>> u'mac_address': u'fa:16:3e:a3:be:5c', u'device':
>>>> u'tap69feb7be-2b', u'port_security_enabled': False, u'port_id':
>>>> u'69feb7be-2b9c-4604-a078-32c984d7075a', u'fixed_ips':
>>>> [{u'subnet_id': u'5ef3df97-d88a-4c60-969c-5a862f04c1e0',
>>>> u'ip_address': u'192.168.0.14'}], u'network_type': u'flat'}
>>>> 2018-07-26 11:29:18.871 3272 INFO
>>>> neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect
>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Skipping ARP
>>>> spoofing rules for port 'tap69feb7be-2b' because it has port
>>>> security disabled
>>>> 2018-07-26 11:29:20.208 3272 ERROR neutron.agent.linux.utils
>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Exit code:
>>>> 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists
>>>>
>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>> neutron.plugins.ml2.drivers.agent._common_agent
>>>> [req-9ba0ca9f-aeaf-44b2-ba24-c08556aae0ac - - - - -] Error in
>>>> agent loop. Devices info: {'current': set(['tap69feb7be-2b',
>>>> 'tap0efd5228-b0', 'tap83a57ce5-a8', 'tapd50d137f-f6']),
>>>> 'timestamps': {'tap0efd5228-b0': 9, 'tap69feb7be-2b': 13,
>>>> 'tap83a57ce5-a8': 10, 'tapd50d137f-f6': 8}, 'removed': set([]),
>>>> 'added': set(['tap69feb7be-2b', 'tap0efd5228-b0',
>>>> 'tap83a57ce5-a8', 'tapd50d137f-f6']), 'updated': set([])}
>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>> neutron.plugins.ml2.drivers.agent._common_agent Traceback (most
>>>> recent call last):
>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>> neutron.plugins.ml2.drivers.agent._common_agent File
>>>> "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 453, in
>>>> daemon_loop
>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>> neutron.plugins.ml2.drivers.agent._common_agent sync =
>>>> self.process_network_devices(device_info)
>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>> neutron.plugins.ml2.drivers.agent._common_agent File
>>>> "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line
>>>> 153, in wrapper
>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>> neutron.plugins.ml2.drivers.agent._common_agent return
>>>> f(*args, **kwargs)
>>>> 2018-07-26 11:29:20.219 3272 ERROR
>>>> neutron.plugins.ml2.drivers.agent._common_agent File
>>>> "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 210, in
>>>> process_network_devices
>>>>
>>>> Has anyone had similar experience?
>>>>
>>>> -- Marcio Prado
>>>> Analista de TI - Infraestrutura e Redes
>>>> Fone: (35) 9.9821-3561
>>>> www.marcioprado.eti.br
>>>>
>>>> _______________________________________________
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to : openstack at lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>> Post to : openstack at lists.openstack.org
>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
> --
> Marcio Prado
> Analista de TI - Infraestrutura e Redes
> Fone: (35) 9.9821-3561
> www.marcioprado.eti.br
More information about the Openstack
mailing list