[Openstack] [OpenStack] neutron linuxbridge agent failed to get msg from rabbitmq

Xu, Rongjie (NSB - CN/Hangzhou) rongjie.xu at nokia-sbell.com
Tue Aug 1 06:17:40 UTC 2017


Hi,

I manually deployed OpenStack with one controller and several computes, which is working well for months. Recently I get one issue after server power off and on again due to high temperate warning.

Issue it neutron-linuxbridge-agent service on compute nodes failed to get messages from rabbitmq on controller nodes. The error log is like follows:

2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent [-] Failed reporting state!
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 130, in _report_state
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     True)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 88, in report_state
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     return method(context, 'report_state', **kwargs)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=self.retry)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 97, in _send
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     timeout=timeout, retry=retry)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     retry=retry)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 453, in _send
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     result = self._waiter.wait(msg_id, timeout)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 336, in wait
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     message = self.waiters.get(msg_id, timeout=timeout)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 238, in get
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent     'to message ID %s' % msg_id)
2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent MessagingTimeout: Timed out waiting for a reply to message ID 6c54bbf0b01d49eaa8babfdd5f9ba80

This is what I configured in /etc/neutron/neutron.conf for rabbitmq connection.
*         transport_url = rabbit://openstack:<password>@controller

And from controller I see the user is valid:
root at controller:~# rabbitmqctl list_users
Listing users ...
guest     [administrator]
openstack           [administrator]
root at controller:~# rabbitmqctl list_user_permissions openstack
Listing permissions for user "openstack" ...
/              .*            .*            .*

And from rabbitmq management GUI, I see there are connections established between controller and compute. What could cause this problem then? And how can I get further? Thanks.

Best Regards
Xu Rongjie (Max)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170801/a91d42cf/attachment.html>


More information about the Openstack mailing list