Open Stack

Hi,


Yeah, rabbitmq server on the same host. rabbitmq service was stopped a little while to try to reproduce a problem. Do you mean report_state always running even the rabbitmq connection is down?  It sounds unreasonable.


Thanks,
Frank

At 2018-01-23 17:58:58, "Sławomir Kapłoński" <slawek at kaplonski.pl> wrote:
>Hi,
>
>Problem which You have here is that Your agent can't connect to rabbitmq. Because of that rpc message with report_state is timeouted and that is the reason of this error.
>Is Your rabbitmq server on same host with ovs agent? Is rabbitmq server works properly?
>
>— 
>Best regards
>Slawek Kaplonski
>slawek at kaplonski.pl
>
>
>
>> Wiadomość napisana przez Frank Wang <wangpeihuixyz at 126.com> w dniu 23.01.2018, o godz. 10:44:
>> 
>> Hi Sławomir,
>>    
>>      Thanks for you quick response. I'm using Ocata, Here is the exception stack. 
>> e: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. Client port: None
>> 2018-01-23 17:37:45.387 8636 ERROR oslo.messaging._drivers.impl_rabbit [-] [7b611083-b03c-4aaa-bb2f-a8f6cb89ab15] AMQP server on 127.0.0.1:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. Client port: None
>> 2018-01-23 17:37:45.395 8636 ERROR oslo.messaging._drivers.impl_rabbit [-] [bc243998-c484-47b3-8a5d-59104470c441] AMQP server on 127.0.0.1:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. Client port: None
>> 2018-01-23 17:37:45.402 8636 ERROR oslo.messaging._drivers.impl_rabbit [-] [f86f1d05-6f72-443a-967e-581c9c3af656] AMQP server on 127.0.0.1:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. Client port: None
>> 2018-01-23 17:37:51.927 8636 ERROR oslo.messaging._drivers.impl_rabbit [-] [4c876705-fd92-4b32-8c45-98ff1ed75846] AMQP server on 127.0.0.1:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. Client port: None
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Failed reporting state!
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 312, in _report_state
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     True)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/rpc.py", line 87, in report_state
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     return method(context, 'report_state', **kwargs)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in call
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     retry=self.retry)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in _send
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     timeout=timeout, retry=retry)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 458, in send
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     retry=retry)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 447, in _send
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     result = self._waiter.wait(msg_id, timeout)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 339, in wait
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     message = self.waiters.get(msg_id, timeout=timeout)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 238, in get
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     'to message ID %s' % msg_id)
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent MessagingTimeout: Timed out waiting for a reply to message ID 5907b7ced96140c693a6fb6dd0698dbc
>> 2018-01-23 17:37:53.807 8636 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent 
>> 2018-01-23 17:37:53.808 8636 WARNING oslo.service.loopingcall [-] Function 'neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent.OVSNeutronAgent._report_state' run outlasted interval by 30.00 sec
>> 2018-01-23 17:37:59.227 8636 INFO oslo.messaging._drivers.impl_rabbit [-] [3e7c487b-9222-4eee-b879-a7981649d49c] Reconnected to AMQP server on 127.0.0.1:5672 via [amqp] client with port 51476.
>> 2018-01-23 17:37:59.270 8636 INFO oslo.messaging._drivers.impl_rabbit [-] [a6262917-f316-43df-83fd-05af30a2b302] Reconnected to AMQP server on 127.0.0.1:5672 via [amqp] client with port 51478.
>> 2018-01-23 17:37:59.308 8636 INFO oslo.messaging._drivers.impl_rabbit [-] [99ff11c3-1662-461
>> 
>> Thanks,
>> Frank
>> At 2018-01-23 17:27:52, "Sławomir Kapłoński" <slawek at kaplonski.pl> wrote:
>> >Hi,
>> >
>> >If both ovs agent and neutron-server reconnect to rabbitmq then it should report state properly again IMO.
>> >Can You maybe send more details about Your issue? What OpenStack version You are running, exact stack trace of exception which You get and so on.
>> >
>> >— 
>> >Best regards
>> >Slawek Kaplonski
>> >slawek at kaplonski.pl
>> >
>> >
>> >
>> >> Wiadomość napisana przez Frank Wang <wangpeihuixyz at 126.com> w dniu 23.01.2018, o godz. 10:08:
>> >> 
>> >> Hi All,
>> >> 
>> >>     I'm really newbie about OpenStack Neutron, Please correct me if I say something wrong. There was a question I'd like to consult.  AMQP is the messaging bus between neutron-server and *agents. we usually use rabbitmq as the back-end of messaging bus.  The problem I encountered is the ovs agent raise an exception while reporting its own state to the server.  Here is my guess, If I restart the controller node, what if the rabbitmq start early than neutron-server. I mean the ovs agent always trying to connect to the rabbitmq. It will report state to the server through RPC once the connection established. if the server is not ready at this time. Does it cause the agent exception?  Any suggestion would be greatly appreciated!
>> >> 
>> >> 
>> >> Thanks,
>> >> Frank
>> >> 
>> >> 
>> >>  
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >__________________________________________________________________________
>> >OpenStack Development Mailing List (not for usage questions)
>> >Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> 
>>  
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180123/97ad7c98/attachment.html>