[Openstack] [Neutron] Openvswitch agent not alive
Kaustubh Kelkar
kaustubh.kelkar at casa-systems.com
Fri May 6 19:15:51 UTC 2016
From the log it seems that metadata agent cannot connect to AMQP server. Can you check if RabbitMQ is running and the credentials used are correct?
-Kaustubh
From: Vikram Choudhary [mailto:vikschw at gmail.com]
Sent: Friday, May 6, 2016 2:31 AM
To: Openstack <openstack at lists.openstack.org>
Subject: Re: [Openstack] [Neutron] Openvswitch agent not alive
Widening audience!
On Thu, May 5, 2016 at 9:03 PM, Silvia Fichera <fichera.sil at gmail.com<mailto:fichera.sil at gmail.com>> wrote:
There is an error in q-meta:
2016-05-05 17:12:52.594 ERROR neutron.agent.metadata.agent [-] Failed reporting state!
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent Traceback (most recent call last):
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/opt/stack/neutron/neutron/agent/metadata/agent.py", line 314, in _report_state
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent use_call=self.agent_state.get('start_flag'))
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/opt/stack/neutron/neutron/agent/rpc.py", line 86, in report_state
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent return method(context, 'report_state', **kwargs)
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent retry=self.retry)
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent timeout=timeout, retry=retry)
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 431, in send
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent retry=retry)
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 420, in _send
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent result = self._waiter.wait(msg_id, timeout)
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 318, in wait
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent message = self.waiters.get(msg_id, timeout=timeout)
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 223, in get
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent 'to message ID %s' % msg_id)
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent MessagingTimeout: Timed out waiting for a reply to message ID b71f29ac5bca496c9dd02c8979bbe556
2016-05-05 17:12:52.594 TRACE neutron.agent.metadata.agent
2016-05-05 17:12:52.622 WARNING oslo.service.loopingcall [-] Function 'neutron.agent.metadata.agent.UnixDomainMetadataProxy._report_state' run outlasted interval by 30.19 sec
and in c-vol:
2016-05-05 17:31:02.799 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:31:12.807 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:31:22.815 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:31:33.041 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:31:43.043 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:31:44.028 DEBUG oslo_service.periodic_task [req-b829e2a7-41cf-4bc6-a86b-ebdec66f9882 None None] Running periodic task VolumeManager._publish_service_capabilities from (pid=4038) run_periodic_tasks /usr/local/lib/python2.7/dist-packages/oslo_service/periodic_task.py:213
2016-05-05 17:31:44.028 DEBUG oslo_service.periodic_task [req-b829e2a7-41cf-4bc6-a86b-ebdec66f9882 None None] Running periodic task VolumeManager._report_driver_status from (pid=4038) run_periodic_tasks /usr/local/lib/python2.7/dist-packages/oslo_service/periodic_task.py:213
2016-05-05 17:31:44.029 WARNING cinder.volume.manager [req-b829e2a7-41cf-4bc6-a86b-ebdec66f9882 None None] Update driver status failed: (config name lvmdriver-1) is uninitialized.
2016-05-05 17:31:53.047 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:32:03.050 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:32:13.060 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:32:23.383 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:32:33.386 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:32:43.391 ERROR cinder.service [-] Manager for service cinder-volume devstack1 at lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
2016-05-05 17:17 GMT+02:00 Silvia Fichera <fichera.sil at gmail.com<mailto:fichera.sil at gmail.com>>:
I have rebooted the machine and the agent is now up.
But I still can't ping from the floating IP that I have assigned.
2016-05-05 14:06 GMT+02:00 Vikram Choudhary <vikschw at gmail.com<mailto:vikschw at gmail.com>>:
Can you please check your disk space.. I can find memory failure ..
2016-05-05 09:58:19.263 TRACE neutron OSError: [Errno 12] Cannot allocate memory
On Thu, May 5, 2016 at 4:55 PM, Silvia Fichera <fichera.sil at gmail.com<mailto:fichera.sil at gmail.com>> wrote:
from the screen there are few errors and than q-agt fails:
2016-05-05 09:58:19.263 TRACE neutron File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
2016-05-05 09:58:19.263 TRACE neutron self.gen.throw(type, value, traceback)
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/polling.py", line 38, in get_polling_manager
2016-05-05 09:58:19.263 TRACE neutron pm.stop()
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/polling.py", line 56, in stop
2016-05-05 09:58:19.263 TRACE neutron self._monitor.stop()
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/async_process.py", line 128, in stop
2016-05-05 09:58:19.263 TRACE neutron self._kill(kill_signal)
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/ovsdb_monitor.py", line 115, in _kill
2016-05-05 09:58:19.263 TRACE neutron super(SimpleInterfaceMonitor, self)._kill(*args, **kwargs)
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/async_process.py", line 161, in _kill
2016-05-05 09:58:19.263 TRACE neutron pid = self.pid
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/async_process.py", line 157, in pid
2016-05-05 09:58:19.263 TRACE neutron run_as_root=self.run_as_root)
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 277, in get_root_helper_child_pid
2016-05-05 09:58:19.263 TRACE neutron pid = find_child_pids(pid)[0]
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 203, in find_child_pids
2016-05-05 09:58:19.263 TRACE neutron log_fail_as_error=False)
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 120, in execute
2016-05-05 09:58:19.263 TRACE neutron addl_env=addl_env)
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 89, in create_process
2016-05-05 09:58:19.263 TRACE neutron stderr=subprocess.PIPE)
2016-05-05 09:58:19.263 TRACE neutron File "/opt/stack/neutron/neutron/common/utils.py", line 199, in subprocess_popen
2016-05-05 09:58:19.263 TRACE neutron close_fds=close_fds, env=env)
2016-05-05 09:58:19.263 TRACE neutron File "/usr/local/lib/python2.7/dist-packages/eventlet/green/subprocess.py", line 53, in __init__
2016-05-05 09:58:19.263 TRACE neutron subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
2016-05-05 09:58:19.263 TRACE neutron File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
2016-05-05 09:58:19.263 TRACE neutron errread, errwrite)
2016-05-05 09:58:19.263 TRACE neutron File "/usr/lib/python2.7/subprocess.py", line 1223, in _execute_child
2016-05-05 09:58:19.263 TRACE neutron self.pid = os.fork()
2016-05-05 09:58:19.263 TRACE neutron OSError: [Errno 12] Cannot allocate memory
2016-05-05 09:58:19.263 TRACE neutron
2016-05-05 09:58:20.310 INFO oslo_rootwrap.client [req-a61d57b4-aa7d-4b1a-ad73-d4eb36c37a4a None None] Stopping rootwrap daemon process with pid=18261
q-agt failed to start
2016-05-05 12:31 GMT+02:00 Vikram Choudhary <vikschw at gmail.com<mailto:vikschw at gmail.com>>:
Can you navigate to the q-agt devstack screen to check what's going on?
Execute screen -x and then go to the q-agt screen by using CTRL+A N
On May 5, 2016 3:53 PM, "Silvia Fichera" <fichera.sil at gmail.com<mailto:fichera.sil at gmail.com>> wrote:
There is no error in q-svc and when I try to open q-agt the system crashes.
Il 05/Mag/2016 12:03, "Vikram Choudhary" <vikschw at gmail.com<mailto:vikschw at gmail.com>> ha scritto:
Can you check q-agt and q-svc logs for abnormalities?
On May 5, 2016 3:29 PM, "Silvia Fichera" <fichera.sil at gmail.com<mailto:fichera.sil at gmail.com>> wrote:
Hi all,
I have installed OpenStack via Devstack, Liberty, I created the external network, the private one and the router to connect each other, launched instances and assigned the floating IP.
BUT when I check the connectivity I can't ping the router interface and from the instances, I can't ping an external address.
I have checked
neutron agent-list
id | agent_type | host | alive | admin_state_up | binary | +--------------------------------------+--------------------+-----------+-------+----------------+---------------------------+ |
0b1a6478-4f1d-4338-a45d-112fda0c46af | DHCP agent | devstack1 | :-) | True | neutron-dhcp-agent |
| 53a726ca-d874-41da-a06e-9e30624e6fce | Open vSwitch agent | devstack1 | xxx | True | neutron-openvswitch-agent |
| a8938910-8ef2-4894-a31c-5047d536156d | L3 agent | devstack1 | :-) | True | neutron-l3-agent |
| ede1be74-12fc-435c-9f4b-ef44ae088a2d | Metadata agent | devstack1 | :-) | True | neutron-metadata-agent | +--------------------------------------+--------------------+-----------+-------+----------------+---------------------------+
the openswitch agent is not alive.
How to debug it?
--
Silvia Fichera
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
--
Silvia Fichera
--
Silvia Fichera
--
Silvia Fichera
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160506/0c1fb8d1/attachment.html>
More information about the Openstack
mailing list