[Openstack] Hypervisor connection lost with upgrading libvirt

Waqas Riaz waqas.riaz at xflowresearch.com
Tue Feb 18 22:14:00 UTC 2014


Hi all,

I am using devstack on Ubuntu 12.04 server with kvm hypervisor. I upgraded
libvirt from 0.9.8 to 1.2.1. Due to which I can not create instance and
hypervisor is not getting connected.

waqas at server2:~$ nova hypervisor-list
+----+---------------------+
| ID | Hypervisor hostname |
+----+---------------------+
+----+---------------------+

Looking in the screens, I found following errors in n-cpu screen.

2014-02-19 03:08:31.165 DEBUG nova.openstack.common.service [-]
********************************************************************************
from (pid=23386) log_opt_values
/opt/stack/oslo.config/oslo/config/cfg.py:1925
2014-02-19 03:08:31.166 AUDIT nova.service [-] Starting compute node
(version 2014.1)
2014-02-19 03:08:31.167 DEBUG nova.virt.libvirt.driver [-] Connecting to
libvirt: qemu:///system from (pid=23386) _get_new_connection
/opt/stack/nova/nova/virt/libvirt/driver.py:647
libvirt: XML-RPC error : Failed to connect socket to
'/var/run/libvirt/libvirt-sock': Permission denied
2014-02-19 03:08:31.173 ERROR nova.virt.libvirt.driver [-] Connection to
libvirt failed: Failed to connect socket to
'/var/run/libvirt/libvirt-sock': Permission denied
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver Traceback (most
recent call last):
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 768, in _connect
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver
libvirt.openAuth, uri, auth, flags)
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver   File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in
proxy_call
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver     rv =
execute(f,*args,**kwargs)
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver   File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in
tworker
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver     rv =
meth(*args,**kwargs)
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver   File
"/usr/lib/python2.7/dist-packages/libvirt.py", line 102, in openAuth
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver     if ret is
None:raise libvirtError('virConnectOpenAuth() failed')
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver libvirtError: Failed
to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied
2014-02-19 03:08:31.173 TRACE nova.virt.libvirt.driver
2014-02-19 03:08:31.178 DEBUG oslo.messaging._drivers.amqpdriver [-] MSG_ID
is f881e9cac9b04702a521c31bb3542511 from (pid=23386) _send
/opt/stack/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py:363
2014-02-19 03:08:31.178 DEBUG oslo.messaging._drivers.amqp [-] UNIQUE_ID is
d9eb144992654fb6b76636b2e7289517. from (pid=23386) _add_unique_id
/opt/stack/oslo.messaging/oslo/messaging/_drivers/amqp.py:333
2014-02-19 03:08:31.202 WARNING nova.virt.libvirt.driver [-] Cannot update
service status on host: server2,since it is not registered.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/poll.py", line
97, in wait
    readers.get(fileno, noop).cb(fileno)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py",
line 194, in main
    result = function(*args, **kwargs)
  File "/opt/stack/nova/nova/openstack/common/service.py", line 480, in
run_service
    service.start()
  File "/opt/stack/nova/nova/service.py", line 177, in start
    self.manager.init_host()
  File "/opt/stack/nova/nova/compute/manager.py", line 811, in init_host
    self.driver.init_host(host=self.host)
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 631, in init_host
    self._do_quality_warnings()
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 619, in
_do_quality_warnings
    caps = self.get_host_capabilities()
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2788, in
get_host_capabilities
    xmlstr = self._conn.getCapabilities()
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 696, in
_get_connection
    wrapped_conn = self._get_new_connection()
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 651, in
_get_new_connection
    wrapped_conn = self._connect(self.uri(), self.read_only)
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 777, in _connect
    raise exception.HypervisorUnavailable(host=CONF.host)
HypervisorUnavailable: Connection to the hypervisor is broken on host:
server2
Removing descriptor: 3
2014-02-19 03:08:31.204 ERROR nova.openstack.common.threadgroup [-]
Connection to the hypervisor is broken on host: server2
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup Traceback
(most recent call last):
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/openstack/common/threadgroup.py", line 117, in wait
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     x.wait()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/openstack/common/threadgroup.py", line 49, in wait
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     return
self.thread.wait()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168,
in wait
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     return
self._exit_event.wait()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in
wait
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     return
hubs.get_hub().switch()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in
switch
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     return
self.greenlet.switch()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194,
in main
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     result
= function(*args, **kwargs)
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/openstack/common/service.py", line 480, in run_service
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup
service.start()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/service.py", line 177, in start
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup
self.manager.init_host()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/compute/manager.py", line 811, in init_host
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup
self.driver.init_host(host=self.host)
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 631, in init_host
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup
self._do_quality_warnings()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 619, in
_do_quality_warnings
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     caps =
self.get_host_capabilities()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 2788, in
get_host_capabilities
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     xmlstr
= self._conn.getCapabilities()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 696, in _get_connection
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup
wrapped_conn = self._get_new_connection()
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 651, in
_get_new_connection
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup
wrapped_conn = self._connect(self.uri(), self.read_only)
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 777, in _connect
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup     raise
exception.HypervisorUnavailable(host=CONF.host)
2014-02-19 03:08:31.204 TRACE nova.openstack.common.threadgroup
HypervisorUnavailable: Connection to the hypervisor is broken on host:
server2

Any help or suggestion would be a great favour.
Thanks,
-- 

Waqas Riaz
Software Development Engineer
xFlow Research <http://www.xflowresearch.com/>
+92-332-6396672 (GMT +5)
Skype: waqas,riaz

This electronic message transmission contains information from the company
that may be proprietary, confidential and/or privileged.  The information
is intended only for the use of the individuals or entity named above.  If
you are not the intended recipient, be aware that any disclosure, copying
or distribution or use of the contents of this information is prohibited.
 If you have received this electronic transmission in error, please notify
the sender immediately by replying to the address listed in the "From:"
field.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140219/19a543c6/attachment.html>


More information about the Openstack mailing list