Open Stack

Thanks for your helping.

But in our system, the problem is that:


If using the nova cli, this libvirt problem not occurred.


But in deleting cluster of the sahara , this problem occurred.

That if our problem.


Can it influence the step on deleting cluster in the process to destroy vm
of libvirtd?

I am wondering about that.


Thank you.


2014-05-22 20:54 GMT+09:00 SMIGIELSKI, Radoslaw (Radoslaw) <
radoslaw.smigielski at alcatel-lucent.com>:

> Looks like one or more of your VMs get stuck and libvirt having problems
> to stop it now:
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher     if
> ret == -1: raise libvirtError ('virDomainDestroy() failed', dom=self)
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> libvirtError: Failed to terminate process 8756 with SIGKILL: Device or
> resource busy
>
>
> I would recommend you to check what exactly is PID (8756) in the error? I
> assume is a qemu process and what's going on with this process?
> Does it have some issue with accessing storage or something else? You can
> also check the libvirtd logs, there should be something more.
>
> ====================================================
> Radosław Śmigielski
> ALCATEL-LUCENT - Cloudband/Node Team
> Office: 238 646 13 or +353 (0)188 646 13
> Mobile: +353 86 820 74 23
>
>
>
>
> ________________________________________
> From: cosmos cosmos [cosmos0703 at gmail.com]
> Sent: 22 May 2014 12:30
> To: openstack-dev at lists.openstack.org
> Subject: [openstack-dev]  [sahara] Deleting cluster in sahara horizon, 500
> Error occured in nova.
>
> Hello My name is inhye park from samsung SDS.
>
> I am a developer about nova and sahara.
>
> Recently Our system updated to icehouse.
>
> After deleting cluster in sahara, VM occurred 500 error.
>
> But this problem not occur in nova cli command.
>
> Only in sahara this error occurred.
>
> This is error log.
>
>
> -----------------------------------------------------------------------------------------
>
> 2014-05-22 20:06:42.631 25640 ERROR oslo.messaging.rpc.dispatcher [-]
> Exception during message handling: Failed to terminate process 8756 with
> SIGKILL: Device or resource busy
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> Traceback (most recent call last):
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 133, in _dispatch_and_reply
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> incoming.message))
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 176, in _dispatch
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> return self._do_dispatch(endpoint, method, ctxt, args)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 122, in _do_dispatch
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> result = getattr(endpoint, method)(ctxt, **new_args)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> payload)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> return f(self, context, *args, **kw)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in
> decorated_function
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> function(self, context, *args, **kwargs)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in
> decorated_function
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher     e,
> sys.exc_info())
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in
> decorated_function
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> return function(self, context, *args, **kwargs)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2262, in
> terminate_instance
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> do_terminate_instance(instance, bdms)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line
> 249, in inner
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> return f(*args, **kwargs)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2260, in
> do_terminate_instance
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> self._set_instance_error_state(context, instance['uuid'])
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2250, in
> do_terminate_instance
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> reservations=reservations)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 103, in inner
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher     rv =
> f(*args, **kwargs)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2220, in
> _delete_instance
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> user_id=user_id)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2190, in
> _delete_instance
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> self._shutdown_instance(context, db_inst, bdms)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2125, in
> _shutdown_instance
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> requested_networks)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2115, in
> _shutdown_instance
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> block_device_info)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 951,
> in destroy
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> self._destroy(instance)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 908,
> in _destroy
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> instance=instance)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 880,
> in _destroy
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> virt_dom.destroy()
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> result = proxy_call(self._autowrap, f, *args, **kwargs)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in
> proxy_call
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher     rv =
> execute(f,*args,**kwargs)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher     rv =
> meth(*args,**kwargs)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/dist-packages/libvirt.py", line 918, in destroy
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher     if
> ret == -1: raise libvirtError ('virDomainDestroy() failed', dom=self)
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
> libvirtError: Failed to terminate process 8756 with SIGKILL: Device or
> resource busy
>
> 2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher
>
> 2014-05-22 20:06:42.633 25640 ERROR oslo.messaging._drivers.common [-]
> Returning exception Failed to terminate process 8756 with SIGKILL: Device
> or resource busy to caller
>
>
> Could you please help about this case?
>
> Thanks.
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140523/f027d504/attachment.html>