<div dir="ltr">Hello My name is inhye park from samsung SDS.<div><br></div><div>I am a developer about nova and sahara.</div><div><br></div><div>Recently Our system updated to icehouse.</div><div><br></div><div>After deleting cluster in sahara, VM occurred 500 error.</div>
<div><br></div><div>But this problem not occur in nova cli command.</div><div><br></div><div>Only in sahara this error occurred.</div><div><br></div><div>This is error log.</div><div><br></div><div>-----------------------------------------------------------------------------------------</div>
<div>
<p class="">2014-05-22 20:06:42.631 25640 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Failed to terminate process 8756 with SIGKILL: Device or resource busy</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher incoming.message))</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher payload)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in decorated_function</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher function(self, context, *args, **kwargs)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher e, sys.exc_info())</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in decorated_function</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2262, in terminate_instance</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher do_terminate_instance(instance, bdms)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line 249, in inner</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2260, in do_terminate_instance</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher self._set_instance_error_state(context, instance['uuid'])</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2250, in do_terminate_instance</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher reservations=reservations)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 103, in inner</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher rv = f(*args, **kwargs)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2220, in _delete_instance</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher user_id=user_id)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2190, in _delete_instance</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher self._shutdown_instance(context, db_inst, bdms)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2125, in _shutdown_instance</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher requested_networks)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2115, in _shutdown_instance</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher block_device_info)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 951, in destroy</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher self._destroy(instance)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 908, in _destroy</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher instance=instance)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 880, in _destroy</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher virt_dom.destroy()</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call</p><p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher rv = execute(f,*args,**kwargs)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker</p><p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher rv = meth(*args,**kwargs)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/libvirt.py", line 918, in destroy</p><p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainDestroy() failed', dom=self)</p>
<p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher libvirtError: Failed to terminate process 8756 with SIGKILL: Device or resource busy</p><p class="">2014-05-22 20:06:42.631 25640 TRACE oslo.messaging.rpc.dispatcher</p>
<p class="">
</p><p class="">2014-05-22 20:06:42.633 25640 ERROR oslo.messaging._drivers.common [-] Returning exception Failed to terminate process 8756 with SIGKILL: Device or resource busy to caller</p><p class=""><br></p><p class="">
Could you please help about this case?</p><p class="">Thanks.</p><p class=""><br></p></div><div><br></div><div><br></div></div>