[Openstack] [Grizzly] Unable to delete VM instances when using Nova with ZooKeeper

Chu Duc Minh chu.ducminh at gmail.com
Tue Sep 17 15:58:00 UTC 2013


For more information, i setup ZooKeeper cluster on 4 nodes, then follow
instructions at
http://docs.openstack.org/trunk/openstack-compute/admin/content/zookeeper-servicegroup-driver.html
On Sep 17, 2013 5:54 PM, "Chu Duc Minh" <chu.ducminh at gmail.com> wrote:

> I configured Nova (both controller-nodes and compute-nodes) with *
> ZooKeeper* (on Ubuntu 12.04).
> I can create instance and other operations seem OK.
> But when i delete instance:
> # nova delete 92d154dd-6f4d-483c-84d4-ad1b8a691156
> The server has either erred or is incapable of performing the requested
> operation. (HTTP 500) (Request-ID: req-48afc072-1e7d-473a-b99f-53c22051f260)
>
> And check nova-api.log and saw errors (quite long):
>
> 2013-09-17 17:39:19.617 ERROR nova.exception
> [req-48afc072-1e7d-473a-b99f-53c22051f260 3f75c75d09ce4d01887c7415ef7c5ae9
> 55634eee777042
> b69cdb478230e88d52] Exception in string format operation
> 2013-09-17 17:39:19.617 20071 TRACE nova.exception Traceback (most recent
> call last):
> 2013-09-17 17:39:19.617 20071 TRACE nova.exception   File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 147, in __init__
> 2013-09-17 17:39:19.617 20071 TRACE nova.exception     message =
> self.message % kwargs
> 2013-09-17 17:39:19.617 20071 TRACE nova.exception TypeError: %d format: a
> number is required, not str
> 2013-09-17 17:39:19.617 20071 TRACE nova.exception
> 2013-09-17 17:39:19.618 ERROR nova.exception
> [req-48afc072-1e7d-473a-b99f-53c22051f260 3f75c75d09ce4d01887c7415ef7c5ae9
> 55634eee777042
> b69cdb478230e88d52] code: 500
> 2013-09-17 17:39:19.618 ERROR nova.exception
> [req-48afc072-1e7d-473a-b99f-53c22051f260 3f75c75d09ce4d01887c7415ef7c5ae9
> 55634eee777042
> b69cdb478230e88d52] driver: ZooKeeperDriver
> 2013-09-17 17:39:19.642 ERROR nova.api.openstack
> [req-48afc072-1e7d-473a-b99f-53c22051f260 3f75c75d09ce4d01887c7415ef7c5ae9
> 55634eee77
> 7042b69cdb478230e88d52] Caught error: The service from servicegroup driver
> %(driver) is temporarily unavailable.
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack Traceback (most
> recent call last):
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line
> 81, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> req.get_response(self.application)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     application,
> catch_exc_info=False)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_
> application
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     app_iter =
> application(self.environ, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> resp(environ, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.p
> y", line 450, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> self.app(env, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> resp(environ, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> resp(environ, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> resp(environ, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __
> call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     response =
> self.app(environ, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> resp(environ, start_response)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     resp =
> self.call_func(req, *args, **self.kwargs)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> self.func(req, *args, **kwargs)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 890, in
> __call__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     content_type,
> body, accept)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 942, in
> _process_stack
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     action_result =
> self.dispatch(meth, request, action_args)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 1022,
> in dispatch
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> method(req=request, **action_args)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py",
> line 1117, in delete
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack
> self._delete(req.environ['nova.context'], req, id)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py",
> line 961, in _delete
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack
> self.compute_api.delete(context, instance)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 163, in wrapped
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> func(self, context, target, *args, **kwargs)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 153, in inner
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> function(self, context, instance, *args, **kwargs)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 134, in inner
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return f(self,
> context, instance, *args, **kw)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1233, in delete
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack
> self._delete_instance(context, instance)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1225, in
> _delete_instance
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack
> task_state=task_states.DELETING)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1122, in
> _delete
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack
> project_id=project_id)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     self.gen.next()
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1093, in
> _delete
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     if
> self.servicegroup_api.service_is_up(service):
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py", line 93, in
> service_is_up
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     return
> self._driver.is_up(member)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/zk.py", line
> 115, in is_up
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     all_members =
> self.get_all(group_id)
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack   File
> "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/zk.py", line
> 138, in get_all
> *2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack     raise
> exception.ServiceGroupUnavailable(driver="ZooKeeperDriver")
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack
> ServiceGroupUnavailable: The service from servicegroup driver %(driver) is
> temporarily unavailable.*
> 2013-09-17 17:39:19.642 20071 TRACE nova.api.openstack
> 2013-09-17 17:39:19.646 INFO nova.api.openstack
> [req-48afc072-1e7d-473a-b99f-53c22051f260 3f75c75d09ce4d01887c7415ef7c5ae9
> 55634eee777042b69cdb478230e88d52]
> http://123.30.234.14:8774/v2/55634eee777042b69cdb478230e88d52/servers/92d154dd-6f4d-483c-84d4-ad1b8a691156returned with HTTP 500
>
>
> Do you have an idea why this bug happen?
>
> Thank you very much!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20130917/dd85443e/attachment.html>


More information about the Openstack mailing list