[Openstack-operators] Fwd: Can't start a vm, from dashboard
Juan José Pavlik Salles
jjpavlik at gmail.com
Fri Aug 9 00:06:39 UTC 2013
You are right Wilson, there's no hostiD in the vm description:
+-----------------------------+-----------------------------------------------------------+
| Property | Value
|
+-----------------------------+-----------------------------------------------------------+
| status | BUILD
|
| updated | 2013-08-08T19:23:01Z
|
| OS-EXT-STS:task_state | scheduling
|
| key_name | None
|
| image | Ubuntu 12.04.2 LTS
(1359ca8d-23a2-40e8-940f-d90b3e68bb39) |
| hostId |
|
| OS-EXT-STS:vm_state | building
|
| flavor | m1.tiny (1)
|
| id | b0583cca-63c2-481f-8b94-7aeb2e86641f
|
| security_groups | [{u'name': u'default'}]
|
| user_id | 20390b639d4449c18926dca5e038ec5e
|
| name | prueba11
|
| created | 2013-08-08T19:19:44Z
|
| tenant_id | d1e3aae242f14c488d2225dcbf1e96d6
|
| OS-DCF:diskConfig | MANUAL
|
| metadata | {}
|
| accessIPv4 |
|
| accessIPv6 |
|
| progress | 0
|
| OS-EXT-STS:power_state | 0
|
| OS-EXT-AZ:availability_zone | nova
|
| config_drive |
|
+-----------------------------+-----------------------------------------------------------+
I found this new log in the scheduler, it must be related to it:
2013-08-08 19:19:46.000 ERROR nova.openstack.common.rpc.amqp
[req-b049c969-c411-4b76-9aa5-ad88d714c4ab 20390b639d4449c18926dca5e038ec5e
d1e3aae242f14c488d2225dcbf1e96d6] Exception during message handling
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
Traceback (most recent call last):
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
430, in _process_data
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp rval
= self.proxy.dispatch(ctxt, version, method, **args)
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
line 133, in dispatch
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
return getattr(proxyobj, method)(ctxt, **kwargs)
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 127, in
run_instance
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
'schedule', *instance_uuids):
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 318, in
__enter__
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
self.conductor.action_event_start(self.context, event)
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 264, in
action_event_start
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
return self._manager.action_event_start(context, values)
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 1348, in wrapper
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
return func(*args, **kwargs)
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 334, in
action_event_start
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp evt
= self.db.action_event_start(context, values)
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/db/api.py", line 1625, in
action_event_start
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
return IMPL.action_event_start(context, values)
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 4624, in
action_event_start
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
instance_uuid=values['instance_uuid'])
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
InstanceActionNotFound: Action for request_id
req-b049c969-c411-4b76-9aa5-ad88d714c4ab on instance
b0583cca-63c2-481f-8b94-7aeb2e86641f not found
2013-08-08 19:19:46.000 29114 TRACE nova.openstack.common.rpc.amqp
It's getting me crazy. "InstanceActionNotFound..." does it mean something
to you?
2013/8/8 Mike Wilson <geekinutah at gmail.com>
> Juan,
>
> If your instance show's the task_state as "scheduling" it's possible that
> your new instance never made it to the scheduler. When you do a nova show
> <instance_id> does it say which host node it is supposed to be on? If it
> does then the scheduler probably did its thing but failed the RPC to the
> compute node, if it doesn't have a host then your problem is that the
> message never made it from nova-api to nova-scheduler.
>
> In any case, it looks like rabbit is not quite setup correctly on your end.
>
> -Mike
>
>
> On Wed, Aug 7, 2013 at 6:55 PM, Juan José Pavlik Salles <
> jjpavlik at gmail.com> wrote:
>
>> Is there any way i can test nova-conductor and nova-scheduler to be sure
>> they are working like they should? If i list nova-manage service list,
>> everything is fine. I'm running out of ideas hahaha.
>>
>>
>> 2013/8/7 Juan José Pavlik Salles <jjpavlik at gmail.com>
>>
>>> According to the doc this problem should be related to some service that
>>> isn't answering to nova-api. I just have 3 servers in my deployment, so i
>>> don't think this is problem related to the amount of messages in the
>>> queues.
>>>
>>>
>>> 2013/8/7 Juan José Pavlik Salles <jjpavlik at gmail.com>
>>>
>>>> Here i have more information, i tried to boot a vm from the CLI and it
>>>> doesn't really fail. But when i check the vms status in the dashboard it
>>>> says "Scheduling" and never changes its state to "running" or "error".
>>>>
>>>>
>>>> 2013/8/7 Juan José Pavlik Salles <jjpavlik at gmail.com>
>>>>
>>>>> I just finished installing everything, i tried to create my first VM
>>>>> from the dashboard but it doesn't work. After choosing flavor and hitting
>>>>> launch it starts "creating" it but after a few seconds it stops saying:
>>>>> "Error: There was an error submitting the form. Please try again.". The
>>>>> only place where i found something related is in nova.log in my compute
>>>>> node, here is the log:
>>>>>
>>>>> *2013-08-07 18:05:55.293 DEBUG nova.openstack.common.rpc.common
>>>>> [req-0cfe760f-2e74-4e92-919c-663ba02c7f2f 20390b639d4449c18926dca5e038ec5e
>>>>> d1e3aae242f14c488d2225dcbf1e96d6] Timed out waiting for RPC response: timed
>>>>> out _error_callback
>>>>> /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py:628
>>>>> *
>>>>> *2013-08-07 18:05:55.479 DEBUG nova.quota
>>>>> [req-0cfe760f-2e74-4e92-919c-663ba02c7f2f 20390b639d4449c18926dca5e038ec5e
>>>>> d1e3aae242f14c488d2225dcbf1e96d6] Rolled back reservations
>>>>> ['3e941a2b-2cc6-4f01-8dc1-13dc09369141',
>>>>> '411f6f70-415e-4a21-aa06-3980070d6095',
>>>>> 'd4791eb7-b75a-4ab8-bfdb-5d5cd201e40d'] rollback
>>>>> /usr/lib/python2.7/dist-packages/nova/quota.py:1012*
>>>>> *2013-08-07 18:05:55.480 ERROR nova.api.openstack
>>>>> [req-0cfe760f-2e74-4e92-919c-663ba02c7f2f 20390b639d4449c18926dca5e038ec5e
>>>>> d1e3aae242f14c488d2225dcbf1e96d6] Caught error: Timeout while waiting on
>>>>> RPC response.*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack Traceback
>>>>> (most recent call last):*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 81,
>>>>> in __call__*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack return
>>>>> req.get_response(self.application)*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
>>>>> *
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack
>>>>> application, catch_exc_info=False)*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in
>>>>> call_application*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack app_iter
>>>>> = application(self.environ, start_response)*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
>>>>> *
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack return
>>>>> resp(environ, start_response)*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
>>>>> line 450, in __call__*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack return
>>>>> self.app(env, start_response)*
>>>>> *...
>>>>> *
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
>>>>> 551, in __iter__
>>>>> *
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack
>>>>> self._iterator.next()*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py",
>>>>> line 648, in iterconsume*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack yield
>>>>> self.ensure(_error_callback, _consume)*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py",
>>>>> line 566, in ensure*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack
>>>>> error_callback(e)*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack File
>>>>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py",
>>>>> line 629, in _error_callback*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack raise
>>>>> rpc_common.Timeout()*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack Timeout:
>>>>> Timeout while waiting on RPC response.*
>>>>> *2013-08-07 18:05:55.480 29278 TRACE nova.api.openstack *
>>>>> *2013-08-07 18:05:55.488 INFO nova.api.openstack
>>>>> [req-0cfe760f-2e74-4e92-919c-663ba02c7f2f 20390b639d4449c18926dca5e038ec5e
>>>>> d1e3aae242f14c488d2225dcbf1e96d6]
>>>>> http://172.19.136.13:8774/v2/d1e3aae242f14c488d2225dcbf1e96d6/serversreturned with HTTP 500
>>>>> *
>>>>> *2013-08-07 18:05:55.488 DEBUG nova.api.openstack.wsgi
>>>>> [req-0cfe760f-2e74-4e92-919c-663ba02c7f2f 20390b639d4449c18926dca5e038ec5e
>>>>> d1e3aae242f14c488d2225dcbf1e96d6] Returning 500 to user: The server has
>>>>> either erred or is incapable of performing the requested operation.
>>>>> __call__ /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1165
>>>>> *
>>>>> *2013-08-07 18:05:55.489 INFO nova.osapi_compute.wsgi.server
>>>>> [req-0cfe760f-2e74-4e92-919c-663ba02c7f2f 20390b639d4449c18926dca5e038ec5e
>>>>> d1e3aae242f14c488d2225dcbf1e96d6] 172.19.136.13 "POST
>>>>> /v2/d1e3aae242f14c488d2225dcbf1e96d6/servers HTTP/1.1" status: 500 len: 335
>>>>> time: 60.5262640*
>>>>>
>>>>> A couple of things about my deployment that may help you help me:
>>>>> -One controller node running: nova-conductor, nova-scheduler,
>>>>> keystone, quantum-server, rabbitmq
>>>>> -One compute node running: nova-api, nova-compute, glance
>>>>> -One storage node running cinder
>>>>>
>>>>> My ideas:
>>>>> -I think it could be a problem related to nova-compute using
>>>>> nova-conductor (i really don't know how to tell nova to use it...), somehow
>>>>> messages from nova-compute doesn't reach nova-conductor on the controller
>>>>> node eventhough that nova-compute is connected to rabbit and so is
>>>>> nova-conductor.
>>>>> -I haven't found any message like "wrong password for rabbit" on any
>>>>> log file.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pavlik Salles Juan José
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Pavlik Salles Juan José
>>>>
>>>
>>>
>>>
>>> --
>>> Pavlik Salles Juan José
>>>
>>
>>
>>
>> --
>> Pavlik Salles Juan José
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>
--
Pavlik Salles Juan José
--
Pavlik Salles Juan José
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20130808/ca0a348c/attachment.html>
More information about the OpenStack-operators
mailing list