Open Stack

Hi,


I recently deploy Bare-metal node instance using Heat Template. However, Nova failed to spawn due to a timeout error. When I look into the code I found that the timeout is related to Nova downloading disk image from Glance. The nova-schedule.log shows below:

2014-02-28 02:49:48.046 2136 ERROR nova.compute.manager [req-09e61b23-436f-4425-8db0-10dd1aea2e39 85bbc1abb4254761a5452654a6934b75 692e595702654930936a65d1a658cff4] [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] Instance failed to spawn

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] Traceback (most recent call last):

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1417, in _spawn/ 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] network_info=network_info,

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/virt/baremetal/pxe.py", line 444, in cache_images 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] self._cache_tftp_images(context, instance, tftp_image_info)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/virt/baremetal/pxe.py", line 335, in _cache_tftp_images 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] project_id=instance['project_id'],

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/virt/baremetal/utils.py", line 33, in cache_image 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] user_id, project_id)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 645, in fetch_image 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] max_size=max_size)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 196, in fetch_to_raw 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] max_size=max_size)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 190, in fetch 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] image_service.download(context, image_id, dst_path=path)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 360, in download 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] for chunk in image_chunks:

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 478, in __iter__ 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] chunk = self.next()

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 494, in next 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] chunk = self._resp.read(CHUNKSIZE)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/httplib.py", line 561, in read 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] s = self.fp.read(amt)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/socket.py", line 380, in read 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] data = self._sock.recv(left)

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 262, in recv 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] timeout_exc=socket.timeout("timed out"))

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/eventlet/hubs/__init__.py", line 155, in trampoline 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] return hub.switch()

2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] return self.greenlet.switch() 2014-02-28 02:49:48.046 2136 TRACE nova.compute.manager [instance: 35d00082-4cb4-45a3-a73b-b72ef2a6e2b2] timeout: timed out

I don't know where the timeout is set. And does Nova download image from Glance in parallel or in sequential way?

Thank you very much.
Best regards,
Nora

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140304/6c7ddecf/attachment.html>