Open Stack

On 05/30/2013 07:37 AM, Martin Mailand wrote:
> Hi Josh,
>
> I am trying to use ceph with openstack (grizzly), I have a multi host setup.
> I followed the instruction http://ceph.com/docs/master/rbd/rbd-openstack/.
> Glance is working without a problem.
> With cinder I can create and delete volumes without a problem.
>
> But I cannot boot from volumes.
> I doesn't matter if use horizon or the cli, the vm goes to the error state.
>
>  From the nova-compute.log I get this.
>
> 2013-05-30 16:08:45.224 ERROR nova.compute.manager
> [req-5679ddfe-79e3-4adb-b220-915f4a38b532
> 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc]
> [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Instance failed block
> device setup
> .....
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] ConnectionError: [Errno 101]
> ENETUNREACH
>
> What tries nova to reach? How could I debug that further?

It's trying to talk to the cinder api, and failing to connect at all.
Perhaps there's a firewall preventing that on the compute host, or
it's trying to use the wrong endpoint for cinder (check the keystone
service and endpoint tables for the volume service).

Josh

> Full Log included.
>
> -martin
>
> Log:
>
> ceph --version
> ceph version 0.61 (237f3f1e8d8c3b85666529860285dcdffdeda4c5)
>
> root at compute1:~# dpkg -l|grep -e ceph-common -e cinder
> ii  ceph-common                      0.61-1precise
>             common utilities to mount and interact with a ceph storage
> cluster
> ii  python-cinderclient              1:1.0.3-0ubuntu1~cloud0
>             python bindings to the OpenStack Volume API
>
>
> nova-compute.log
>
> 2013-05-30 16:08:45.224 ERROR nova.compute.manager
> [req-5679ddfe-79e3-4adb-b220-915f4a38b532
> 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc]
> [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Instance failed block
> device setup
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] Traceback (most recent call last):
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1071,
> in _prep_block_device
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     return
> self._setup_block_device_mapping(context, instance, bdms)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 721, in
> _setup_block_device_mapping
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     volume =
> self.volume_api.get(context, bdm['volume_id'])
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/nova/volume/cinder.py", line 193, in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]
> self._reraise_translated_volume_exception(volume_id)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/nova/volume/cinder.py", line 190, in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     item =
> cinderclient(context).volumes.get(volume_id)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/cinderclient/v1/volumes.py", line 180,
> in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     return self._get("/volumes/%s"
> % volume_id, "volume")
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/cinderclient/base.py", line 141, in _get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     resp, body =
> self.api.client.get(url)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 185, in get
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     return self._cs_request(url,
> 'GET', **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 153, in
> _cs_request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 123, in
> request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     return
> session.request(method=method, url=url, **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 279, in
> request
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     resp = self.send(prep,
> stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 374, in send
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     r = adapter.send(request,
> **kwargs)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]   File
> "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 206, in send
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]     raise ConnectionError(sockerr)
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc] ConnectionError: [Errno 101]
> ENETUNREACH
> 2013-05-30 16:08:45.224 19614 TRACE nova.compute.manager [instance:
> 059589a3-72fc-444d-b1f0-ab1567c725fc]
> 2013-05-30 16:08:45.329 AUDIT nova.compute.manager
> [req-5679ddfe-79e3-4adb-b220-915f4a38b532
> 8f9630095810427d865bc90c5ea04d35 43b2bbbf5daf4badb15d67d87ed2f3dc]
> [instance: 059589a3-72fc-444d-b1f0-ab1567c725fc] Terminating instance
>