[Openstack-operators] Instances failing to launch when rbd backed (ansible Liberty setup)

Grant Morley grant at absolutedevops.io
Fri Oct 21 15:49:09 UTC 2016


Thanks Kris,

I have run the commands as suggested and there is no rbd installed. 
However when we try to manually install rbd with pip we get the 
following error:

pip install rbd
DEPRECATION: --allow-all-external has been deprecated and will be 
removed in the future. Due to changes in the repository protocol, it no 
longer has any effect.
Ignoring indexes: https://pypi.python.org/simple
Collecting rbd
   Could not find a version that satisfies the requirement rbd (from 
versions: )
No matching distribution found for rbd

I assume the playbooks are coming across the same issue which is why we 
are having this problem. I will also ask in the #openstack-ansible 
channel for some help.

Pip is using the local repo servers that are created by 
openstack-ansible (I assume) so looking at this, it appears they don't 
have the correct packages.

Regards,



On 21/10/16 16:42, Kris G. Lindgren wrote:
>
> From the traceback it looks like nova-compute is running out of a venv.
>
> You need to activate the venv, most likely via: source 
> /openstack/venvs/nova-12.0.16/.venv/bin/activate then run: pip 
> freeze.  If you don’t see the RBD stuff – then that is your issue.  
> You might be able to fix via: pip install rbd.
>
> Venv’s are self-contained python installs, so they do not use the 
> system level python packages at all.
>
> I would also ask for some help in the #openstack-ansible channel on 
> irc as well.
>
> ___________________________________________________________________
>
> Kris Lindgren
>
> Senior Linux Systems Engineer
>
> GoDaddy
>
> *From: *Grant Morley <grant at absolutedevops.io>
> *Date: *Friday, October 21, 2016 at 6:14 AM
> *To: *OpenStack Operators <openstack-operators at lists.openstack.org>
> *Cc: *"ian.banks at serverchoice.com" <ian.banks at serverchoice.com>
> *Subject: *[Openstack-operators] Instances failing to launch when rbd 
> backed (ansible Liberty setup)
>
> Hi all,
>
> We have a openstack-ansible setup and have ceph installed for the 
> backend. However whenever we try and launch a new instance it fails to 
> launch and we get the following error:
>
> 2016-10-21 12:08:06.241 70661 INFO nova.virt.libvirt.driver 
> [req-79811c40-8394-4e33-b16d-ff5fa7341b6a 
> 41c60f65ae914681b6a6ca27a42ff780 324844c815084205995aff10b03a85e1 - - 
> -] [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Creating image
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager 
> [req-79811c40-8394-4e33-b16d-ff5fa7341b6a 
> 41c60f65ae914681b6a6ca27a42ff780 324844c815084205995aff10b03a85e1 - - 
> -] [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Instance failed to 
> spawn
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f] Traceback (most recent call last):
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/compute/manager.py", 
> line 2156, in _build_resources
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]     yield resources
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/compute/manager.py", 
> line 2009, in _build_and_run_instance
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f] block_device_info=block_device_info)
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", 
> line 2527, in spawn
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f] admin_pass=admin_password)
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", 
> line 2939, in _create_image
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]     backend = image('disk')
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", 
> line 2884, in image
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]     fname + suffix, image_type)
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", 
> line 967, in image
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]     return 
> backend(instance=instance, disk_name=disk_name)
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", 
> line 748, in __init__
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f] rbd_user=self.rbd_user)
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", 
> line 117, in __init__
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]     raise RuntimeError(_('rbd 
> python libraries not found'))
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f] RuntimeError: rbd python 
> libraries not found
>
> It moans about the rbd python libraries not being found, however all 
> of the rbd libraries appear to be installed fine via apt. ( We are 
> running Ubuntu)
>
> Compute host packages:
>
> dpkg -l | grep ceph
> ii  ceph-common 10.2.3-1trusty                        amd64        
> common utilities to mount and interact with a ceph storage cluster
> ii  libcephfs1 10.2.3-1trusty                        amd64        Ceph 
> distributed file system client library
> ii  python-ceph 10.2.3-1trusty                        amd64 
> Meta-package for python libraries for the Ceph libraries
> ii  python-cephfs 10.2.3-1trusty                        amd64        
> Python libraries for the Ceph libcephfs library
>
>  dpkg -l | grep rbd
> ii  librbd1 10.2.3-1trusty                        amd64        RADOS 
> block device client library
> ii  python-rbd 10.2.3-1trusty                        amd64        
> Python libraries for the Ceph librbd library
>
> Has anyone come across this before? Ceph is working fine for Glance, 
> it just seems to be with the nova compute hosts.
>
> Many thanks,
>
> -- 
>
> Grant Morley
>
> Cloud Lead
>
> AbsoluteDevOps Ltd
> Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
>
> www.absolutedevops.io <http://www.absolutedevops.io/> 
> grant at absolutedevops.io <mailto:grant at absolutedevops.i> 0845 874 0580
>

-- 
Grant Morley
Cloud Lead
Absolute DevOps Ltd
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
www.absolutedevops.io <http://www.absolutedevops.io/> 
grant at absolutedevops.io <mailto:grant at absolutedevops.i> 0845 874 0580
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161021/1a6f7923/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 4370 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161021/1a6f7923/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ado_new.png
Type: image/png
Size: 4369 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161021/1a6f7923/attachment-0001.png>


More information about the OpenStack-operators mailing list