[Openstack-operators] Instances failing to launch when rbd backed (ansible Liberty setup)

Grant Morley grant at absolutedevops.io
Fri Oct 21 14:23:21 UTC 2016


Hi Chris,

That bug suggests there has been a fix in 12.0.8. Hopefully 12.0.16 
should have that fixed still with a bit of luck , unless the fix hasn't 
been carried over.

I can see that the bug report says there is also a fix 12.0.9 and 12.0.11.

Do you know if there is a workaround we can try for this? As ideally I 
don't want to be downgrading the openstack_ansible version.

Regards,


On 21/10/16 14:09, Chris Sarginson wrote:
> It seems like it may be an occurrence of this bug, as you look to be 
> using python venvs:
>
> https://bugs.launchpad.net/openstack-ansible/+bug/1509837
>
> 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 
> 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File 
> "*/openstack/venvs/*nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", 
> line 117, in __init__
>
> Chris
>
> On Fri, 21 Oct 2016 at 13:19 Grant Morley <grant at absolutedevops.io 
> <mailto:grant at absolutedevops.io>> wrote:
>
>     Hi all,
>
>     We have a openstack-ansible setup and have ceph installed for the
>     backend. However whenever we try and launch a new instance it
>     fails to launch and we get the following error:
>
>     2016-10-21 12:08:06.241 70661 INFO nova.virt.libvirt.driver
>     [req-79811c40-8394-4e33-b16d-ff5fa7341b6a
>     41c60f65ae914681b6a6ca27a42ff780 324844c815084205995aff10b03a85e1
>     - - -] [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Creating image
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [req-79811c40-8394-4e33-b16d-ff5fa7341b6a
>     41c60f65ae914681b6a6ca27a42ff780 324844c815084205995aff10b03a85e1
>     - - -] [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Instance
>     failed to spawn
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Traceback (most
>     recent call last):
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/compute/manager.py",
>     line 2156, in _build_resources
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]     yield resources
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/compute/manager.py",
>     line 2009, in _build_and_run_instance
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]
>     block_device_info=block_device_info)
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",
>     line 2527, in spawn
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]
>     admin_pass=admin_password)
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",
>     line 2939, in _create_image
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] backend =
>     image('disk')
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",
>     line 2884, in image
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]     fname +
>     suffix, image_type)
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py",
>     line 967, in image
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] return
>     backend(instance=instance, disk_name=disk_name)
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py",
>     line 748, in __init__
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]
>     rbd_user=self.rbd_user)
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]   File
>     "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py",
>     line 117, in __init__
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f]     raise
>     RuntimeError(_('rbd python libraries not found'))
>     2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager
>     [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] RuntimeError: rbd
>     python libraries not found
>
>     It moans about the rbd python libraries not being found, however
>     all of the rbd libraries appear to be installed fine via apt. ( We
>     are running Ubuntu)
>
>     Compute host packages:
>
>     dpkg -l | grep ceph
>     ii  ceph-common 10.2.3-1trusty                        amd64       
>     common utilities to mount and interact with a ceph storage cluster
>     ii  libcephfs1 10.2.3-1trusty                        amd64       
>     Ceph distributed file system client library
>     ii  python-ceph 10.2.3-1trusty                        amd64
>     Meta-package for python libraries for the Ceph libraries
>     ii  python-cephfs 10.2.3-1trusty                       
>     amd64        Python libraries for the Ceph libcephfs library
>
>      dpkg -l | grep rbd
>     ii  librbd1 10.2.3-1trusty                        amd64       
>     RADOS block device client library
>     ii  python-rbd 10.2.3-1trusty                        amd64       
>     Python libraries for the Ceph librbd library
>
>     Has anyone come across this before? Ceph is working fine for
>     Glance, it just seems to be with the nova compute hosts.
>
>     Many thanks,
>
>
>     -- 
>     Grant Morley
>     Cloud Lead
>     Absolute DevOps Ltd
>     Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
>     www.absolutedevops.io <http://www.absolutedevops.io/>
>     grant at absolutedevops.io <mailto:grant at absolutedevops.i> 0845 874 0580
>     _______________________________________________
>     OpenStack-operators mailing list
>     OpenStack-operators at lists.openstack.org
>     <mailto:OpenStack-operators at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

-- 
Grant Morley
Cloud Lead
Absolute DevOps Ltd
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
www.absolutedevops.io <http://www.absolutedevops.io/> 
grant at absolutedevops.io <mailto:grant at absolutedevops.i> 0845 874 0580
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161021/650210b6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ado_new.png
Type: image/png
Size: 4369 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161021/650210b6/attachment.png>


More information about the OpenStack-operators mailing list