<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mv="http://macVmlSchemaUri" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Lucida Grande";
panose-1:2 11 6 0 4 5 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
p.gmailmsg, li.gmailmsg, div.gmailmsg
{mso-style-name:gmail_msg;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
span.gmailmsg1
{mso-style-name:gmail_msg1;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Hi Grant/Chris,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Can you verify whether there is a symlink from inside the venv to the location of the RBD python libs in the system packages? If there isn’t one then you should be able to re-execute the
os-nova-install.yml playbook as it gets executed here: <a href="https://github.com/openstack/openstack-ansible/blob/liberty/playbooks/roles/os_nova/tasks/nova_compute_kvm_install.yml#L103-L123">
https://github.com/openstack/openstack-ansible/blob/liberty/playbooks/roles/os_nova/tasks/nova_compute_kvm_install.yml#L103-L123</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">If you can file a bug with some information regarding what actions led you to get to this point it’d be great for us to try and figure out what went wrong so that we can improve things
for the future.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Jesse<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">IRC: odyssey4me<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-family:Calibri;color:black">From:
</span></b><span style="font-family:Calibri;color:black">Grant Morley <grant@absolutedevops.io><br>
<b>Date: </b>Friday, October 21, 2016 at 3:23 PM<br>
<b>To: </b>Chris Sarginson <csargiso@gmail.com>, OpenStack Operators <openstack-operators@lists.openstack.org><br>
<b>Cc: </b>"ian.banks@serverchoice.com" <ian.banks@serverchoice.com><br>
<b>Subject: </b>Re: [Openstack-operators] Instances failing to launch when rbd backed (ansible Liberty setup)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<p style="margin-left:.5in">Hi Chris,<o:p></o:p></p>
<p style="margin-left:.5in">That bug suggests there has been a fix in 12.0.8. Hopefully 12.0.16 should have that fixed still with a bit of luck , unless the fix hasn't been carried over.<o:p></o:p></p>
<p style="margin-left:.5in">I can see that the bug report says there is also a fix 12.0.9 and 12.0.11.
<o:p></o:p></p>
<p style="margin-left:.5in">Do you know if there is a workaround we can try for this? As ideally I don't want to be downgrading the openstack_ansible version.<o:p></o:p></p>
<p style="margin-left:.5in">Regards,<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">On 21/10/16 14:09, Chris Sarginson wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:.5in">It seems like it may be an occurrence of this bug, as you look to be using python venvs:
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><a href="https://bugs.launchpad.net/openstack-ansible/+bug/1509837">https://bugs.launchpad.net/openstack-ansible/+bug/1509837</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:#212121">2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "<b>/openstack/venvs/</b>nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py",
line 117, in __init__</span> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Chris <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">On Fri, 21 Oct 2016 at 13:19 Grant Morley <<a href="mailto:grant@absolutedevops.io">grant@absolutedevops.io</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="gmailmsg" style="margin-left:.5in">Hi all,<o:p></o:p></p>
<p class="gmailmsg" style="margin-left:.5in">We have a openstack-ansible setup and have ceph installed for the backend. However whenever we try and launch a new instance it fails to launch and we get the following error:<o:p></o:p></p>
<p class="gmailmsg" style="mso-margin-top-alt:5.0pt;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">
2016-10-21 12:08:06.241 70661 INFO nova.virt.libvirt.driver [req-79811c40-8394-4e33-b16d-ff5fa7341b6a 41c60f65ae914681b6a6ca27a42ff780 324844c815084205995aff10b03a85e1 - - -] [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Creating image<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [req-79811c40-8394-4e33-b16d-ff5fa7341b6a 41c60f65ae914681b6a6ca27a42ff780 324844c815084205995aff10b03a85e1 - - -] [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Instance failed to spawn<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] Traceback (most recent call last):<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 2156, in _build_resources<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] yield resources<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] block_device_info=block_device_info)<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2527, in spawn<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] admin_pass=admin_password)<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2939, in _create_image<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] backend = image('disk')<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2884, in image<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] fname + suffix, image_type)<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 967, in image<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] return backend(instance=instance, disk_name=disk_name)<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 748, in __init__<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] rbd_user=self.rbd_user)<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "/openstack/venvs/nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 117, in __init__<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] raise RuntimeError(_('rbd python libraries not found'))<br>
2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] RuntimeError: rbd python libraries not found<o:p></o:p></p>
<p class="gmailmsg" style="margin-left:.5in">It moans about the rbd python libraries not being found, however all of the rbd libraries appear to be installed fine via apt. ( We are running Ubuntu)<o:p></o:p></p>
<p class="gmailmsg" style="margin-left:.5in">Compute host packages:<o:p></o:p></p>
<p class="gmailmsg" style="margin-left:.5in">dpkg -l | grep ceph<br>
ii ceph-common 10.2.3-1trusty amd64 common utilities to mount and interact with a ceph storage cluster<br>
ii libcephfs1 10.2.3-1trusty amd64 Ceph distributed file system client library<br>
ii python-ceph 10.2.3-1trusty amd64 Meta-package for python libraries for the Ceph libraries<br>
ii python-cephfs 10.2.3-1trusty amd64 Python libraries for the Ceph libcephfs library<o:p></o:p></p>
<p class="gmailmsg" style="margin-left:.5in"> dpkg -l | grep rbd<br>
ii librbd1 10.2.3-1trusty amd64 RADOS block device client library<br>
ii python-rbd 10.2.3-1trusty amd64 Python libraries for the Ceph librbd library<o:p></o:p></p>
<p class="gmailmsg" style="margin-left:.5in">Has anyone come across this before? Ceph is working fine for Glance, it just seems to be with the nova compute hosts.<o:p></o:p></p>
<p class="gmailmsg" style="margin-left:.5in">Many thanks,<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">-- <o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.5pt;font-family:"Lucida Grande";color:#A8D8FF"><img border="0" width="32" height="32" id="_x0000_i1025" src="cid:part1.9296627C.98B6FDF0@absolutedevops.io"></span><o:p></o:p></p>
<div style="margin-left:60.0pt;margin-bottom:3.75pt">
<p class="MsoNormal" style="margin-left:.5in;mso-line-height-alt:11.25pt"><span class="gmailmsg1"><span style="font-size:13.5pt;font-family:"Lucida Grande";color:#239DFF">Grant Morley</span></span><span style="color:#6DBEFF">
<o:p></o:p></span></p>
</div>
<div style="margin-left:60.0pt;margin-bottom:7.5pt">
<p class="MsoNormal" style="margin-left:.5in;mso-line-height-alt:11.25pt"><span class="gmailmsg1"><span style="font-size:11.5pt;font-family:"Lucida Grande";color:#6DBEFF">Cloud Lead</span></span><span style="font-family:"Lucida Grande";color:#6DBEFF">
</span><span style="color:#6DBEFF"><o:p></o:p></span></p>
</div>
<div style="margin-left:60.0pt">
<p class="MsoNormal" style="margin-left:.5in;line-height:11.25pt"><span class="gmailmsg1"><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#616161">Absolute</span></span><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#999999">
</span><span class="gmailmsg1"><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#98A5B0">DevOps Ltd</span></span><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#999999">
<br>
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP <o:p></o:p></span></p>
</div>
<div style="margin-left:60.0pt">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:Helvetica"><a href="http://www.absolutedevops.io/" target="_blank"><span style="font-family:"Lucida Grande";color:#239DFF;text-decoration:none">www.absolutedevops.io</span></a>
<a href="mailto:grant@absolutedevops.i" target="_blank"><span style="font-family:"Lucida Grande";color:#239DFF;text-decoration:none">grant@absolutedevops.io</span></a> 0845 874 0580
<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in">_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><o:p></o:p></p>
</blockquote>
</div>
</blockquote>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">-- <br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:8.5pt;font-family:"Lucida Grande";color:#A8D8FF"><img border="0" width="60" height="28" id="_x0000_i1026" src="cid:image001.png@01D22BB1.3B2EE320"></span><o:p></o:p></p>
<div style="margin-left:60.0pt;margin-bottom:3.75pt">
<p class="MsoNormal" style="margin-left:.5in;mso-line-height-alt:11.25pt"><span style="font-size:13.5pt;font-family:"Lucida Grande";color:#239DFF">Grant Morley</span><span style="color:#6DBEFF">
<o:p></o:p></span></p>
</div>
<div style="margin-left:60.0pt;margin-bottom:7.5pt">
<p class="MsoNormal" style="margin-left:.5in;mso-line-height-alt:11.25pt"><span style="font-size:11.5pt;font-family:"Lucida Grande";color:#6DBEFF">Cloud Lead</span><span style="font-family:"Lucida Grande";color:#6DBEFF">
</span><span style="color:#6DBEFF"><o:p></o:p></span></p>
</div>
<div style="margin-left:60.0pt">
<p class="MsoNormal" style="margin-left:.5in;line-height:11.25pt"><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#616161">Absolute</span><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#999999">
</span><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#98A5B0">DevOps Ltd</span><span style="font-size:10.0pt;font-family:"Lucida Grande";color:#999999">
<br>
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP <o:p></o:p></span></p>
</div>
<div style="margin-left:60.0pt">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:Helvetica"><a href="http://www.absolutedevops.io/"><span style="font-family:"Lucida Grande";color:#239DFF;text-decoration:none">www.absolutedevops.io</span></a>
<a href="mailto:grant@absolutedevops.i"><span style="font-family:"Lucida Grande";color:#239DFF;text-decoration:none">grant@absolutedevops.io</span></a> 0845 874 0580
<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
<hr>
Rackspace Limited is a company registered in England & Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy
- This e-mail message may contain confidential or privileged information intended for the recipient. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately
by e-mail at abuse@rackspace.com and delete the original message. Your cooperation is appreciated.
</body>
</html>