<div dir="ltr"><div><div><div>Hello,<br><br></div>what is the state of the instance before asking the snapshot ? Is it running or paused ?<br><br></div>Check on the hypervisor when the snapshot starts if you see files in these folders:<br><br>
/var/lib/libvirt/qemu/save/<br> /var/lib/nova/instances/<span class="">snapshots</span>/<br><br></div><div>How is your glance implemented ? Also with ceph rbd ? Remember that a "nova snapshot" is a glance image.<br></div><div><br></div>Saverio<br><br><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">2016-06-03 12:17 GMT+02:00 Grant <span dir="ltr"><<a href="mailto:grant@absolutedevops.io" target="_blank">grant@absolutedevops.io</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hi all,<br>
<br>
I was wondering if someone could shed any light on an issue we are
seeing. We are running Kilo in our production environment and when
we are trying to create a snapshot for a particular instance it
gets stuck in a "saving" state and doesn't actually ever save the
image.<br>
<br>
We are using a ceph back-end and the user that is trying to take a
snapshot is able to take snapshots of all of their other instances,
it is just one that is failing.<br>
<br>
Error log from the nova compute host below:<br>
<br>
2016-06-02 17:13:48.594 52559 WARNING urllib3.connectionpool
[req-8200a3b0-ad2a-406e-969e-c22762db3455
bb07f987fbae485c9e05f06fb0d422c2 a22e503869c34a92bceb66b0c1da7231 -
- -] HttpConnectionPool is full, discarding connection: 10.5.0.205<br>
2016-06-02 17:14:00.042 52559 ERROR nova.compute.manager
[req-8200a3b0-ad2a-406e-969e-c22762db3455
bb07f987fbae485c9e05f06fb0d422c2 a22e503869c34a92bceb66b0c1da7231 -
- -] [instance: 70d42d14-66f6-4374-9038-4b6f840193e0] Error while
trying<br>
to clean up image f9844dd5-5a92-4cd4-956d-8ad04cfc5e84<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] Traceback (most recent call
last):<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
405, in decorated_function<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0]
self.image_api.delete(context, image_id)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] File
"/usr/lib/python2.7/dist-packages/nova/image/api.py", line 141, in
delete<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] return
session.delete(context, image_id)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] File
"/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 410,
in delete<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] self._client.call(context,
1, 'delete', image_id)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] File
"/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 218,
in call<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] return
getattr(client.images, method)(*args, **kwargs)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] File
"/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line
255, in delete<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] resp, body =
self.client.delete(url)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] File
"/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line
271, in delete<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] return
self._request('DELETE', url, **kwargs)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] File
"/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line
227, in _request<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] raise
exc.from_response(resp, resp.content)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] HTTPUnauthorized: <html><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] <head><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] <title>401
Unauthorized</title><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] </head><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] <body><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] <h1>401
Unauthorized</h1><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] This server could not verify
that you are authorized to access the document you requested. Either
you supplied the wrong credential<br>
s (e.g., bad password), or your browser does not understand how to
supply the credentials required.<br /><br /><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] <br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] </body><br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] </html> (HTTP 401)<br>
2016-06-02 17:14:00.042 52559 TRACE nova.compute.manager [instance:
70d42d14-66f6-4374-9038-4b6f840193e0] <br>
2016-06-02 17:14:00.173 52559 ERROR oslo_messaging.rpc.dispatcher
[req-8200a3b0-ad2a-406e-969e-c22762db3455
bb07f987fbae485c9e05f06fb0d422c2 a22e503869c34a92bceb66b0c1da7231 -
- -] Exception during message handling: Not authorized for imag<br>
e f9844dd5-5a92-4cd4-956d-8ad04cfc5e84.<br>
<br>
Any help will be appreciated.<br>
<br>
Regards,<span class="HOEnZb"><font color="#888888"><br>
<br>
<div>-- <br>
<div style="min-height:13px"> </div>
<div> <font face="Lucida Grande, Verdana, Arial,
sans-serif" color="#A8D8FF"> <span style="font-size:11px;line-height:15px;float:left;margin-right:10px"> <img src="cid:part1.06040509.03080108@absolutedevops.io" style="color:#a8d8ff;font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:11px;line-height:15px;float:left;margin-top:0px;margin-right:10px"> </span>
</font>
<div style="margin-left:80px;line-height:15px;color:rgb(109,190,255);margin-bottom:5px"> <span style="font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;font-size:18px;font-weight:200;color:rgb(35,157,255)">Grant Morley</span> <span style="color:rgb(35,157,255);font-size:15px"> </span> <font face="Hind, Lucida Grande, Verdana, Arial, sans-serif"> <span style="font-size:15px"> </span> </font> </div>
<div style="margin-left:80px;line-height:15px;color:rgb(109,190,255);margin-bottom:10px"> <font face="Hind,
Lucida Grande, Verdana, Arial, sans-serif"> <span style="font-size:15px">Cloud Lead</span> </font> </div>
<div style="font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;font-size:13px;margin-left:80px;line-height:15px;color:rgb(153,153,153)"> <span style="color:#616161">Absolute</span> <span style="color:#98a5b0">DevOps
Ltd</span> <br>
Units H, J & K, Gateway 1000, Whittle Way, Stevenage,
Herts, SG1 2FP
</div>
<div style="font-family:Hind,helvetica,arial;font-size:13px;margin-left:80px"> <a href="http://www.absolutedevops.io/" style="color:#239dff;font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;line-height:15px;margin-right:6px;text-decoration:none" target="_blank"></a><a href="http://www.absolutedevops.io" target="_blank">www.absolutedevops.io</a> <a href="mailto:grant@absolutedevops.i" style="color:#239dff;font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;line-height:15px;margin-right:6px;text-decoration:none" target="_blank"></a><a href="mailto:grant@absolutedevops.io" target="_blank">grant@absolutedevops.io</a> <a style="color:#239dff;font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;line-height:15px;margin-right:6px">0845 874 0580</a> </div>
</div>
</div>
</font></span></div>
<br>_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br></blockquote></div><br></div>