<div dir="ltr">I'm guessing that you are probably using multiple cinder-volume services to manage the glusterfs volumes. If this is the case, and you didn't set 'host' in cinder.conf to a common name between the cinder-volume services then you can end up in a case where you can't delete a volume because the registered 'host' is offline or no longer around.<div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Oct 27, 2015 at 12:43 AM Matt Riedemann <<a href="mailto:mriedem@linux.vnet.ibm.com">mriedem@linux.vnet.ibm.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
On 6/9/2015 11:28 AM, Jesse Keating wrote:<br>
> We've used<br>
> <a href="https://github.com/snemetz/openstack-scripts/blob/master/cinder-volume-delete.txt" rel="noreferrer" target="_blank">https://github.com/snemetz/openstack-scripts/blob/master/cinder-volume-delete.txt</a><br>
> with apparent success to delete stuck volumes.<br>
><br>
><br>
> - jlk<br>
><br>
> On Tue, Jun 9, 2015 at 6:37 AM, Alvise Dorigo <<a href="mailto:alvise.dorigo@pd.infn.it" target="_blank">alvise.dorigo@pd.infn.it</a><br>
> <mailto:<a href="mailto:alvise.dorigo@pd.infn.it" target="_blank">alvise.dorigo@pd.infn.it</a>>> wrote:<br>
><br>
> Hi,<br>
> I've a cinder volume which is permanently in "deleting" state. I<br>
> cannot retrace the full history of actions that brought to this<br>
> scenario. What I cat reporto is that<br>
><br>
> 1. Cinder is backed by GlusterFS mounted with the glusterfuse,<br>
> 2. a "cinder delete" (or "cinder force-delete" as admin) doesn't<br>
> produce any effect,<br>
> 3. In the api.log, scheduler.log and volume.log I do not see any<br>
> useful information but this:<br>
><br>
> api.log-20150607:2015-06-05 14:45:20.956 17383 <tel:20.956%2017383><br>
> INFO eventlet.wsgi.server [req-bed01706-77ff-48bb-b12c-a0ccdcfd2e25<br>
> d7b3d4f7d20444adb7fda140553c25bf 3beba6dd3f2648378263bc04d9c205fa -<br>
> - -] 192.135.16.31,192.168.60.21 - - [05/Jun/2015 14:45:20] "DELETE<br>
> /v1/3beba6dd3f2648378263bc04d9c205fa/volumes/b056b016-8337-4e64-9069-6c84ab242152<br>
> HTTP/1.1" 400 338 0.053790<br>
><br>
><br>
> (the verbosity and debug are ON of course).<br>
><br>
> This is not a single event... unfortunately more that one volume<br>
> cannot be deleted and we're, so far, unable to understand why.<br>
><br>
> My question is if a manual procedure (operating on the relevant<br>
> cinder's tables) which gracefully cleans things up (which is not a<br>
> trivial "delete from volume where id='...'") exists.<br>
><br>
> Thanks,<br>
><br>
> Alvise<br>
><br>
> _______________________________________________<br>
> OpenStack-operators mailing list<br>
> <a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
> <mailto:<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a>><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-operators mailing list<br>
> <a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
><br>
<br>
FYI, there is an effort going on here [1] to add a force volume<br>
attachment removal command to nova-manage. Operator input on that change<br>
would be valuable.<br>
<br>
[1] <a href="https://review.openstack.org/#/c/184537/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/184537/</a><br>
<br>
--<br>
<br>
Thanks,<br>
<br>
Matt Riedemann<br>
<br>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</blockquote></div><div dir="ltr">-- <br></div><div dir="ltr"><p dir="ltr">--</p><p dir="ltr"><span style="font-size:13.1999998092651px">Andrew Woodward</span></p><p dir="ltr"><span style="font-size:13.1999998092651px">Mirantis</span></p><p dir="ltr"><span style="font-size:13.1999998092651px">Fuel Community Ambassador</span></p><p dir="ltr"><span style="font-size:13.1999998092651px">Ceph Community</span></p>
</div>