[openstack-dev] [Cinder] [Nova]Do you think volume force delete operation should not apply to the volume being used?

John Griffith john.griffith at solidfire.com
Wed Feb 26 06:01:59 UTC 2014


On Tue, Feb 25, 2014 at 10:28 PM, yunling <yunlingzeng at hotmail.com> wrote:

> According to some further analysis, I tend to consider what I found out as
> a bug now...
>
>
>
> The execution flow of attaching a volume to an instance can be decoupled
> as the following steps:
>
>
>
> Step 1. Nova to Cinder: Volume reservation                  Volume status:
> attaching
>
> Step 2. Nova to Cinder: Connection initialization                 Volume
> status: attaching
>
> Step 3. Nova: Volume protocol checking                    Volume status:
> attaching
>
> Step 4. Nova connects to cinder-volume volume                Volume
> status: attaching
>
> Step 5. Nova adds a volume to a VM via libvirt. The VM can access this
> volume from now on.
>
>                                                                      Volume
> status: attaching (*but we can do "force delete" now!!*)
>
> Step 6. Nova to Cinder: Volume attaching and volume status update
> Volume status: attached
>
>
>
> To verify that, I tested Nova and Cinder by force deleting a volume being
> attached to an instance and in "attaching" status.
>
> After the volume is deleted, Disk IO errors just happened (I used "fdisk
> -l" to test it). A screen snapshot is attached below.
>
>
>
>
>
> Moreover, disk info was still left as below:
>
>
>
> Therefore, I think a "force delete" operation to a volume in "attaching"
> status is really risky. We need to get it fixed.
>
>
> Thanks!
> ------------------------------
> From: zhangyu11 at huawei.com
> To: openstack-dev at lists.openstack.org
> Date: Wed, 26 Feb 2014 00:56:17 +0000
> Subject: Re: [openstack-dev] [Cinder] [Nova]Do you think volume force
> delete operation should not apply to the volume being used?
>
>
>  If I understand your question correctly, the case you describe should be
> like the following:
>
>
>
> Assume we have created both an instance and a volume, then we try to
>  attach that volume to the instance.
>
> Before that operation is completed (the status of the volume is
> "attaching" now), for whatever reasons we decide to apply a "force delete"
> operation on that volume.
>
> Then, after we applied that force delete, we come to see that, from the
> Cinder side, the volume has been successfully deleted and the status is
> surely "deleted".
>
> However, from the Nova side, we see that the status of the deleted volume
> remains to be "attaching".
>
>
>
> If this is truly your case, I think it is a bug. The reason might lie in
> that, Cinder forgets to refresh the attach_status attribute of a volume in
> DB when applying a "force delete" operation.
>
> Is there any other suggestions?
>
>
>
> Thanks!
>
>
>
>
>
>
>
> *From:* yunling [mailto:yunlingzeng at hotmail.com]
> *Sent:* Monday, February 17, 2014 9:14 PM
> *To:* openstack-dev at lists.openstack.org
> *Subject:* [openstack-dev] [Cinder]Do you think volume force delete
> operation should not apply to the volume being used?
>
>
>
> Hi stackers:
>
>
>
>
> I found that volume status become inconsistent (nova volume status is
> attaching, verus cinder volume status is deleted) between nova and cinder
> when doing volume force delete operation on an attaching volume.
>
> I think volume force delete operation should not apply to the volume being
> used, which included the attached status of attaching, attached and
> detached.
>
>
>
>
> How do you think?
>
>
>
>
>
> thanks
>
> _______________________________________________ OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> The reason the attaching and detaching states are not on the black-list
for force-delete is because those are states that historically were
problematic.  Things can go wrong on the nova attach/detach side when
dealing with the bdm.  The reality is that detach/attach on the cinder side
is just a DB status update (and now in Icehouse a create/remove export).
 If you search the ML archives or the Cinder bug list for "stuck in
attaching/detaching" you'll find why we implemented it this way to begin
with.  Here's just one example [1].

I'd prefer we didn't remove the capability to force-delete attaching or
detaching.  If you take this away the only option left if something goes
wrong external to Cinder is to use the change-state command (not terrible,
but I'd rather call the force delete to make sure things like exports are
removed).

[1] https://lists.launchpad.net/openstack/msg13401.html

Thanks,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140225/3913bca6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 32106 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140225/3913bca6/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 68815 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140225/3913bca6/attachment-0003.png>


More information about the OpenStack-dev mailing list