[openstack-dev] [Nova][Cinder] Multi-attach, determining when to call os-brick's connector.disconnect_volume

Walter A. Boring IV walter.boring at hpe.com
Tue Feb 9 19:49:33 UTC 2016

Hey folks,
    One of the challenges we have faced with the ability to attach a 
single volume to multiple instances, is how to correctly detach that 
volume.  The issue is a bit complex, but I'll try and explain the 
problem, and then describe one approach to solving one part of the 
detach puzzle.

   When a volume is attached to multiple instances on the same host. 
There are 2 scenarios here.

   1) Some Cinder drivers export a new target for every attachment on a 
compute host.  This means that you will get a new unique volume path on 
a host, which is then handed off to the VM instance.

   2) Other Cinder drivers export a single target for all instances on a 
compute host.  This means that every instance on a single host, will 
reuse the same host volume path.

When a user issues a request to detach a volume, the workflow boils down 
to first calling os-brick's connector.disconnect_volume before calling 
Cinder's terminate_connection and detach. disconnect_volume's job is to 
remove the local volume from the host OS and close any sessions.

There is no problem under scenario 1.  Each disconnect_volume only 
affects the attached volume in question and doesn't affect any other VM 
using that same volume, because they are using a different path that has 
shown up on the host.  It's a different target exported from the Cinder 

The problem comes under scenario 2, where that single volume is shared 
for every instance on the same compute host.  Nova needs to be careful 
and not call disconnect_volume if it's a shared volume, otherwise the 
first disconnect_volume call will nuke every instance's access to that 

Proposed solution:
   Nova needs to determine if the volume that's being detached is a 
shared or non shared volume.  Here is one way to determine that.

   Every Cinder volume has a list of it's attachments.  In those 
attachments it contains the instance_uuid that the volume is attached 
to.  I presume Nova can find which of the volume attachments are on the 
same host.  Then Nova can call Cinder's initialize_connection for each 
of those attachments to get the target's connection_info dictionary.  
This connection_info dictionary describes how to connect to the target 
on the cinder backend.  If the target is shared, then each of the 
connection_info dicts for each attachment on that host will be 
identical.  Then Nova would know that it's a shared target, and then 
only call os-brick's disconnect_volume, if it's the last attachment on 
that host.  I think at most 2 calls to cinder's initialize_connection 
would suffice to determine if the volume is a shared target.  This would 
only need to be done if the volume is multi-attach capable and if there 
are more than 1 attachments on the same host, where the detach is happening.


More information about the OpenStack-dev mailing list