[openstack-dev] [Nova][Cinder] Multi-attach, determining when to call os-brick's connector.disconnect_volume
Walter A. Boring IV
walter.boring at hpe.com
Tue Feb 9 19:49:33 UTC 2016
One of the challenges we have faced with the ability to attach a
single volume to multiple instances, is how to correctly detach that
volume. The issue is a bit complex, but I'll try and explain the
problem, and then describe one approach to solving one part of the
When a volume is attached to multiple instances on the same host.
There are 2 scenarios here.
1) Some Cinder drivers export a new target for every attachment on a
compute host. This means that you will get a new unique volume path on
a host, which is then handed off to the VM instance.
2) Other Cinder drivers export a single target for all instances on a
compute host. This means that every instance on a single host, will
reuse the same host volume path.
When a user issues a request to detach a volume, the workflow boils down
to first calling os-brick's connector.disconnect_volume before calling
Cinder's terminate_connection and detach. disconnect_volume's job is to
remove the local volume from the host OS and close any sessions.
There is no problem under scenario 1. Each disconnect_volume only
affects the attached volume in question and doesn't affect any other VM
using that same volume, because they are using a different path that has
shown up on the host. It's a different target exported from the Cinder
The problem comes under scenario 2, where that single volume is shared
for every instance on the same compute host. Nova needs to be careful
and not call disconnect_volume if it's a shared volume, otherwise the
first disconnect_volume call will nuke every instance's access to that
Nova needs to determine if the volume that's being detached is a
shared or non shared volume. Here is one way to determine that.
Every Cinder volume has a list of it's attachments. In those
attachments it contains the instance_uuid that the volume is attached
to. I presume Nova can find which of the volume attachments are on the
same host. Then Nova can call Cinder's initialize_connection for each
of those attachments to get the target's connection_info dictionary.
This connection_info dictionary describes how to connect to the target
on the cinder backend. If the target is shared, then each of the
connection_info dicts for each attachment on that host will be
identical. Then Nova would know that it's a shared target, and then
only call os-brick's disconnect_volume, if it's the last attachment on
that host. I think at most 2 calls to cinder's initialize_connection
would suffice to determine if the volume is a shared target. This would
only need to be done if the volume is multi-attach capable and if there
are more than 1 attachments on the same host, where the detach is happening.
More information about the OpenStack-dev