I'd like to propose the following api as a replacement. I'll describe the contract up top and put some example implementations below. This new flow for volume migration would be driven entirely by cinder, with no callbacks from nova, so all of the following are new nova apis.
Overall I really like this proposal from a high level. Some additional data points below that folks need to keep in mind as we think through how this could work.
Nova expects that dst is a copy of src at the time that volume-migration-start() was called. Nova will detach from src and attach dst in its place. Nova will resume reading and writing to dst immediately.
You actually mean volume-migration-complete, right?
The implementation on the cinder side would be:
volume-migration-start(src_attachment_id) copy(src, dst) volume-migration-complete(src, dst)
Cinder doesn't need to be concerned with whether the instance or running or not, and it always does the copy itself.
This could be problematic. This works great if the migration is from one volume to another volume on the same backend. In that case it could be very fast (for some backends almost instantaneous). If the migration needs to take place between two different backends, then we could have a problem. In that case, both the source and destination volumes would need to be mounted on some host to perform basically a dd from one to the other. The Nova host already has access to the volume, so doing it there avoids the need to set up any other access to the volumes. Otherwise we need to mount both volumes to the c-vol host to do the copy, putting Cinder in the data path. We have some existing gotchas with things like image copy for this that we've considered creating some kind of c-data data mover service that could be scaled out to multiple nodes and support different protocols. The biggest issue today is ensuring the Cinder node supports the storage transport protocol needed to access the volume - fibre channel HBAs or network connectivity for iSCSI. Not saying this is a showstopper. I'm just making sure folks are aware of it so we don't start off under any assumptions that having Cinder take care of the migration is going to be a no brainer.