On Tue, 23 Apr 2019 at 14:55, Sean McGinnis sean.mcginnis@gmx.com wrote:
I'd like to propose the following api as a replacement. I'll describe the contract up top and put some example implementations below. This new flow for volume migration would be driven entirely by cinder, with no callbacks from nova, so all of the following are new nova apis.
Overall I really like this proposal from a high level. Some additional data points below that folks need to keep in mind as we think through how this could work.
Nova expects that dst is a copy of src at the time that volume-migration-start() was called. Nova will detach from src and attach dst in its place. Nova will resume reading and writing to dst immediately.
You actually mean volume-migration-complete, right?
No. src goes read-only after start(), then cinder copies it to dst. So at the time cinder calls complete(), dst is going to be a copy of src from the time start() was called. cinder doesn't know (or need to know) anything about the overlay nova created locally. All cinder cares about is that nothing writes to src after it calls start(), and it must copy this data to dst.
The implementation on the cinder side would be:
volume-migration-start(src_attachment_id) copy(src, dst) volume-migration-complete(src, dst)
Cinder doesn't need to be concerned with whether the instance or running or not, and it always does the copy itself.
This could be problematic.
This works great if the migration is from one volume to another volume on the same backend. In that case it could be very fast (for some backends almost instantaneous).
If the migration needs to take place between two different backends, then we could have a problem. In that case, both the source and destination volumes would need to be mounted on some host to perform basically a dd from one to the other.
The Nova host already has access to the volume, so doing it there avoids the need to set up any other access to the volumes. Otherwise we need to mount both volumes to the c-vol host to do the copy, putting Cinder in the data path.
We have some existing gotchas with things like image copy for this that we've considered creating some kind of c-data data mover service that could be scaled out to multiple nodes and support different protocols. The biggest issue today is ensuring the Cinder node supports the storage transport protocol needed to access the volume - fibre channel HBAs or network connectivity for iSCSI.
Right, I was assuming there would be a fallback, and it would basically be as slow and ugly as if nova did it. The difference is that if nova does it it's always going to be slow as we can never take advantage of backend features, and it's always going to be ugly because storage isn't our thing, and we have no plans for an n-data data mover service ;)
Not saying this is a showstopper. I'm just making sure folks are aware of it so we don't start off under any assumptions that having Cinder take care of the migration is going to be a no brainer.
Understood.
Matt