Open Stack

On Tue, 23 Oct 2018 10:01:42 -0400, Jon Bernard wrote:
> * melanie witt <melwittt at gmail.com> wrote:
>> On Mon, 22 Oct 2018 11:45:55 +0800 (GMT+08:00), Boxiang Zhu wrote:
>>> I created a new vm and a new volume with type 'ceph'[So that the volume
>>> will be created on one of two hosts. I assume that the volume created on
>>> host dev at rbd-1#ceph this time]. Next step is to attach the volume to the
>>> vm. At last I want to migrate the volume from host dev at rbd-1#ceph to
>>> host dev at rbd-2#ceph, but it failed with the exception
>>> 'NotImplementedError(_("Swap only supports host devices")'.
>>>
>>> So that, my real problem is that is there any work to migrate
>>> volume(*in-use*)(*ceph rbd*) from one host(pool) to another host(pool)
>>> in the same ceph cluster?
>>> The difference between the spec[2] with my scope is only one is
>>> *available*(the spec) and another is *in-use*(my scope).
>>>
>>>
>>> [1] http://docs.ceph.com/docs/master/rbd/rbd-openstack/
>>> [2] https://review.openstack.org/#/c/296150
>>
>> Ah, I think I understand now, thank you for providing all of those details.
>> And I think you explained it in your first email, that cinder supports
>> migration of ceph volumes if they are 'available' but not if they are
>> 'in-use'. Apologies that I didn't get your meaning the first time.
>>
>> I see now the code you were referring to is this [3]:
>>
>> if volume.status not in ('available', 'retyping', 'maintenance'):
>>      LOG.debug('Only available volumes can be migrated using backend '
>>                'assisted migration. Falling back to generic migration.')
>>      return refuse_to_migrate
>>
>> So because your volume is not 'available', 'retyping', or 'maintenance',
>> it's falling back to generic migration, which will end up with an error in
>> nova because the source_path is not set in the volume config.
>>
>> Can anyone from the cinder team chime in about whether the ceph volume
>> migration could be expanded to allow migration of 'in-use' volumes? Is there
>> a reason not to allow migration of 'in-use' volumes?
> 
> Generally speaking, Nova must facilitate the migration of a live (or
> in-use) volume.  A volume attached to a running instance requires code
> in the I/O path to correctly route traffic to the correct location - so
> Cinder must refuse (or defer) a migrate operation if the volume is
> attached.  Until somewhat recently Qemu and Libvirt did not support the
> migration to non-block (RBD) targets which is the reason for lack of
> support.  I believe we now have all of the pieces to perform this
> operation successfully, but I suspect it will require a setup with
> correct versions of all the related software.  I will try to verify this
> during the current release cycle and report back.

OK, thanks for this info, Jon. I'll be interested in your findings.

Cheers,
-melanie