<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<style>
font{
line-height: 1.6;
}
ul,ol{
padding-left: 20px;
list-style-position: inside;
}
</style>
<div style="font-family:微软雅黑,Verdana,"Microsoft Yahei",SimSun,sans-serif; line-height:1.6;">
<div>
<div>
<span>
<br>
</span>
</div>
<div>Jay and Melanie, It's my fault to let you misunderstand the problem. <span style="line-height: 22.4px;">I should describe my problem more clearly.</span><span style="line-height: 22.4px;"> </span><span style="line-height: 1.6;">My problem </span><b style="line-height: 1.6;">is not</b><span style="line-height: 1.6;"> to migrate volumes between </span><b style="line-height: 1.6;">two ceph clusters</b><span style="line-height: 1.6;">. </span></div><div><span><br></span></div><div><span>I have two clusters, one is openstack cluster(allinone env, hostname is dev) and another is ceph cluster. Omit the integrated configurations for openstack and ceph.[1] The special config of cinder.conf is as followed:</span></div><div><span><font size="2" style="font-size: 12px;"><br></font></span></div><div><span><font size="2" style="font-size: 12px;">[DEFAULT]
</font></span></div><div><span><font size="2" style="font-size: 12px;"><b>enabled_backends = rbd-1,rbd-2
</b></font></span></div><div><span><font size="2" style="font-size: 12px;">......</font></span></div><div><div><font size="2" style="font-size: 12px;"><b>[rbd-1]</b></font></div><div><font size="2" style="font-size: 12px;">volume_driver = cinder.volume.drivers.rbd.RBDDriver</font></div><div><font size="2" style="font-size: 12px;"><b>volume_backend_name = ceph</b></font></div><div><font size="2" style="font-size: 12px;"><b>rbd_pool = volumes001</b></font></div><div><font size="2" style="font-size: 12px;">rbd_ceph_conf = /etc/ceph/ceph.conf</font></div><div><font size="2" style="font-size: 12px;">rbd_flatten_volume_from_snapshot = true</font></div><div><font size="2" style="font-size: 12px;">rbd_max_clone_depth = 2</font></div><div><font size="2" style="font-size: 12px;">rbd_store_chunk_size = 4</font></div><div><font size="2" style="font-size: 12px;">rados_connect_timeout = 5</font></div><div><font size="2" style="font-size: 12px;">rbd_user = cinder</font></div><div><font size="2" style="font-size: 12px;">rbd_secret_uuid = 86d3922a-b471-4dc1-bb89-b46ab7024e81</font></div><div><span style="line-height: 1.6;"><font size="2" style="font-size: 12px;"><b>[rbd-2]</b></font></span></div><div><font size="2" style="font-size: 12px;">volume_driver = cinder.volume.drivers.rbd.RBDDriver</font></div><div><font size="2" style="font-size: 12px;"><b>volume_backend_name = ceph</b></font></div><div><font size="2" style="font-size: 12px;"><b>rbd_pool = volumes002</b></font></div><div><font size="2" style="font-size: 12px;">rbd_ceph_conf = /etc/ceph/ceph.conf</font></div><div><font size="2" style="font-size: 12px;">rbd_flatten_volume_from_snapshot = true</font></div><div><font size="2" style="font-size: 12px;">rbd_max_clone_depth = 2</font></div><div><font size="2" style="font-size: 12px;">rbd_store_chunk_size = 4</font></div><div><font size="2" style="font-size: 12px;">rados_connect_timeout = 5</font></div><div><font size="2" style="font-size: 12px;">rbd_user = cinder</font></div><div><font size="2" style="font-size: 12px;">rbd_secret_uuid = 86d3922a-b471-4dc1-bb89-b46ab7024e81</font></div><div><font size="2" style="font-size: 14px;"><br></font></div><div><font size="2" style="font-size: 14px;">There will be two hosts named <b>dev@rbd-1#ceph</b> and <b>dev@rbd-2#ceph</b>.</font></div><div>Then I create a volume type named 'ceph' with the command 'cinder type-create ceph' and add extra_spec 'volume_backend_name=ceph'<span style="line-height: 22.4px;"> </span><span style="line-height: 22.4px;">for it</span><span style="line-height: 1.6;"> with the command 'cinder type-key <vtype> set volume_backend_name=ceph'. </span></div></div><div><span style="line-height: 1.6;"><br></span></div><div><span>I created a new vm and a new volume with type 'ceph'[So that the volume will be created on one of two hosts. I assume that the volume created on host dev@rbd-1#ceph this time</span><span style="line-height: 1.6;">]. Next step is to attach the volume to the vm. At last I want to migrate the volume from host dev@rbd-1#ceph to host dev@rbd-2#ceph, but it failed with the exception 'NotImplementedError(_("Swap only supports host devices")'.</span></div><div><span style="line-height: 1.6;"><br></span></div><div><span style="line-height: 1.6;">So that, my real problem is that is there any work to migrate volume(<b>in-use</b>)(<b>ceph rbd</b>) from one host(pool) to another host(pool) in the same ceph cluster?</span></div><div><span style="line-height: 1.6;">The difference between the spec[2] with my scope is only one is <b>available</b>(the spec) and another is <b>in-use</b>(my scope).</span></div><div><span><br></span></div><div><span><br></span></div><div><span>[1] </span><a href="http://docs.ceph.com/docs/master/rbd/rbd-openstack/" _src="http://docs.ceph.com/docs/master/rbd/rbd-openstack/" style="line-height: 1.6;">http://docs.ceph.com/docs/master/rbd/rbd-openstack/</a></div><div>[2] <a href="https://review.openstack.org/#/c/296150" _src="https://review.openstack.org/#/c/296150" moz-do-not-send="true" style="line-height: 1.6;">https://review.openstack.org/#/c/296150</a></div><div><br></div><div>Cheers,</div><div>Boxiang</div>
<div style="font-family:'微软雅黑'">
<div style="font-size:14px; padding: 0; margin:0;">
</div>
</div>
</div><div class="J-reply" style="background-color:#f2f2f2;color:black;padding-top:6px;padding-bottom:6px;border-radius:3px;-moz-border-radius:3px;-webkit-border-radius:3px;margin-top:45px;margin-bottom:20px;font-family:'微软雅黑';">
<div style="font-size:12px;line-height:1.5;word-break:break-all;margin-left:10px;margin-right:10px">On <span class="mail-date">10/21/2018 23:19</span>，<a class="mail-to" style="text-decoration:none;color:#2a83f2;" href="mailto:jungleboyj@gmail.com">Jay S. Bryant<jungleboyj@gmail.com></a> wrote： </div>
</div>
<blockquote id="ntes-pcmail-quote" style="margin: 0; padding: 0; font-size: 14px; font-family: '微软雅黑';">
<p>Boxiang,</p>
<p>I have not herd any discussion of extending this functionality
for Ceph to work between different Ceph Clusters. I wasn't aware,
however, that the existing spec was limited to one Ceph cluster.
So, that is good to know.</p>
<p>I would recommend reaching out to Jon Bernard or Eric Harney for
guidance on how to proceed. They work closely with the Ceph
driver and could provide insight.</p>
<p>Jay<br>
</p>
<br>
<div class="moz-cite-prefix">On 10/19/2018 10:21 AM, Boxiang Zhu
wrote:<br>
</div>
<blockquote type="cite" cite="mid:47653a27.59b1.1668cea5d9b.Coremail.bxzhu_5355@163.com">
<style>
font{
line-height: 1.6;
}
ul,ol{
padding-left: 20px;
list-style-position: inside;
}
</style>
<div style="font-family:微软雅黑,Verdana,"Microsoft
Yahei",SimSun,sans-serif; line-height:1.6;">
<div>
<div> <span> <br>
</span> </div>
<div><span style="line-height: 22.4px;"><font style="font-size: 14px;" size="2">Hi melanie, thanks for
your reply.</font></span></div>
<font style="font-size: 14px;" size="2"> </font>
<div style="font-family:'微软雅黑'">
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2"><br>
</font></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2">The version of my
cinder and nova is Rocky. <span style="line-height:
1.6;">The scope of the cinder spec[1] </span></font></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><span style="line-height: 1.6;"><font style="font-size: 14px;" size="2">is only for available volume migration
between two pools from the same ceph cluster.</font></span></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2">If the volume is
in-use status[2], it will call the generic migration
function. <span style="line-height: 1.6;">So that as
you </span></font></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><span style="line-height: 1.6;"><font style="font-size: 14px;" size="2">describe it, on the nova side, it raises
NotImplementedError(_("Swap only supports host
devices"). </font></span></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2">The get_config of net
volume[3] has not source_path.</font></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2"><br>
</font></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2">So does anyone try to
succeed to migrate volume(in-use) with ceph backend or
is anyone doing something of it?</font></div>
<div style="font-size:14px; padding: 0;
margin:0;line-height:14px;"><br>
</div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2">[1] <a href="https://review.openstack.org/#/c/296150" _src="https://review.openstack.org/#/c/296150" style="line-height: 1.6;" moz-do-not-send="true">https://review.openstack.org/#/c/296150</a></font></div>
<div style="font-size:14px; padding: 0;
margin:0;line-height:14px;">[2] <a href="https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py" _src="https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py" style="line-height: 1.6;" moz-do-not-send="true">https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py</a></div>
<div style="font-size:14px; padding: 0;
margin:0;line-height:14px;">[3] <a href="https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101" _src="https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101" style="line-height: 1.6;" moz-do-not-send="true">https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101</a></div>
<div style="font-size:14px; padding: 0;
margin:0;line-height:14px;"><br>
</div>
<div style="font-size:14px; padding: 0;
margin:0;line-height:14px;"><br>
</div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2">Cheers,</font></div>
<div style="padding: 0px; margin: 0px; line-height: 14px;"><font style="font-size: 14px;" size="2">Boxiang</font></div>
</div>
</div>
<div class="J-reply" style="background-color:#f2f2f2;color:black;padding-top:6px;padding-bottom:6px;border-radius:3px;-moz-border-radius:3px;-webkit-border-radius:3px;margin-top:45px;margin-bottom:20px;font-family:'微软雅黑';">
<div style="font-size:12px;line-height:1.5;word-break:break-all;margin-left:10px;margin-right:10px">On
<span class="mail-date">10/19/2018 22:39</span>，<a class="mail-to" style="text-decoration:none;color:#2a83f2;" href="mailto:melwittt@gmail.com" moz-do-not-send="true">melanie
witt<melwittt@gmail.com></a> wrote： </div>
</div>
<blockquote id="ntes-pcmail-quote" style="margin: 0; padding: 0;
font-size: 14px; font-family: '微软雅黑';">
On Fri, 19 Oct 2018 11:33:52 +0800 (GMT+08:00), Boxiang Zhu
wrote:<br>
<blockquote class="mmbqc1">When I use the LVM backend to
create the volume, then attach it to a vm. <br>
I can migrate the volume(in-use) from one host to another.
The nova <br>
libvirt will call the 'rebase' to finish it. But if using
ceph backend, <br>
it raises exception 'Swap only supports host devices'. So
now it does <br>
not support to migrate volume(in-use). Does anyone do this
work now? Or <br>
Is there any way to let me migrate volume(in-use) with ceph
backend?<br>
</blockquote>
<br>
What version of cinder and nova are you using?<br>
<br>
I found this question/answer on ask.openstack.org:<br>
<br>
<a class="moz-txt-link-freetext" href="https://ask.openstack.org/en/question/112954/volume-migration-fails-notimplementederror-swap-only-supports-host-devices/">https://ask.openstack.org/en/question/112954/volume-migration-fails-notimplementederror-swap-only-supports-host-devices/</a><br>
<br>
and it looks like there was some work done on the cinder side
[1] to <br>
enable migration of in-use volumes with ceph semi-recently
(Queens).<br>
<br>
On the nova side, the code looks for the source_path in the
volume <br>
config, and if there is not one present, it raises <br>
NotImplementedError(_("Swap only supports host devices"). So
in your <br>
environment, the volume configs must be missing a source_path.<br>
<br>
If you are using at least Queens version, then there must be
something <br>
additional missing that we would need to do to make the
migration work.<br>
<br>
[1]
<a class="moz-txt-link-freetext" href="https://blueprints.launchpad.net/cinder/+spec/ceph-volume-migrate">https://blueprints.launchpad.net/cinder/+spec/ceph-volume-migrate</a><br>
<br>
Cheers,<br>
-melanie<br>
<br>
<br>
<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br></blockquote><!--😀-->
</div>
</body>
</html>