<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:756445432;
mso-list-template-ids:930399724;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">There is another fix proposed by Eli Qiao long time ago:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><a href="https://review.openstack.org/#/c/310707">https://review.openstack.org/#/c/310707</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Basically it blocks block live migration with BDMs and tunneling during pre-checks.
<o:p></o:p></span></p>
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></a></p>
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Timofei Durakov [mailto:tdurakov@mirantis.com]
<br>
<b>Sent:</b> Tuesday, June 7, 2016 9:04 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br>
<b>Subject:</b> Re: [openstack-dev] [nova][libvirt] Deprecating the live_migration_flag and block_migration_flag config options<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">I've submitted one more patch with potential fix:<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://review.openstack.org/#/c/326258/">https://review.openstack.org/#/c/326258/</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Timofey<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Mon, Jun 6, 2016 at 11:58 PM, Timofei Durakov <<a href="mailto:tdurakov@mirantis.com" target="_blank">tdurakov@mirantis.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<p class="MsoNormal">On Mon, Jun 6, 2016 at 11:26 PM, Matt Riedemann <<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a>> wrote:<o:p></o:p></p>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal">On 6/6/2016 12:15 PM, Matt Riedemann wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">On 1/8/2016 12:28 PM, Mark McLoughlin wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">On Fri, 2016-01-08 at 14:11 +0000, Daniel P. Berrange wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">On Thu, Jan 07, 2016 at 09:07:00PM +0000, Mark McLoughlin wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">On Thu, 2016-01-07 at 12:23 +0100, Sahid Orentino Ferdjaoui<br>
wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">On Mon, Jan 04, 2016 at 09:12:06PM +0000, Mark McLoughlin<br>
wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">Hi<br>
<br>
commit 8ecf93e[1] got me thinking - the live_migration_flag<br>
config option unnecessarily allows operators choose arbitrary<br>
behavior of the migrateToURI() libvirt call, to the extent<br>
that we allow the operator to configure a behavior that can<br>
result in data loss[1].<br>
<br>
I see that danpb recently said something similar:<br>
<br>
<a href="https://review.openstack.org/171098" target="_blank">https://review.openstack.org/171098</a><br>
<br>
"Honestly, I wish we'd just kill off 'live_migration_flag'<br>
and 'block_migration_flag' as config options. We really<br>
should not be exposing low level libvirt API flags as admin<br>
tunable settings.<br>
<br>
Nova should really be in charge of picking the correct set of<br>
flags for the current libvirt version, and the operation it<br>
needs to perform. We might need to add other more sensible<br>
config options in their place [..]"<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Nova should really handle internal flags and this serie is<br>
running in the right way.<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">...<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">4) Add a new config option for tunneled versus native:<br>
<br>
[libvirt] live_migration_tunneled = true<br>
<br>
This enables the use of the VIR_MIGRATE_TUNNELLED flag. We<br>
have historically defaulted to tunneled mode because it<br>
requires the least configuration and is currently the only<br>
way to have a secure migration channel.<br>
<br>
danpb's quote above continues with:<br>
<br>
"perhaps a "live_migration_secure_channel" to indicate that<br>
migration must use encryption, which would imply use of<br>
TUNNELLED flag"<br>
<br>
So we need to discuss whether the config option should<br>
express the choice of tunneled vs native, or whether it<br>
should express another choice which implies tunneled vs<br>
native.<br>
<br>
<a href="https://review.openstack.org/263434" target="_blank">https://review.openstack.org/263434</a><o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
We probably have to consider that operator does not know much<br>
about internal libvirt flags, so options we are exposing for<br>
him should reflect benefice of using them. I commented on your<br>
review we should at least explain benefice of using this option<br>
whatever the name is.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
As predicted, plenty of discussion on this point in the review<br>
:)<br>
<br>
You're right that we don't give the operator any guidance in the<br>
help message about how to choose true or false for this:<br>
<br>
Whether to use tunneled migration, where migration data is<br>
transported over the libvirtd connection. If True, we use the<br>
VIR_MIGRATE_TUNNELLED migration flag<br>
<br>
libvirt's own docs on this are here:<br>
<br>
<a href="https://libvirt.org/migration.html#transport" target="_blank">https://libvirt.org/migration.html#transport</a><br>
<br>
which emphasizes:<br>
<br>
- the data copies involved in tunneling - the extra configuration<br>
steps required for native - the encryption support you get when<br>
tunneling<br>
<br>
The discussions I've seen on this topic wrt Nova have revolved<br>
around:<br>
<br>
- that tunneling allows for an encrypted transport[1] - that<br>
qemu's NBD based drive-mirror block migration isn't supported<br>
using tunneled mode, and that danpb is working on fixing this<br>
limitation in libvirt - "selective" block migration[2] won't work<br>
with the fallback qemu block migration support, and so won't<br>
currently work in tunneled mode<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
I'm not working on fixing it, but IIRC some other dev had proposed<br>
patches.<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><br>
So, the advise to operators would be:<br>
<br>
- You may want to choose tunneled=False for improved block<br>
migration capabilities, but this limitation will go away in<br>
future. - You may want to choose tunneled=False if you wish to<br>
trade and encrypted transport for a (potentially negligible)<br>
performance improvement.<br>
<br>
Does that make sense?<br>
<br>
As for how to name the option, and as I said in the review, I<br>
think it makes sense to be straightforward here and make it<br>
clearly about choosing to disable libvirt's tunneled transport.<br>
<br>
If we name it any other way, I think our explanation for<br>
operators will immediately jump to explaining (a) that it<br>
influences the TUNNELLED flag, and (b) the differences between<br>
the tunneled and native transports. So, if we're going to have to<br>
talk about tunneled versus native, why obscure that detail?<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Ultimately we need to recognise that libvirt's tunnelled mode was<br>
added as a hack, to work around fact that QEMU lacked any kind of<br>
native security capabilities & didn't appear likely to ever get<br>
them at that time. As well as not working with modern NBD based<br>
block device encryption, it really sucks for performance because it<br>
introduces many extra data copies. So it is going to be quite poor<br>
for large VMs with heavy rate of data dirtying.<br>
<br>
The only long term relative "benefit" of tunnelled mode is that it<br>
avoids the need to open extra firewall ports.<br>
<br>
IMHO, the long term future is to *never* use tunnelled mode for<br>
QEMU. This will be viable when my support for native TLS support in<br>
QEMU migration + NBD protocols is merged. I'm hopeful this wil be<br>
for QEMU 2.6<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">But, Pawel strongly disagrees.<br>
<br>
One last point I'd make is this isn't about adding a *new*<br>
configuration capability for operators. As we deprecate and<br>
remove these configuration options, we need to be careful not to<br>
remove a capability that operators are currently depending on for<br>
arguably reasonable reasons.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
My view is that "live_migration_tunneled" is a reasonable<br>
parameter to add, because there is a genuine need to let admins<br>
choose this behaviour. We should make sure it is correctly done as<br>
a tri-state flag though, when it is 'None', Nova should pick what<br>
it things is the best approach based on QEMU version. Probably to<br>
use QEMU native when it supports TLS, otherwise use tunnelled if<br>
possible to get security.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Great feedback. I buy that.<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal" style="margin-bottom:12.0pt">[1] - <a href="https://review.openstack.org/#/c/171098/" target="_blank">
https://review.openstack.org/#/c/171098/</a> [2] -<br>
<a href="https://review.openstack.org/#/c/227278" target="_blank">https://review.openstack.org/#/c/227278</a><br>
<br>
<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">5) Add a new config option for additional migration flags:<br>
<br>
[libvirt] live_migration_extra_flags =<br>
VIR_MIGRATE_COMPRESSED<br>
<br>
This allows operators to continue to experiment with libvirt<br>
behaviors in safe ways without each use case having to be<br>
accounted for.<br>
<br>
<a href="https://review.openstack.org/263435" target="_blank">https://review.openstack.org/263435</a><br>
<br>
We would disallow setting the following flags via this<br>
option:<br>
<br>
VIR_MIGRATE_LIVE VIR_MIGRATE_PEER2PEER VIR_MIGRATE_TUNNELLED<br>
VIR_MIGRATE_PERSIST_DEST VIR_MIGRATE_UNDEFINE_SOURCE<br>
VIR_MIGRATE_NON_SHARED_INC VIR_MIGRATE_NON_SHARED_DISK<br>
<br>
which would allow the following currently available flags to<br>
be set:<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">VIR_MIGRATE_PAUSED VIR_MIGRATE_CHANGE_PROTECTION<br>
VIR_MIGRATE_UNSAFE VIR_MIGRATE_OFFLINE<br>
VIR_MIGRATE_COMPRESSED VIR_MIGRATE_ABORT_ON_ERROR<br>
VIR_MIGRATE_AUTO_CONVERGE VIR_MIGRATE_RDMA_PIN_ALL<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
We can probably consider to provide VIR_MIGRATE_PAUSED and<br>
VIR_MIGRATE_COMPRESSED as dedicated options too ?<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
Yes, I think any options we see regularly added to extra_flags<br>
by operators, and as we understand the use cases better, then we<br>
can add dedicated options for them.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
I really don't see a case for letting the admin set<br>
VIR_MIGRATE_PAUSED at a host level. If we want the ability to force<br>
a running VM to end up paused after migration, this is a feature to<br>
be added to the Nova migration API.<br>
<br>
The VIR_MIGRATE_COMPRESSED is not as simple as just enabling a<br>
flag, there are other associated runtime tunables that need<br>
setting. There was a spec discussing this which was not approved as<br>
a suitable strategy for using it could not be agreed.<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">In the review, Pawel is making a case for not allowing the<br>
operator to enable COMPRESSED or AUTO_CONVERGE.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
I agree really. As per my comments, I in fact struggle to see a<br>
credible case for allowing any of the remaining flags to be<br>
enabled. They are all cases that Nova should be made todo the right<br>
thing, possibly in relation to API parameters or other deployment<br>
choices.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Fair enough. I figured it would be a necessary safety valve against<br>
operators who value the flexibility of the current configuration<br>
options, but you make a good case.<br>
<br>
I'll drop the extra_flags option and make tunneled a tri-state.<br>
<br>
Thanks, Mark.<br>
<br>
__________________________________________________________________________<br>
<br>
<br>
<o:p></o:p></p>
</blockquote>
<p class="MsoNormal">OpenStack Development Mailing List (not for usage questions)<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal" style="margin-bottom:12.0pt">Unsubscribe:<br>
<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
I'm going to be a jackass and skip this entire thread with a question.<br>
<br>
The live migration CI job we have in the experimental queue is blocked<br>
on this failure right now [1].<br>
<br>
"Live Migration failure: Operation not supported: Selecting disks to<br>
migrate is not implemented for tunnelled migration"<br>
<br>
That's running with ubuntu 16.04 and libvirt 1.2.17.<br>
<br>
I was looking at the deprecated live_migration_flag which tells you to<br>
use the live_migration_tunnelled flag, which defaults to None and is a<br>
no-op, even though the help text on the option says if it's omitted<br>
we'll do our best to figure out if you want to use tunneling based on<br>
the hypervisor's encryption support. That doesn't actually happen by<br>
default [2].<br>
<br>
So what should happen? It seems some of this series was stalled or<br>
incomplete?<br>
<br>
We could change the configuration of our live migration CI job, but I'd<br>
like to see the job work with the defaults that we ship.<br>
<br>
I'm not entirely familiar with the thing that's causing the failure<br>
except I believe we're calling migrateToURI3 with the tunnelled flag<br>
(because it's in the default live_migration_flag option) and a list of<br>
devices [3][4].<br>
<br>
Would it be better to use migrateToURI2 if migrate_disks is in params<br>
but the VIR_MIGRATE_TUNNELLED flag is set?<br>
<br>
[1] <a href="https://bugs.launchpad.net/nova/+bug/1589591" target="_blank">https://bugs.launchpad.net/nova/+bug/1589591</a><br>
[2]<br>
<a href="https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/driver.py#L559" target="_blank">https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/driver.py#L559</a><br>
<br>
[3]<br>
<a href="https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/driver.py#L5740-5751" target="_blank">https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/driver.py#L5740-5751</a><br>
<br>
[4]<br>
<a href="https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/guest.py#L523" target="_blank">https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/guest.py#L523</a><br>
<br>
<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal">Timofey is working on a fix here:<br>
<br>
<a href="https://review.openstack.org/#/c/326111/" target="_blank">https://review.openstack.org/#/c/326111/</a><br>
<br>
I've left some comments inline, but I'm mostly wondering about this now:<br>
<br>
<a href="https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/driver.py#L5360" target="_blank">https://github.com/openstack/nova/blob/f5c9ebd56075f8eb04f9f0e683f85bacdcd68c38/nova/virt/libvirt/driver.py#L5360</a><br>
<br>
The code there says that if there are BDMs and libvirt<1.2.17 and you're doing block migration, it's unsupported and fails. The comment also says that it can't do that in tunneled method either - which is the bug we're hitting.<br>
<br>
But should we add the block_migration_flag check in check_can_live_migrate_source? Or does it work to live migrate in tunneled mode as long as we don't pass 'migrate_disks'?<o:p></o:p></p>
</blockquote>
</div>
</div>
<div>
<p class="MsoNormal">So we got 2 options:<o:p></o:p></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
raise exception in case of tunneled block migration with migrate_disks param <o:p></o:p></li></ul>
<ul type="disc">
<ul type="circle">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level2 lfo1">
as migrate flags are supposed to be nova internals this is bad option<o:p></o:p></li></ul>
</ul>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
get rid of one of these option: <o:p></o:p></li></ul>
<ul type="disc">
<ul type="circle">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level2 lfo1">
don't use VIR_MIGRATE_TUNNELLED - looks most preferable for now<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level2 lfo1">
don't pass migrate disks param (it will force block live migration to fail for instances with attached volumes, and volume-backed instances with config-drive(vfat)) <o:p></o:p></li></ul>
</ul>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"> <o:p></o:p></p>
</blockquote>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><br>
-- <br>
<br>
Thanks,<br>
<br>
Matt Riedemann<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>