[openstack-dev] [nova][libvirt] VIR_MIGRATE_NON_SHARED_INC works more like full copy in block migration ?
lgy181 at foxmail.com
Sun Jan 10 10:07:33 UTC 2016
Do you test the difference between within and without VIR_MIGRATE_NON_SHARED_INC ?
When I add VIR_MIGRATE_NON_SHARED_INC in block_migration_flags in nova, nova block migration behaves more like a full copy of disk.
The disk in destination nodes is large(10GB+) and the process of live migration is slow.
However, when I remove VIR_MIGRATE_NON_SHARED_INC in block_migration_flags in nova, nova block migration behaves more like a incremental copy of disk.
The disk in destination nodes is small(10MB-) and the process of live migration is very fast.
So I was confused about what VIR_MIGRATE_NON_SHARED_INC really means.
Can someone give me some hints?
Luo Gangyiluogangyi at cmss.chinamobile.com
------------------ 原始邮件 ------------------
发件人: "Mark McLoughlin";<markmc at redhat.com>;
发送时间: 2016年1月5日(星期二) 凌晨5:12
收件人: "openstack-dev"<openstack-dev at lists.openstack.org>;
主题: [openstack-dev] [nova][libvirt] Deprecating the live_migration_flagand block_migration_flag config options
commit 8ecf93e got me thinking - the live_migration_flag config
option unnecessarily allows operators choose arbitrary behavior of the
migrateToURI() libvirt call, to the extent that we allow the operator
to configure a behavior that can result in data loss.
I see that danpb recently said something similar:
"Honestly, I wish we'd just kill off 'live_migration_flag' and
'block_migration_flag' as config options. We really should not be
exposing low level libvirt API flags as admin tunable settings.
Nova should really be in charge of picking the correct set of flags
for the current libvirt version, and the operation it needs to
perform. We might need to add other more sensible config options in
their place [..]"
I've just proposed a series of patches, which boils down to the
1) Modify the approach taken in commit 8ecf93e so that instead of
just warning about unsafe use of NON_SHARED_INC, we fix up the
config option to a safe value.
2) Hard-code the P2P flag for live and block migrations as
appropriate for the libvirt driver being used.
For the qemu driver, We should always use VIR_MIGRATE_PEER2PEER
both live and block migrations. Without this option, you get:
Live Migration failure: Requested operation is not valid: direct migration is not supported by the connection driver
OTOH, the Xen driver does not support P2P, and only supports
"unmanaged direct connection".
3) Require the use of the UNDEFINE_SOURCE flag, and the non-use of
the PERSIST_DEST flag.
Nova itself persists the domain configuration on the destination
host, but it assumes the libvirt migration call removes it from
source host. So it makes no sense to allow operators configure
4) Add a new config option for tunneled versus native:
live_migration_tunneled = true
This enables the use of the VIR_MIGRATE_TUNNELLED flag. We have
historically defaulted to tunneled mode because it requires the
least configuration and is currently the only way to have a
secure migration channel.
danpb's quote above continues with:
"perhaps a "live_migration_secure_channel" to indicate that
migration must use encryption, which would imply use of
So we need to discuss whether the config option should express the
choice of tunneled vs native, or whether it should express another
choice which implies tunneled vs native.
5) Add a new config option for additional migration flags:
live_migration_extra_flags = VIR_MIGRATE_COMPRESSED
This allows operators to continue to experiment with libvirt behaviors
in safe ways without each use case having to be accounted for.
We would disallow setting the following flags via this option:
which would allow the following currently available flags to be set:
6) Deprecate the existing live_migration_flag and block_migration_flag
config options. Operators would be expected to migrate to using the
live_migration_tunneled or live_migration_extra_flags config options.
During the deprecation period we would invite feedback as to whether
additional config options are needed to cover unanticipated use cases.
Thanks in advance for any feedback.
I'm going to guess that one piece of feedback will be that some subset
of this needs a blueprint (and maybe a spec), and that the blueprint
freeze was a month ago, so that subset needs to be punted until after
Mitaka? I'd love to be wrong about that, though :)
 - https://review.openstack.org/228853
 - Data loss can occur when you have disk images on shared storage and
you specify the VIR_MIGRATE_NON_SHARED_INC or VIR_MIGRATE_NON_SHARED_DISK because during the block migration the disk is copied back over itself
while it is in use from another node.
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev