Nova live migration

3 Jun 2024

    (tkajinam) Gap between legacy deprecated live migration options vs new
live migration options

    Courtesy ping:

    https://bugs.launchpad.net/nova/+bug/1671288

    https://bugs.launchpad.net/nova/+bug/1843883

    The live_migration_uri option was deprecated in favor of
live_migration_scheme and live_migration_inbound_addr

    However the new options lacks support for customizing the following
elements, used by ssh

    User, Port, Path, Queries

    (kashyap) How many of these four options truly need to be customized?

    User, Port and Queries based on the occurences I could find

    Devstack: qemu+ssh://$STACK_USER@%s/system


https://github.com/openstack/devstack/blob/fca44cc375657da3d1a20f16c5c04574c6234376/lib/nova_plugins/hypervisor-libvirt#L48

    OSA: qemu+ssh://nova@%s/system?no_verify=1&keyfile={{
nova_system_home_folder }}/.ssh/id_rsa

    User and queries:

https://github.com/openstack/openstack-ansible-os_nova/blob/501cf14342dd5519f064bca2ec999d204c00bc67/templates/nova.conf.j2#L275

    Charm: qemu+ssh://%s/system


https://github.com/openstack/charm-nova-compute/blob/f3bf6be831880eb9a939674252ece9a5cc964847/hooks/nova_compute_context.py#L245

    Kolla: None because it does not support ssh transport

    TripleO: User, port and queries


https://github.com/openstack-archive/puppet-tripleo/blob/a10816eb81d02aa8104990523b0844fa428a0597/manifests/profile/base/nova/migration/client.pp#L72-L74

    OKO: User and queries: qemu+ssh://nova@
%s/system?keyfile=/var/lib/nova/.ssh/ssh-privatekey


https://github.com/openstack-k8s-operators/nova-operator/blob/a2d67e46f9f26bfe54cdcb5670866b5f7fdc3492/templates/nova.conf#L197

    Path can be theoretically customized because nova allows customizing
the whole connection url string ([libvirt]
connection_uri). However requirement to customize the path element is
quite rare

    The deprecation options have been kept for ~7 years. Can we move
forward the situation

    (sean) so i don tpersonally agreee with the deprecation and im not a
fan of the replacements in general

    A potential fix was proposed 6+ years ago but has been stuck

    Should we move conf: Add four new '[libvirt] live_migration_*' options
https://review.opendev.org/c/openstack/nova/+/456571 forward ?

    (kashyap) Title of the above patch (please add it too) "conf: Add four
new '[libvirt] live_migration_*' options"
Done

    I recall that patch, it is adding four config options. It increases
the live-migration testing matrix.  Without
careful testing, this can subtly break or cause regressions.

    Given the limited resources we have, I'm not confident we should go
ahead with it, unless there's someone dedicated
to shepherd this.

    alternative approaches even if we remove the deprecated options ?

    Create customized ssh_config in the nova user's home directory

    This is currently required if the customization should also affects
ssh/rsync in remotefs operations (eg. file
transfer during cold migration)

    Doesn't allow different behavior fo live and cold migration (eg reject
live migration over ssh but supports ssh
during cold migration) though I don't know if hits very popular

    nova pacakge in RDO provides nova-migration package which provides the
customized sshd_config for this purpose


https://github.com/rdo-packages/nova-distgit/blob/60a6c9392d57eade49a0cc4598f0fd34dd6658c3/openstack-nova.spec#L475


https://github.com/rdo-packages/nova-distgit/blob/60a6c9392d57eade49a0cc4598f0fd34dd6658c3/nova-ssh-config

    Leave the existing options

    Question

    Can we check feasibility of the alternative approach with customized
ssh_config to determine if we need nova options
or not ?

    tkajinam will propose patches for devstack, OSA and Charm

    (kashyap) Yeah, the alternative approach does look like much less
maintenance churn

    Notes (half way through)

    (Sylvain) Why do we need a specific URI scheme

    (Sean) Originally, we only had URI ... open to "un-deprecate" the URI
scheme

    (kashyap) I'm also open to it; it reduces a lot of maintenance burden
(including testing extensively many migration
paths, etc)

    Using just the URI approach is also "much less work for us" (not to be
under-estimated) and rely on libvirt docs

    (Takashi) Worried about the existing options being around too long


    AGREE :

    (takashi) Deprecate this cycle live_migration_scheme

    (SvenKieske): what would be the timeline for complete removal here,
because we need to change this in kolla

    (kashyap) Usually a release of deprecation warning, and removal in the
release after.  So in 2 cycles it "should" be
gone?

    (SvenKieske): I honestly find this a little fast when a decision
basically get's reversed (this was supposed to be
the future instead of now being the past).

    (tkajinam): I share similar concern given some deployment tools are
using scheme already and asking them to revert
to the old option is a kind of confusing.

    (kashyap) Also, frankly, there's nothing stopping us to delay it a
release _further_, if there are good reasons.
+1+1

    (SvenKieske): Other deprecated stuff is in for like 7 releases and I
don't see the point in being that fast, I just
need some time to recode this in kolla (and test, and migrate existing
users), please :-)

    (tkajinam): I may eventually lean towards to not deprecating the
scheme option but we can discuss this further in
the patch for recording purpose

    (SvenKieske): I'm actually not against deprecation, I see the points
for it. I'm only asking for some time to
implement this in a sane way and migrate users; rewriting our tests
probably is most of the work as we currently don't
use tempest ( various people working on tests now ) :)

    (tkajinam) The most "fast" timeine allowed would be deprecation in D
and removal in F based on SLURP policy

    (takashi) Undeprecate live_migration_uri

Gk Gk

Bernd Bausch

Gk Gk

Bernd Bausch

Gk Gk

Bernd Bausch

Gk Gk

smooney＠redhat.com

smooney＠redhat.com

Gk Gk

tags (0)

participants (3)