On Tue, 2024-06-04 at 11:50 +0100, smooney@redhat.com wrote:
On Tue, 2024-06-04 at 12:47 +0530, Gk Gk wrote:
Ok. We are using ssh as the scheme. So, using this new variable live_migration _scheme, how do we specify the ssh options like key location, no strict key checking etc ? I don't see it documented anywhere ? Let me reiterate if I am not clear. I want to know how we can use the new variable live_migration _scheme, since it is mentioned that the live_migration_uri is deprecated ? So, if we use live_migration _scheme, then live_migration_uri is no longer needed. Then as in our case, how do we specify the ssh scheme and its uri options ? Hope I am clear ...
so we recently as in this cycle undeprecated live_migration_uri you shoudl just continue to use that.
there was an effort a few years ago to remove it but none of the installer every used the alternate options and our concenous at the last ptg is that having seperate config options instead of one that is effectivly a direct passthough to libvirt is adding more technially debt and maintance burden then the ux improvement it promised.
so my recommendataion is avoid live_migration_scheme and keep using live_migration_uri as you do today.
oh the patch for that undeprection has not been done yet. ill see if i can follow up on that but this was the ptg discussion on this topic
https://etherpad.opendev.org/p/r.3d37f484b24bb0415983f345582508f7#L557 ``` (tkajinam) Gap between legacy deprecated live migration options vs new live migration options
Courtesy ping:
https://bugs.launchpad.net/nova/+bug/1671288
https://bugs.launchpad.net/nova/+bug/1843883
The live_migration_uri option was deprecated in favor of live_migration_scheme and live_migration_inbound_addr
However the new options lacks support for customizing the following elements, used by ssh
User, Port, Path, Queries
(kashyap) How many of these four options truly need to be customized?
User, Port and Queries based on the occurences I could find
Devstack: qemu+ssh://$STACK_USER@%s/system
https://github.com/openstack/devstack/blob/fca44cc375657da3d1a20f16c5c04574c...
OSA: qemu+ssh://nova@%s/system?no_verify=1&keyfile={{ nova_system_home_folder }}/.ssh/id_rsa
User and queries: https://github.com/openstack/openstack-ansible-os_nova/blob/501cf14342dd5519...
Charm: qemu+ssh://%s/system
https://github.com/openstack/charm-nova-compute/blob/f3bf6be831880eb9a939674...
Kolla: None because it does not support ssh transport
TripleO: User, port and queries
https://github.com/openstack-archive/puppet-tripleo/blob/a10816eb81d02aa8104...
OKO: User and queries: qemu+ssh://nova@%s/system?keyfile=/var/lib/nova/.ssh/ssh-privatekey
https://github.com/openstack-k8s-operators/nova-operator/blob/a2d67e46f9f26b...
Path can be theoretically customized because nova allows customizing the whole connection url string ([libvirt] connection_uri). However requirement to customize the path element is quite rare
The deprecation options have been kept for ~7 years. Can we move forward the situation
(sean) so i don tpersonally agreee with the deprecation and im not a fan of the replacements in general
A potential fix was proposed 6+ years ago but has been stuck
Should we move conf: Add four new '[libvirt] live_migration_*' options https://review.opendev.org/c/openstack/nova/+/456571 forward ?
(kashyap) Title of the above patch (please add it too) "conf: Add four new '[libvirt] live_migration_*' options" Done
I recall that patch, it is adding four config options. It increases the live-migration testing matrix. Without careful testing, this can subtly break or cause regressions.
Given the limited resources we have, I'm not confident we should go ahead with it, unless there's someone dedicated to shepherd this.
alternative approaches even if we remove the deprecated options ?
Create customized ssh_config in the nova user's home directory
This is currently required if the customization should also affects ssh/rsync in remotefs operations (eg. file transfer during cold migration)
Doesn't allow different behavior fo live and cold migration (eg reject live migration over ssh but supports ssh during cold migration) though I don't know if hits very popular
nova pacakge in RDO provides nova-migration package which provides the customized sshd_config for this purpose
https://github.com/rdo-packages/nova-distgit/blob/60a6c9392d57eade49a0cc4598...
https://github.com/rdo-packages/nova-distgit/blob/60a6c9392d57eade49a0cc4598...
Leave the existing options
Question
Can we check feasibility of the alternative approach with customized ssh_config to determine if we need nova options or not ?
tkajinam will propose patches for devstack, OSA and Charm
(kashyap) Yeah, the alternative approach does look like much less maintenance churn
Notes (half way through)
(Sylvain) Why do we need a specific URI scheme
(Sean) Originally, we only had URI ... open to "un-deprecate" the URI scheme
(kashyap) I'm also open to it; it reduces a lot of maintenance burden (including testing extensively many migration paths, etc)
Using just the URI approach is also "much less work for us" (not to be under-estimated) and rely on libvirt docs
(Takashi) Worried about the existing options being around too long
AGREE :
(takashi) Deprecate this cycle live_migration_scheme
(SvenKieske): what would be the timeline for complete removal here, because we need to change this in kolla
(kashyap) Usually a release of deprecation warning, and removal in the release after. So in 2 cycles it "should" be gone?
(SvenKieske): I honestly find this a little fast when a decision basically get's reversed (this was supposed to be the future instead of now being the past).
(tkajinam): I share similar concern given some deployment tools are using scheme already and asking them to revert to the old option is a kind of confusing.
(kashyap) Also, frankly, there's nothing stopping us to delay it a release _further_, if there are good reasons. +1+1
(SvenKieske): Other deprecated stuff is in for like 7 releases and I don't see the point in being that fast, I just need some time to recode this in kolla (and test, and migrate existing users), please :-)
(tkajinam): I may eventually lean towards to not deprecating the scheme option but we can discuss this further in the patch for recording purpose
(SvenKieske): I'm actually not against deprecation, I see the points for it. I'm only asking for some time to implement this in a sane way and migrate users; rewriting our tests probably is most of the work as we currently don't use tempest ( various people working on tests now ) :)
(tkajinam) The most "fast" timeine allowed would be deprecation in D and removal in F based on SLURP policy
(takashi) Undeprecate live_migration_uri ```
On Tue, Jun 4, 2024 at 12:27 PM Bernd Bausch berndbausch@gmail.com wrote:
The values for live_migration_scheme are the schemes accepted by libvirt. Thus my recommendation to read the libvirt doc, but sadly, the libvirt page on migration, https://libvirt.org/migration.html, doesn't say much about the migration URIs. Perhaps you need to read the source code for more clarity.
A better place might be Redhat; such as https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/htm... .
On Tue, Jun 4, 2024 at 7:41 AM Gk Gk ygk.kmr@gmail.com wrote:
Appreciate your response. I would like to know what are all the valid values for live_migration_schema variable ?
On Tue, 4 Jun 2024, 09:17 Bernd Bausch, berndbausch@gmail.com wrote:
Check the section on live_migration_uri in https://docs.openstack.org/nova/latest/configuration/config.html. For more info, read the live migration documentation for libvirt.
On Mon, Jun 3, 2024, 11:27 PM Gk Gk ygk.kmr@gmail.com wrote:
Can you point me to a link on how to use this ?
Thanks
On Mon, 3 Jun 2024, 17:05 Bernd Bausch, berndbausch@gmail.com wrote:
> Documentation says "It is not recommended that you change this unless > you are very sure that hypervisor supports a particular scheme". In other > words, if you don't know possible schemes, leave the config option alone. > > Having said that, a few examples of schemes can be found in the > documentation for live_migration_uri. > > On Mon, Jun 3, 2024, 11:06 AM Gk Gk ygk.kmr@gmail.com wrote: > > > Hi, > > > > I see this in the latest nova configuration: > > > > This option is deprecated for removal since 15.0.0. Its value may be > > silently ignored in the future. > > Reason: > > > > live_migration_uri is deprecated for removal in favor of two other > > options that allow to change live migration scheme and target URI: > > live_migration_scheme and live_migration_inbound_addr respectively. > > > > > > So, if I want to use live_migration_scheme, what are its values ? > > > > > > If I use the live_migration_scheme option, can I remove the live_migration_uri > > option ? > > > > > > Thanks > > > > > > > >