Sean; I have some spare resources, I'll try to setup a recreation, once my current problems are resolved. Thank you, Dominic L. Hilsbos, MBA Vice President – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com -----Original Message----- From: Sean Mooney [mailto:smooney@redhat.com] Sent: Wednesday, July 28, 2021 10:35 AM To: Dominic Hilsbos; openstack-discuss@lists.openstack.org Subject: Re: [nova] Early migrate error - pain point? On Wed, 2021-07-28 at 16:50 +0000, DHilsbos@performair.com wrote:
Sean;
This is Victoria. I was trying to migrate from a CentOS 8 (libvirt 6.6.0) to a Ubuntu 20.04 (libvirt 6.0.0).
I may not have any logs from it, as I've wiped out the host OS, and installed 20.10, with a side-track into 21.04. ok so in that case live migration should be blocked by the check i linked in the code since you are going form libvirt 6.6.0 to 6.0.0 its possibel that that puts the vm in the error state but it really should just fail the migration and leave it in active.
i tested goign form centos 8 to centos 7 recnently and confirm the isntance remained in active and the migration failed due to the hypervior veriosn. its possibel that since centos and ubunutu sotre libvirt and qemu in different locaiton it fails in a slightly different way that we dont handel properly if you have time to reproduce it that woudl be great if not i might see if i can replicate your case but im not sure when ill have time to do that. cold migration should work in this case by the way if its blocking your move to ubuntu. i dont belvie we are as stict in that case in terms of not migrating to and older hypervior version. with live migration it can break in hard to fix ways so its blocked. you could possibley get a newer version of libvirt form the ubuntu cloud arcive too althogh i have not checked.
Thank you,
Dominic L. Hilsbos, MBA Vice President – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com
-----Original Message----- From: Sean Mooney [mailto:smooney@redhat.com] Sent: Wednesday, July 28, 2021 9:32 AM To: Dominic Hilsbos; openstack-discuss@lists.openstack.org Subject: Re: [nova] Early migrate error - pain point?
All;
Yesterday I ran into an issue where a live migration errored before starting the migration itself (libvirt version incompatibility). in general that shoudl not result in the vm entering error.
On Wed, 2021-07-28 at 15:49 +0000, DHilsbos@performair.com wrote: the migration should enter error but the vm should remain in active state provided this failed in prelive migration.
what release did this happen in.
we shoudl raise a DestinationHypervisorTooOld excption https://github.com/openstack/nova/blob/master/nova/exception.py#L557-L559 if the target hypervior is too old https://github.com/openstack/nova/blob/97e1a6bece29e383f55bb969c69983153df9f...
if source_version > destination_version: raise exception.DestinationHypervisorTooOld()
that check was added 8 years ago https://github.com/openstack/nova/commit/2d7beddc0bb7830c2a8bf893b9221c0de56...
if you are seeing this today i would file a bug for it with the full traceback and opentack versions and hypervior verions
This left the instance in an error state, even though it continued to run normally on the original host.
Has anyone else run into this? Could this be a candidate for the current pain point hunt?
it has happened in the past due to bugs but im not aware of this happening any more on master. im not sure if this shoudl be added into the pain point since if its happing it just a normal bug but it proably does not hurt either. nova however should already be handeling it.
failures in prelive migration should always leave teh vm in the active sttate. failure in post livemigration should generall leave the vm in the error since we cant rollback. failure migrate when we call libvirt sometime are recoverable and sometimes not so it will depend.
Thank you,
Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com