A question about nova live-migration across AZ
Buddies, I can migration nova instance across available zones with the command below. Suppose instance is in AZ1 and host is in AZ2. nova --os-compute-api-version 2.67 live-migration --force [--block-migrate] <server> [<host>] After migration, the instance’s available zone is changed to AZ2. However, the available zone information in request_specs was not changed. So next time when trying to migrate the instance without specifying a target, the migration target will be selected through nova scheduler, by the availability zone value (AZ1) saved in request_specs. This caused some error. Is this an expected behavior ? Yingji.
It seems to be the expected behavior from the AZ doc. https://docs.openstack.org/nova/latest/admin/availability-zones.html Knowing this, it is dangerous to force a server to another host with evacuate or live migrate if the server is restricted to a zone and is then forced to move to a host in another zone, because that will create an inconsistency in the internal tracking of where that server should live and may require manually updating the database for that server. For example, if a user creates a server in zone A and then the admin force live migrates the server to zone B, and then the user resizes the server, the scheduler will try to move it back to zone A which may or may not work, e.g. if the admin deleted or renamed zone A in the interim. On Mon, Aug 30, 2021 at 9:18 PM Yingji Sun <yingjisun@vmware.com> wrote:
Buddies,
I can migration nova instance across available zones with the command below. Suppose instance is in AZ1 and host is in AZ2.
nova --os-compute-api-version 2.67 live-migration --force [--block-migrate] <server> [<host>]
After migration, the instance’s available zone is changed to AZ2.
However, the available zone information in request_specs was not changed.
So next time when trying to migrate the instance without specifying a target, the migration target will be selected through nova scheduler, by the availability zone value (AZ1) saved in request_specs. This caused some error.
Is this an expected behavior ?
Yingji.
On 30-08-21 22:52:14, Laurent Dumont wrote:
It seems to be the expected behavior from the AZ doc.
https://docs.openstack.org/nova/latest/admin/availability-zones.html
Knowing this, it is dangerous to force a server to another host with evacuate or live migrate if the server is restricted to a zone and is then forced to move to a host in another zone, because that will create an inconsistency in the internal tracking of where that server should live and may require manually updating the database for that server. For example, if a user creates a server in zone A and then the admin force live migrates the server to zone B, and then the user resizes the server, the scheduler will try to move it back to zone A which may or may not work, e.g. if the admin deleted or renamed zone A in the interim.
As listed in that doc the better approach is to shelve and then unshelve into the specific target AZ. This should update the request spec avoiding the post move issues described above. https://docs.openstack.org/api-ref/compute/?expanded=unshelve-restore-shelve...
On Mon, Aug 30, 2021 at 9:18 PM Yingji Sun <yingjisun@vmware.com> wrote:
Buddies,
I can migration nova instance across available zones with the command below. Suppose instance is in AZ1 and host is in AZ2.
nova --os-compute-api-version 2.67 live-migration --force [--block-migrate] <server> [<host>]
After migration, the instance’s available zone is changed to AZ2.
However, the available zone information in request_specs was not changed.
So next time when trying to migrate the instance without specifying a target, the migration target will be selected through nova scheduler, by the availability zone value (AZ1) saved in request_specs. This caused some error.
Is this an expected behavior ?
Yingji.
-- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76
On Mon, 2021-08-30 at 22:52 -0400, Laurent Dumont wrote:
It seems to be the expected behavior from the AZ doc.
https://docs.openstack.org/nova/latest/admin/availability-zones.html
Knowing this, it is dangerous to force a server to another host with evacuate or live migrate if the server is restricted to a zone and is then forced to move to a host in another zone, because that will create an inconsistency in the internal tracking of where that server should live and may require manually updating the database for that server. For example, if a user creates a server in zone A and then the admin force live migrates the server to zone B, and then the user resizes the server, the scheduler will try to move it back to zone A which may or may not work, e.g. if the admin deleted or renamed zone A in the interim.
yes AZ are not really something that shoudl chage over the lifetime of an instance. in nova we cannot assume connectivnty between hosts in different AZ at teh hyperviors level. e.g. we cannot assume that libvirt runing on the host can comunicate directly. as a result we do not support live migrateion or cold migration between AZs. glance however can be assumed to be abvaiable across AZ so we do allow unshleve to unshleve to a different AZ. that is really the only way that we support moving instance between azs.
On Mon, Aug 30, 2021 at 9:18 PM Yingji Sun <yingjisun@vmware.com> wrote:
Buddies,
I can migration nova instance across available zones with the command below. Suppose instance is in AZ1 and host is in AZ2.
nova --os-compute-api-version 2.67 live-migration --force [--block-migrate] <server> [<host>]
After migration, the instance’s available zone is changed to AZ2.
However, the available zone information in request_specs was not changed.
So next time when trying to migrate the instance without specifying a target, the migration target will be selected through nova scheduler, by the availability zone value (AZ1) saved in request_specs. This caused some error.
Is this an expected behavior ?
Yingji.
participants (4)
-
Laurent Dumont
-
Lee Yarwood
-
Sean Mooney
-
Yingji Sun