Ah, yes. config-drive. That's a good point.

When config-drive is used then nova creates a local disk file which stores instance metadata
and attach the disk to an instance. The cloud-init service in the instance detects the disk and
read metadata from the disk rather than metadata api. In case an instance with config drive
is migrated then the drive file should be also transferred to the destination compute node,
and that's why block migration is required.

AFAIK Octavia is heavily dependent on instance metadata. There is an option to use usual
user data instead of config driver but the option is deprecated because of known problems
caused by sizelimit in user data[1], and now config drive is required for all amphora instances.

So as a result, live migration of amphora instances need block migration, even if volume
boot is used, unless you have shared NFS storage in your compute nodes.

[1] https://review.opendev.org/c/openstack/octavia/+/855441

On 11/29/23 21:01, Nguyễn Hữu Khôi wrote:

Hello.

I mean I use volume boot for Octavia Instance. I check and no local disk is used by any instance on my compute node.

However, I don't know why I can do live migration with block migration.

I read from redhat:

If the instance uses config-drive, block migration is required to live-migrate the instance

This command to check:

openstack server show <instance name/id> -c config_drive

My Octavia instance returns "True".

Nguyen Huu Khoi

On Wed, Nov 29, 2023 at 6:12 PM Takashi Kajinami <kajinamit@oss.nttdata.com> wrote:

On 11/29/23 17:34, Nguyễn Hữu Khôi wrote:

Hi,

I am using share storage for Octavia instance.

Do you mind elaborating the "share strage" here ?

Are you using volume backed amphora or shared storage in your compute nodes ?

In case you have shared storage among some compute nodes then I'd suggest you check
source-destination compute nodes and see whether these two nodes are actually using
the same shared storage for instance data (usually located at /var/lib/nova/instances )

If you are using volume backed amphora then I may check the libvirt xml of the amphora
instance and also the nova flavor used, to find out any local disk device attached
(eg. If the flavor has swap then it may attach an additional local device)

We can mv Octavia instances by failover but if there is a hundred of lb then it will be a big problem. I am planning to use specific computers for Octavia only and use Active-Standby Topologies.

Nguyen Huu Khoi

On Wed, Nov 29, 2023 at 3:16 PM Takashi Kajinami <kajinamit@oss.nttdata.com> wrote:

This is not an issue with Octavia but is more about usage of Nova.
By default, amphora instances are deployed using image boot, and use
ephemeral disks stored in compute nodes. Unless you have a shared storage
in your compute nodes to store ephemeral disks(nfs, ceph and so on),
you have to enable block-migration to live migrate the instance.
(See --block-migration option in openstack CLI)

However, AFAIK the general recommendation is to try amphora failover
(which internally recreates the amphora instance) instead of live-migrating it,
as is described in the guide[1]. You might want to try that method instead.

[1] https://docs.openstack.org/octavia/latest/admin/guides/operator-maintenance.html#evacuating-a-specific-amphora- from-a-host

On 11/29/23 14:17, Nguyễn Hữu Khôi wrote:

Hello everyone.

I deploy amphora instances using shared storage but when I do live migrate it to another host, I encounter this error.

Error: Failed to live migrate instance to host "AUTO_SCHEDULE". Details

compute12 is not on shared storage: Shared storage live-migration requires either shared storage or boot-from-volume with no local disks

Woud we have a solution for this?

Thank you.

Nguyen Huu Khoi