Thank you, I was reading the Nova docs that only cite NFS for live migration (https://docs.openstack.org/nova/latest/admin/configuring-migrations.html#sha...), do you know if there is documentation about live migration with Ceph? We're planning on deploying it, in case switching from local storage to Ceph (instead of NFS) would be the more sensible approach Regards Francesco Di Nucci On 17/06/24 15:54, smooney@redhat.com wrote:
Hello,
AFAIK live migration requires to have /var/lib/nova/instances shared between the compute nodes (eg. through a NFS export)
On Mon, 2024-06-17 at 14:02 +0200, Francesco Di Nucci wrote: that is not the case and has never been required to my knowladge.
perhaps in the very early days i.e. pre 2013 when i started working on openstack but we have supported live migration with local storage without nfs since at least the hevana release and likely before that.
nova instructs libvirt to export the vm storage via the nbd (network block device) server so that it can be transferred to the destination hosts.
- If this has not been configured initially and each compute node has its own folder, is it possible to merge them somehow without disruptive effects? not safely with running workloads without a lot of work
the only way to move form local non shared stoarge to local shared storage is to copy the disk which is generally unsafe to do manually.
the only way i can think of to do this with running workloads is via live migration in effect you need an empty compute node that is on nfs, then you need to live migrate to live migrate all insatnce form an non nfs host to the nfs host. once the non-nfs host is drained of workloads you can mount the nfs share on /var/lib/nova/instance and use it as the next "empty" host. then you have to repeat that proceture for every host until all your isntances are on nfs.
that will likely reduce the perfroamce of your guest and may cause addtional operation issues as nfs is know to have locking/consitency issues espically v3. we recommend a minium nfs version of 4.2
while /var/lib/nova/instance on nfs is technially supproted upstream our best practices recommendation is to avoid that configuration and either use local stoage, boot form cinder volume or ceph if you want a shared network storage solution.
Regards
Francesco Di Nucci