Open Stack

> Side question... does either approach touch PCI device management during
> live migration?

Nope. I'd need to do some research to see what, if anything, is needed
at the lower levels (kernel, libvirt) to enable this.

> I ask because the only workloads I've ever seen that pin guest vCPU threads
> to specific host processors -- or make use of huge pages consumed from a
> specific host NUMA node -- have also made use of SR-IOV and/or PCI
> passthrough. [1]
>
> If workloads that use PCI passthrough or SR-IOV VFs cannot be live migrated
> (due to existing complications in the lower-level virt layers) I don't see
> much of a point spending lots of developer resources trying to "fix" this
> situation when in the real world, only a mythical workload that uses CPU
> pinning or huge pages but *doesn't* use PCI passthrough or SR-IOV VFs would
> be helped by it.

It's definitely a paint point for at least some of our customers - I
don't know their use cases exactly, but live migration with CPU
pinning but no other "high performance" features has come up a few
times in our downstream bug tracker. In any case, incremental progress
is better than no progress at all, so if we can improve how NUMA live
migration works, we'll be in a better position to make it work with
PCI devices down the road.

> [Mooney, Sean K]  I would generally agree but with the extention of include dpdk based vswitch like ovs-dpdk or vpp.
> Cpu pinned or hugepage backed guests generally also have some kind of high performance networking solution or use a hardware
> Acclaortor like a gpu to justify the performance assertion that pinning of cores or ram is required.
> Dpdk networking stack would however not require the pci remaping to be addressed though I belive that is planned to be added in stine.

I think Stephen Finucane's NUMA-aware vswitches work depends on mine
to work with live migration - ie, it'll work just fine on its own, but
to live migrate an instance with a NUMA vswitch (I know I'm abusing
language here, apologies) this spec will need to be implemented first.