Hi community,

We would like to report an issue we are encountering with nested virtualization on Windows 11 instances running on OpenStack. On our OpenStack platform, a Windows 11 VM becomes unable to boot after enabling Virtual Machine Platform/Hyper-V inside the VM and performing a hard reboot (VM does not show a BSOD but fails to boot into the OS and gets stuck in a boot loop in Tianocore logo). This issue does not occur on Windows 10 under the same conditions.

System:

Compute node CPU: Intel Xeon Gold 6538Y+ (Emerald Rapids) with nested virtualization is enabled.
OpenStack 2024.2
libvirt 8.0.0
QEMU API 8.0.0
QEMU hypervisor 6.2.0

Initially, our Nova CPU configuration was:

cpu_mode=host-model

cpu_model_extra_flags=+vmx,-hypervisor,-xsaves

According to virsh dumpxml, host-model maps to Icelake-Server.

We tested several cpu_mode=custom configurations and observed the following:

cpu_models=Icelake-Server-noTSX → error (boot loop)
cpu_models=Icelake-Server → error (boot loop)
cpu_models=Broadwell-noTSX-IBRS → works
cpu_models=Cascadelake-Server-noTSX → works

The working models correspond to the Preferred CPU models recommended by QEMU:

https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html#preferred-cpu-models-for-intel-x86-hosts

We would like to ask:

Has anyone encountered this issue with Windows 11 + nested virtualization on Icelake/Emerald Rapids hosts?
Are there known root causes explaining why newer CPU models fail while preferred (older) models work?
Is there a recommended fix other than switching to a preferred older CPU model?

Best regards,

Hai Pham