Hi community,

We would like to report an issue we are encountering with nested virtualization on Windows 11 instances running on OpenStack. On our OpenStack platform, a Windows 11 VM becomes unable to boot after enabling Virtual Machine Platform/Hyper-V inside the VM and performing a hard reboot (VM does not show a BSOD but fails to boot into the OS and gets stuck in a boot loop in Tianocore logo). This issue does not occur on Windows 10 under the same conditions.

System:

• Compute node CPU: Intel Xeon Gold 6538Y+ (Emerald Rapids) with nested virtualization is enabled.
• OpenStack 2024.2
• libvirt 8.0.0
• QEMU API 8.0.0
• QEMU hypervisor 6.2.0

Initially, our Nova CPU configuration was:

• cpu_mode=host-model

• cpu_model_extra_flags=+vmx,-hypervisor,-xsaves

According to virsh dumpxml, host-model maps to Icelake-Server.

We tested several cpu_mode=custom configurations and observed the following:

• cpu_models=Icelake-Server-noTSX → error (boot loop)
•  cpu_models=Icelake-Server → error (boot loop)
• cpu_models=Broadwell-noTSX-IBRS → works
• cpu_models=Cascadelake-Server-noTSX → works

The working models correspond to the Preferred CPU models recommended by QEMU:

https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html#preferred-cpu-models-for-intel-x86-hosts

We would like to ask:

1. Has anyone encountered this issue with Windows 11 + nested virtualization on Icelake/Emerald Rapids hosts?
2. Are there known root causes explaining why newer CPU models fail while preferred (older) models work?
3. Is there a recommended fix other than switching to a preferred older CPU model?

Best regards,

Hai Pham