Hi Neil

On Tue, Jun 9, 2020 at 11:59 AM Neil Jerram <neil@tigera.io> wrote:

I run tests with GCP instances as the OpenStack hypervisors.  Obviously it's better if those can use libvirt_type kvm, i.e. nested virtualization, and this has been possible prior to my current Ussuri upgrade work.

With Ussuri on Ubuntu, IIUC, we get qemu 4.2 from cloud-archive:ussuri, but qemu 4.2 has a bug that was fixed by this commit prior to 5.0.0: https://github.com/qemu/qemu/commit/4a910e1f6ab4155ec8b24c49b2585cc486916985

    target/i386: do not set unsupported VMX secondary execution controls

    Commit 048c951 ("target/i386: work around KVM_GET_MSRS bug for
    secondary execution controls") added a workaround for KVM pre-dating
    commit 6defc591846d ("KVM: nVMX: include conditional controls in /dev/kvm
    KVM_GET_MSRS") which wasn't setting certain available controls. The
    workaround uses generic CPUID feature bits to set missing VMX controls. [...]

The bug manifests on a GCP instance with nested virtualization enabled [1], because such a GCP instance doesn't support MSR features.  The OpenStack-level symptom is that a VM can't be scheduled onto that GCP instance.

Is this a well-known problem?  For CentOS/RHEL, [2] looks similar and maybe fixed, but it's difficult to be sure.

I could not an existing bug in Ubuntu describing these symptoms - any chance you can report a bug here:

  https://bugs.launchpad.net/ubuntu/+source/qemu/+filebug

Cheers