[ubuntu] Known that qemu 4.2 is incompatible with GCP instances?
I run tests with GCP instances as the OpenStack hypervisors. Obviously it's better if those can use libvirt_type kvm, i.e. nested virtualization, and this has been possible prior to my current Ussuri upgrade work. With Ussuri on Ubuntu, IIUC, we get qemu 4.2 from cloud-archive:ussuri, but qemu 4.2 has a bug that was fixed by this commit prior to 5.0.0: https://github.com/qemu/qemu/commit/4a910e1f6ab4155ec8b24c49b2585cc486916985 target/i386: do not set unsupported VMX secondary execution controls Commit 048c951 ("target/i386: work around KVM_GET_MSRS bug for secondary execution controls") added a workaround for KVM pre-dating commit 6defc591846d ("KVM: nVMX: include conditional controls in /dev/kvm KVM_GET_MSRS") which wasn't setting certain available controls. The workaround uses generic CPUID feature bits to set missing VMX controls. [...] The bug manifests on a GCP instance with nested virtualization enabled [1], because such a GCP instance doesn't support MSR features. The OpenStack-level symptom is that a VM can't be scheduled onto that GCP instance. Is this a well-known problem? For CentOS/RHEL, [2] looks similar and maybe fixed, but it's difficult to be sure. Best wishes, Neil [1] https://cloud.google.com/compute/docs/instances/enable-nested-virtualization... [2] https://bugzilla.redhat.com/show_bug.cgi?id=1722360
Hi Neil On Tue, Jun 9, 2020 at 11:59 AM Neil Jerram <neil@tigera.io> wrote:
I run tests with GCP instances as the OpenStack hypervisors. Obviously it's better if those can use libvirt_type kvm, i.e. nested virtualization, and this has been possible prior to my current Ussuri upgrade work.
With Ussuri on Ubuntu, IIUC, we get qemu 4.2 from cloud-archive:ussuri, but qemu 4.2 has a bug that was fixed by this commit prior to 5.0.0: https://github.com/qemu/qemu/commit/4a910e1f6ab4155ec8b24c49b2585cc486916985
target/i386: do not set unsupported VMX secondary execution controls
Commit 048c951 ("target/i386: work around KVM_GET_MSRS bug for secondary execution controls") added a workaround for KVM pre-dating commit 6defc591846d ("KVM: nVMX: include conditional controls in /dev/kvm KVM_GET_MSRS") which wasn't setting certain available controls. The workaround uses generic CPUID feature bits to set missing VMX controls. [...]
The bug manifests on a GCP instance with nested virtualization enabled [1], because such a GCP instance doesn't support MSR features. The OpenStack-level symptom is that a VM can't be scheduled onto that GCP instance.
Is this a well-known problem? For CentOS/RHEL, [2] looks similar and maybe fixed, but it's difficult to be sure.
I could not an existing bug in Ubuntu describing these symptoms - any chance you can report a bug here: https://bugs.launchpad.net/ubuntu/+source/qemu/+filebug Cheers James
On Tue, Jun 9, 2020 at 1:08 PM James Page <james.page@canonical.com> wrote:
Hi Neil
On Tue, Jun 9, 2020 at 11:59 AM Neil Jerram <neil@tigera.io> wrote:
I run tests with GCP instances as the OpenStack hypervisors. Obviously it's better if those can use libvirt_type kvm, i.e. nested virtualization, and this has been possible prior to my current Ussuri upgrade work.
With Ussuri on Ubuntu, IIUC, we get qemu 4.2 from cloud-archive:ussuri, but qemu 4.2 has a bug that was fixed by this commit prior to 5.0.0: https://github.com/qemu/qemu/commit/4a910e1f6ab4155ec8b24c49b2585cc486916985
target/i386: do not set unsupported VMX secondary execution controls
Commit 048c951 ("target/i386: work around KVM_GET_MSRS bug for secondary execution controls") added a workaround for KVM pre-dating commit 6defc591846d ("KVM: nVMX: include conditional controls in /dev/kvm KVM_GET_MSRS") which wasn't setting certain available controls. The workaround uses generic CPUID feature bits to set missing VMX controls. [...]
The bug manifests on a GCP instance with nested virtualization enabled [1], because such a GCP instance doesn't support MSR features. The OpenStack-level symptom is that a VM can't be scheduled onto that GCP instance.
Is this a well-known problem? For CentOS/RHEL, [2] looks similar and maybe fixed, but it's difficult to be sure.
I could not an existing bug in Ubuntu describing these symptoms - any chance you can report a bug here:
https://bugs.launchpad.net/ubuntu/+source/qemu/+filebug
Cheers
On Tue, Jun 9, 2020 at 2:04 PM James Page <james.page@canonical.com> wrote:
On Tue, Jun 9, 2020 at 1:08 PM James Page <james.page@canonical.com> wrote:
Hi Neil
On Tue, Jun 9, 2020 at 11:59 AM Neil Jerram <neil@tigera.io> wrote:
I run tests with GCP instances as the OpenStack hypervisors. Obviously it's better if those can use libvirt_type kvm, i.e. nested virtualization, and this has been possible prior to my current Ussuri upgrade work.
With Ussuri on Ubuntu, IIUC, we get qemu 4.2 from cloud-archive:ussuri, but qemu 4.2 has a bug that was fixed by this commit prior to 5.0.0: https://github.com/qemu/qemu/commit/4a910e1f6ab4155ec8b24c49b2585cc486916985
target/i386: do not set unsupported VMX secondary execution controls
Commit 048c951 ("target/i386: work around KVM_GET_MSRS bug for secondary execution controls") added a workaround for KVM pre-dating commit 6defc591846d ("KVM: nVMX: include conditional controls in /dev/kvm KVM_GET_MSRS") which wasn't setting certain available controls. The workaround uses generic CPUID feature bits to set missing VMX controls. [...]
The bug manifests on a GCP instance with nested virtualization enabled [1], because such a GCP instance doesn't support MSR features. The OpenStack-level symptom is that a VM can't be scheduled onto that GCP instance.
Is this a well-known problem? For CentOS/RHEL, [2] looks similar and maybe fixed, but it's difficult to be sure.
I could not an existing bug in Ubuntu describing these symptoms - any chance you can report a bug here:
https://bugs.launchpad.net/ubuntu/+source/qemu/+filebug
Cheers
Many thanks James, that's exactly it. I've just commented on the bug to ask if it would be easy to build packages for Bionic (as well as for Focal). Best wishes, Neil
participants (2)
-
James Page
-
Neil Jerram