Open Stack

Hi All,

I'm encountering an issue with live migrating between 2 Compute nodes with different CPU models. Host A has a Westmere CPU and host B has a Broadwell.

I can migrate an instance from the Westmere host A to the Broadwell host B,  but when I attempt to live migrate a VM from the Broadwell host B to the older Westmere host I get the following error;

Live Migration failure: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: fma, x2apic, movbe, tsc-deadline, xsave, osxsave, avx, f16c, rdrand, fsgsbase, tsc_adjust, bmi1, hle, avx2, smep, bmi2, erms, invpcid, rtm, rdseed, adx, smap, xsaveopt, abm, 3dnowprefetch; try using 'Broadwell-noTSX' CPU model

How can I avoid this error, without rebooting any VMs?


Host B with the Broadwell CPU has the libvirt CPU masking enabled in its nova.conf;
...
[libvirt]
cpu_mode = custom
cpu_model = Westmere

Host A with the Westmere CPU, never had the cpu_mode set in nova.conf but as its KVM it should default to the following;
[libvirt]
cpu_mode = host-model


The "host-model" cpu_mode appears to enable more CPU features on the guest the then what is enabled by cpu_model = Westmere. For instance, when a VM is started on Host A (Westmere CPU & cpu_mode = host-model) the VM get the following CPU configuration;

<cpu mode='host-model'>
    <model fallback='allow'>Westmere</model>
    <vendor>Intel</vendor>
    <topology sockets='1' cores='1' threads='1'/>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='pclmuldq'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='ds_cpl'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='pcid'/>
    <feature policy='require' name='dca'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='rdtscp'/>
  </cpu>

But when the instance is booted on Host B (Broadwell CPU & cpu_model = Westmere)

<cpu mode='custom' match='exact'>
    <model fallback='allow'>Westmere</model>
    <topology sockets='1' cores='1' threads='1'/>
</cpu>

Interestingly once a host is booted on Host B and has the above CPU config, it can be migrated back to Host A without any error.

Host A Capabilities;
<cpu>
      <arch>x86_64</arch>
      <model>Westmere</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='4' threads='2'/>
      <feature name='vme'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='pclmuldq'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='pcid'/>
      <feature name='dca'/>
      <feature name='arat'/>
      <feature name='pdpe1gb'/>
      <feature name='rdtscp'/>
      <feature name='invtsc'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
</cpu>

Host B Capabilities;
<cpu>
      <arch>x86_64</arch>
      <model>Broadwell</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='12' threads='2'/>
      <feature name='vme'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='f16c'/>
      <feature name='rdrand'/>
      <feature name='arat'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='xsaveopt'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='pdpe1gb'/>
      <feature name='abm'/>
      <feature name='invtsc'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
</cpu>

Oisin O'Malley
Systems Engineer
Iocane Pty Ltd
763 South Road
Black Forest SA 5035

Office:+61 (8) 8413 1010
Fax:+61 (8) 8231 2050
Email:oisin.omalley at iocane.com.au
Web:www.iocane.com.au

Better for business