Open Stack

Ok, so here is what I have done.  Using "virsh capabilities" on the old
CPUs I got:
    <cpu>
      <arch>x86_64</arch>
      <model>Opteron_G3</model>
      <vendor>AMD</vendor>
      <topology sockets='1' cores='4' threads='1'/>
      <feature name='nodeid_msr'/>
      <feature name='wdt'/>
      <feature name='skinit'/>
      <feature name='ibs'/>
      <feature name='osvw'/>
      <feature name='3dnowprefetch'/>
      <feature name='cr8legacy'/>
      <feature name='extapic'/>
      <feature name='cmp_legacy'/>
      <feature name='3dnow'/>
      <feature name='3dnowext'/>
      <feature name='pdpe1gb'/>
      <feature name='fxsr_opt'/>
      <feature name='mmxext'/>
      <feature name='ht'/>
      <feature name='vme'/>
    </cpu>

Using "virsh capabilities" on the new CPUs I got:
    <cpu>
      <arch>x86_64</arch>
      <model>Opteron_G5</model>
      <vendor>AMD</vendor>
      <topology sockets='1' cores='64' threads='1'/>
      <feature name='bmi1'/>
      <feature name='perfctr_nb'/>
      <feature name='perfctr_core'/>
      <feature name='topoext'/>
      <feature name='nodeid_msr'/>
      <feature name='tce'/>
      <feature name='lwp'/>
      <feature name='wdt'/>
      <feature name='skinit'/>
      <feature name='ibs'/>
      <feature name='osvw'/>
      <feature name='cr8legacy'/>
      <feature name='extapic'/>
      <feature name='cmp_legacy'/>
      <feature name='fxsr_opt'/>
      <feature name='mmxext'/>
      <feature name='osxsave'/>
      <feature name='monitor'/>
      <feature name='ht'/>
      <feature name='vme'/>
    </cpu>

So I came up with this CPU as a common feature set:
    <cpu>
      <arch>x86_64</arch>
      <model>Opteron_G3</model>
      <vendor>AMD</vendor>
      <topology sockets='1' cores='4' threads='1'/>
      <feature name='nodeid_msr'/>
      <feature name='wdt'/>
      <feature name='skinit'/>
      <feature name='ibs'/>
      <feature name='osvw'/>
      <feature name='3dnowprefetch'/>
      <feature name='cr8legacy'/>
      <feature name='extapic'/>
      <feature name='cmp_legacy'/>
      <feature name='pdpe1gb'/>
      <feature name='fxsr_opt'/>
      <feature name='mmxext'/>
      <feature name='ht'/>
      <feature name='vme'/>
    </cpu>


using "virsh cpu-compare" I got the following:

root at old:~# virsh cpu-compare model
Host CPU is a superset of CPU described in model

root at new:~# virsh cpu-compare model
Host CPU is a superset of CPU described in model

Excellent!

So I convert this over to the cpu_map.xml format:
    <model name='ACG_COMMON'>
      <model name='Opteron_G3'/>
      <feature name='nodeid_msr'/>
      <feature name='wdt'/>
      <feature name='skinit'/>
      <feature name='ibs'/>
      <feature name='osvw'/>
      <feature name='3dnowprefetch'/>
      <feature name='cr8legacy'/>
      <feature name='extapic'/>
      <feature name='cmp_legacy'/>
      <feature name='pdpe1gb'/>
      <feature name='fxsr_opt'/>
      <feature name='mmxext'/>
      <feature name='ht'/>
      <feature name='vme'/>
   </model>


And launch a VM on the new node.  Here are the CPU flags:
-cpu
Opteron_G3,+nodeid_msr,+wdt,+skinit,+ibs,+osvw,+3dnowprefetch,+cr8legacy,+extapic,+cmp_legacy,+pdpe1gb,+fxsr_opt,+mmxext,+ht,+vme
 (from NEW processor)

Here are the cpu flags from a vm running on the old processors:
-cpu
Opteron_G3,+nodeid_msr,+wdt,+skinit,+ibs,+osvw,+3dnowprefetch,+cr8legacy,+extapic,+cmp_legacy,+3dnow,+3dnowext,+pdpe1gb,+fxsr_opt,+mmxext,+ht,+vme
(from the OLD processor)

Again this looks great right?  The new processor does not appear to support
3dnow or 3dnowext.

So when I try to migrate from NEW -> OLD I get this error on the OLD node:
http://paste.openstack.org/show/53203/

Any ideas what I should try next?  It seems like this should have worked.
 Do I need to set the CPU model in the OLD nova.conf to the same custom cpu
as in the nova.conf on the NEW node?

Thanks,
Sam



On Wed, Nov 13, 2013 at 4:47 PM, Samuel Winchenbach <swinchen at gmail.com>wrote:

> Thanks Steve,
>
> I actually found which extensions the two CPUs had in common and created
> my own in cpu_map.xml (http://pastie.org/pastes/8478554/text).  The
> extensions commented out are ones that I could not find referenced in the
> cpu_map...  I may try un-commenting them to see what happens :P
>
> It is strange... I start up a VM on the new compute node and it is running
> fine, when I try to do a live migration I get: "2013-11-13 16:43:18.063
> ERROR nova.virt.libvirt.driver [req-7591a8a0-17ad-4539-b969-6b4a7d156963
> 43b2e3d7bed64bacab2f69b0cb7ce387 417de5251c804c45baabd0fa2873b43f] CPU
> doesn't have compatibility." which is odd because I only specified
> extensions the two CPUs have in common.  Hrmmmm.
>
> Sam
>
>
> On Wed, Nov 13, 2013 at 3:47 PM, Steve Gordon <sgordon at redhat.com> wrote:
>
>> ----- Original Message -----
>> > From: "Samuel Winchenbach" <swinchen at gmail.com>
>> > To: openstack-operators at lists.openstack.org
>> > Sent: Wednesday, November 13, 2013 2:57:32 PM
>> > Subject: [Openstack-operators] New compute node, new libvirt_cpu_mode
>> >
>> > Hi All.
>> >
>> > I was wondering what the implications of changing libvirt_cpu_mode from
>> the
>> > default (host-passthrough) to none might be?   I setup a new compute
>> node
>> > that has a slightly newer processor model than the other nodes and live
>> > migration is NOT happy about that.   I have done some reading and
>> > discovered setting libvirt_cpu_mode=none should help solve the problem.
>> >  What can I expect if I do this?   I am assuming a slight performance
>> hit
>> > as the VM will not have access to certain processor extensions.  Here
>> is a
>> > list of /proce/cpuinfo for two vms (one with =none and the other
>> default).
>> >
>> > http://paste.openstack.org/show/52519/
>> >
>> > It is clear that the default mode enables more processor extensions.
>> >
>> > Is there any other way around this problem?
>> >
>> > Thanks,
>> > Sam
>>
>> If you can identify the Libvirt CPU model that best maps to the lowest
>> common denominator for your cluster (that is includes the CPU flags that
>> are common between the CPUs in your compute nodes) then you can set
>> libvirt_cpu_mode to "custom" and set libvirt_cpu_model to the specific
>> model [1]. This should be Opteron_G3 for the CPU exposed when you used
>> "host-passthrough" in your output.
>>
>> This will give you better performance than the QEMU baseline CPU and more
>> importantly working live migration but obviously means some of the flags
>> exposed by the newest CPU in the cluster aren't made available to guests.
>>
>> -Steve
>>
>> [1] https://wiki.openstack.org/wiki/LibvirtXMLCPUModel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20131115/2525bc51/attachment.html>