Open Stack

On 2013-06-21 08:58:38, Giorgio Franceschi wrote:
> On 21-06-13 07:24, Kashyap Chamarthy wrote:
>> On 06/20/2013 10:18 PM, Russell Bryant wrote:
>>> On 06/20/2013 10:36 AM, Giorgio Franceschi wrote:
>>>> Hello, I created a blueprint for the implementation of:
>>>>
>>>> A tool for pinning automatically each running virtual CPU to a physical
>>>> one in the most efficient way, balancing load across sockets/cores and
>>>> maximizing cache sharing/minimizing cache misses. Ideally able to be run
>>>> on-demand, as a periodic job, or be triggered by events on the host (vm
>>>> spawn/destroy).
>>>>
>>>> Find it at https://blueprints.launchpad.net/nova/+spec/auto-cpu-pinning
>>>>
>>>> Any inputappreciated!
>>> I'm actually surprised to see a new tool for this kind of thing.
>>>
>>> Have you seen numad?
>>
>> And a related post by Dan Berrange (but on lower layers -- libvirt) which explains how to
>> do vcpu pinning and control NUMA affinity --
>> http://berrange.com/posts/2010/02/12/controlling-guest-cpu-numa-affinity-in-libvirt-with-qemu-kvm-xen/
>>
>>
>>
>>
> Yes Kashyap, thanks for the link, I had read that article while
> researching the problem. It addresses the same issues, but the solution
> is based on install-time, static configuration. We want something
> requiring as little config as possible and able to allocate VMs at
> runtime. Basically, what the author suggests in section "Fine tuning CPU
> affinity at runtime", but automated for large-scale, hands-off environments.
>
> Russell, thanks for your suggestion, I did not know of numad. It looks
> interesting, but the way I understand it, it is a system-wide
> NUMA-binding daemon without any configuration options or fine-tuning
> capabilities. We want something that only deals with relevant kvm
> processes, not all processes on the system, and also we would like to
> make it configurable so that VM can advertise their "pinnability",
> because sometimes you might not want all running domains on a host to be
> treated the same. This is planned for a future release. Would numad be
> suited to this task, in your opinion? I suppose one could use it as a
> querying tool, with the -w switch, and apply its suggestions
> selectively. Then, it would basically replace the strategy-making part
> of pinhead. I will investigate this shortly.
>
> Thanks everyone for your help, any further input much appreciated!

We've used vm-balancer.py in the past, and it seems to work quite well
(I don't see numad packaged for ubuntu, nor numad support built into
libvirt for ubuntu)

   http://code.google.com/p/vm-balancer-numa/downloads

The following blog post explains it, and the comments contain
different versions of a shell script that controls it:

   http://blog.opennebula.org/?p=3193

Regards,
Marcus.