Open Stack

On 06/22/2017 01:47 AM, Henning Schild wrote:
> Am Wed, 21 Jun 2017 11:40:14 -0600
> schrieb Chris Friesen <chris.friesen at windriver.com>:
>
>> On 06/21/2017 10:46 AM, Henning Schild wrote:

>>> As we know from our setup, and as Luiz confirmed - it is _not_
>>> "critical to separate emulator threads for different KVM instances".
>>> They have to be separated from the vcpu-cores but not from each
>>> other. At least not on the "cpuset" basis, maybe "blkio" and
>>> cgroups like that.
>>
>> I'm reluctant to say conclusively that we don't need to separate
>> emulator threads since I don't think we've considered all the cases.
>> For example, what happens if one or more of the instances are being
>> live-migrated?  The migration thread for those instances will be very
>> busy scanning for dirty pages, which could delay the emulator threads
>> for other instances and also cause significant cross-NUMA traffic
>> unless we ensure at least one core per NUMA-node.
>
> Realtime instances can not be live-migrated. We are talking about
> threads that can not even be moved between two cores on one numa-node
> without missing a deadline. But your point is good because it could
> mean that such an emulator_set - if defined - should not be used for all
> VMs.

I'd suggest that realtime instances cannot be live-migrated *while meeting 
realtime commitments*.  There may be reasons to live-migrate realtime instances 
that aren't currently providing service.

>> Also, I don't think we've determined how much CPU time is needed for
>> the emulator threads.  If we have ~60 CPUs available for instances
>> split across two NUMA nodes, can we safely run the emulator threads
>> of 30 instances all together on a single CPU?  If not, how much
>> "emulator overcommit" is allowable?
>
> That depends on how much IO your VMs are issuing and can not be
> answered in general. All VMs can cause high load with IO/emulation,
> rt-VMs are probably less likely to do so.

I think the result of this is that in addition to "rt_emulator_pin_set" you'd 
probably want a config option for "rt_emulator_overcommit_ratio" or something 
similar.

Chris