Queens steal time [nova]
Hello all, we are comparing the behavior of our queens openstack with kilo. In queens we are observing an increase in the steal time reported in the guest along with the increase of the load averages. All this is happening while the host is not overloaded, and reports 80+ idle time Initially we have suspected that the overcommit might be the reason of the steal, so we have migrated vms, and now there are 42 vCPUs used out of the 48 pCPUs, but in the guest we still observe the steal time. with similar configuration in openstack kilo we see smaller load, and almost no steal time at all. what could be the reason of this steal time when there is no CPU overcommit? Thank you for any ideas. Kind regards, Laszlo
Hi, Technically, I think that you always run the chance of "steal" time if you don't pin CPUs. I'm not sure if Openstack is "smart" enough to allocate CPUs that are not mapped to anyone in sequence (and start to overcommit once it's necessary. You might have 42 out of 48 free CPUs but I don't think that means that Openstack will prevent two VM from getting the same CPU scheduled (without CPU pinning). On Thu, Nov 12, 2020 at 9:09 AM Budai Laszlo <laszlo.budai@gmail.com> wrote:
Hello all,
we are comparing the behavior of our queens openstack with kilo. In queens we are observing an increase in the steal time reported in the guest along with the increase of the load averages. All this is happening while the host is not overloaded, and reports 80+ idle time
Initially we have suspected that the overcommit might be the reason of the steal, so we have migrated vms, and now there are 42 vCPUs used out of the 48 pCPUs, but in the guest we still observe the steal time.
with similar configuration in openstack kilo we see smaller load, and almost no steal time at all.
what could be the reason of this steal time when there is no CPU overcommit?
Thank you for any ideas.
Kind regards, Laszlo
Hi Laurent, Thank you for your answer. I agree with you that without the pinning the steal time can appear anytime. What is strange to me that in openstack Kilo the steal is significantly smaller even when there is some overcommit. So I am wondering where to look for the difference? Kind regards, Laszlo On 11/12/20 5:01 PM, Laurent Dumont wrote:
Hi,
Technically, I think that you always run the chance of "steal" time if you don't pin CPUs. I'm not sure if Openstack is "smart" enough to allocate CPUs that are not mapped to anyone in sequence (and start to overcommit once it's necessary. You might have 42 out of 48 free CPUs but I don't think that means that Openstack will prevent two VM from getting the same CPU scheduled (without CPU pinning).
On Thu, Nov 12, 2020 at 9:09 AM Budai Laszlo <laszlo.budai@gmail.com <mailto:laszlo.budai@gmail.com>> wrote:
Hello all,
we are comparing the behavior of our queens openstack with kilo. In queens we are observing an increase in the steal time reported in the guest along with the increase of the load averages. All this is happening while the host is not overloaded, and reports 80+ idle time
Initially we have suspected that the overcommit might be the reason of the steal, so we have migrated vms, and now there are 42 vCPUs used out of the 48 pCPUs, but in the guest we still observe the steal time.
with similar configuration in openstack kilo we see smaller load, and almost no steal time at all.
what could be the reason of this steal time when there is no CPU overcommit?
Thank you for any ideas.
Kind regards, Laszlo
Is the vCPU placement the same between the two? (if you do a virsh dumpxml of all the VMs on the compute, you should be able to see the which vCPU we're mapped to which VM CPU). The same VMs under a Kilo compute are seeing different Steal % when the compute is upgraded to Queens? I guess that overall % Steal will also be impacted by how busy and noisy the VMs are. My knowledge is far from exhaustive but I would be surprised at a loss of CPU performance between Kilo and Queens. On Thu, Nov 12, 2020 at 12:35 PM Budai Laszlo <laszlo.budai@gmail.com> wrote:
Hi Laurent,
Thank you for your answer. I agree with you that without the pinning the steal time can appear anytime. What is strange to me that in openstack Kilo the steal is significantly smaller even when there is some overcommit. So I am wondering where to look for the difference?
Kind regards, Laszlo
On 11/12/20 5:01 PM, Laurent Dumont wrote:
Hi,
Technically, I think that you always run the chance of "steal" time if you don't pin CPUs. I'm not sure if Openstack is "smart" enough to allocate CPUs that are not mapped to anyone in sequence (and start to overcommit once it's necessary. You might have 42 out of 48 free CPUs but I don't think that means that Openstack will prevent two VM from getting the same CPU scheduled (without CPU pinning).
On Thu, Nov 12, 2020 at 9:09 AM Budai Laszlo <laszlo.budai@gmail.com> wrote:
Hello all,
we are comparing the behavior of our queens openstack with kilo. In queens we are observing an increase in the steal time reported in the guest along with the increase of the load averages. All this is happening while the host is not overloaded, and reports 80+ idle time
Initially we have suspected that the overcommit might be the reason of the steal, so we have migrated vms, and now there are 42 vCPUs used out of the 48 pCPUs, but in the guest we still observe the steal time.
with similar configuration in openstack kilo we see smaller load, and almost no steal time at all.
what could be the reason of this steal time when there is no CPU overcommit?
Thank you for any ideas.
Kind regards, Laszlo
participants (2)
-
Budai Laszlo
-
Laurent Dumont