Open Stack

On 2/9/17, 10:59 PM, "Ihar Hrachyshka" <ihrachys at redhat.com> wrote:

>Hi all,
>
>I noticed lately a number of job failures in neutron gate that all
>result in job timeouts. I describe
>gate-tempest-dsvm-neutron-dvr-ubuntu-xenial job below, though I see
>timeouts happening in other jobs too.
>
>The failure mode is all operations, ./stack.sh and each tempest test
>take significantly more time (like 50% to 150% more, which results in
>job timeout triggered). An example of what I mean can be found in [1].
>
>A good run usually takes ~20 minutes to stack up devstack; then ~40
>minutes to pass full suite; a bad run usually takes ~30 minutes for
>./stack.sh; and then 1:20h+ until it is killed due to timeout.
>
>It affects different clouds (we see rax, internap, infracloud-vanilla,
>ovh jobs affected; we haven't seen osic though). It can't be e.g. slow
>pypi or apt mirrors because then we would see slowdown in ./stack.sh
>phase only.
>
>We can't be sure that CPUs are the same, and devstack does not seem to
>dump /proc/cpuinfo anywhere (in the end, it's all virtual, so not sure

I don’t think that logging this information could be useful mainly because this depends on enabling *host-passthrough*[3] in nova-compute configuration of Public cloud providers

>if it would help anyway). Neither we have a way to learn whether
>slowliness could be a result of adherence to RFC1149. ;)
>
>We discussed the matter in neutron channel [2] though couldn't figure
>out the culprit, or where to go next. At this point we assume it's not
>neutron's fault, and we hope others (infra?) may have suggestions on
>where to look.
>
>[1] http://logs.openstack.org/95/429095/2/check/gate-tempest-dsvm-neutron-dvr-ubuntu-xenial/35aa22f/console.html#_2017-02-09_04_47_12_874550
>[2] http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2017-02-10.log.html#t2017-02-10T04:06:01
[3] http://docs.openstack.org/newton/config-reference/compute/hypervisor-kvm.html 

>
>Thanks,
>Ihar
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev