[openstack-dev] [nova] [neutron] [rally] Neutron or nova degradation?

Carl Baldwin carl at ecbaldwin.net
Tue Sep 8 17:06:34 UTC 2015


This sounds like a good candidate for a "git bisect" operation [1]
since we already have a pretty tight window where things changed.

Carl

[1] http://git-scm.com/docs/git-bisect

On Thu, Sep 3, 2015 at 7:07 AM, Assaf Muller <amuller at redhat.com> wrote:
>
>
> On Thu, Sep 3, 2015 at 8:43 AM, Andrey Pavlov <andrey.mp at gmail.com> wrote:
>>
>> Hello,
>>
>> We have rally job with fake virt driver. And we run it periodically.
>> This job runs 200 servers and measures 'show' operations.
>>
>> On 18.08 it was run well[1]. But on 21.08 it was failed by timeout[2].
>> I tried to understand what happens.
>> I tried to check this job with 20 servers only[3]. It passed but I see
>> that
>> operations with neutron take more time now (list subnets, list network
>> interfaces).
>> and as result start and show instances take more time also.
>>
>> Maybe anyone knows what happens?
>
>
> Looking at the merged Neutron patches between the 18th and 21st, there's a
> lot of
> candidates, including QoS and work around quotas.
>
> I think the best way to find out would be to run a profiler against Neutron
> from the 18th,
> and Neutron from the 21st while running the Rally tests, and finding out if
> the major
> bottlenecks moved. Last time I profiled Neutron I used GeventProfiler:
> https://pypi.python.org/pypi/GreenletProfiler
>
> Ironically I was having issued with the profiler that comes with Eventlet.
>
>>
>>
>>
>> [1]
>> http://logs.openstack.org/13/211613/6/experimental/ec2-api-rally-dsvm-fakevirt/fac263e/
>> [2]
>> http://logs.openstack.org/74/213074/7/experimental/ec2-api-rally-dsvm-fakevirt/91d0675/
>> [3]
>> http://logs.openstack.org/46/219846/1/experimental/ec2-api-rally-dsvm-fakevirt/dad98f0/
>>
>> --
>> Kind regards,
>> Andrey Pavlov.
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list