Open Stack

Hi,

This is the fix for this image listing performance issue of glance v2
api: https://review.openstack.org/#/c/105381/ , welcome to
review/input.

IFY, by this change, the execution time of image listing is about four
times as fast as old logic (16.606s .vs. 4.151s).

zhiyan

On Thu, Jul 3, 2014 at 9:34 PM, Zhi Yan Liu <lzy.dev at gmail.com> wrote:
> Hi,
>
> I have done some tests [0] to try to figure out what's perf problem
> there. As comparing, I did it by two separated cases a) sending
> request by glanceclient cli normally, just like what you did before.
> b) to around the glanceclient cli to send those equal api calls to
> glance server directly. The results as below [1][2].
>
> From the result of my test, it seems we have more overhead in v2
> handling stuff on glanceclient side but service side, IMO.
>
> In the result [1], you can see glanceclient handles v1 requests is
> faster then v2 stuff, 0m6.958s .vs. 0m9.624s (user's real time), this
> is close to your result, but as the comparing, from [2] you can see v2
> api handling on service side is not bad than v1, even ~2 times faster,
> 0m6.189s .vs. 0m3.446s, and based on the api spec requirement, v2
> needs send two more requests for the schema validation of image and
> member resource.
>
> It will be great if you can try this on your env as a
> verification/double-check for my test result, and I'd like to know
> more test details of your result if follow comparison test needed.
>
> Next step I will look into glanceclient.
>
> [0]
> * I did it in my local dev env, it's not devstack but a setup based on
> master sroucecode manually (all services).
> * I prepared 1000 images for the test which under queue status. (so
> there's no data)
> * I didn't test v2 with registry yet.
>
> [1]
> // v1 api with glanceclient cli
>
> $ glance --os-image-api-version 1 image-list | grep 'test-' | wc -l
> 1000
>
> $ time glance --os-image-api-version 1 image-list >/dev/null
> real 0m6.958s
> user 0m0.812s
> sys 0m0.100s
>
> // v2 api with glanceclient cli
>
> $ glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
> 1000
>
> $time glance --os-image-api-version 2 image-list >/dev/null
> real 0m9.624s
> user 0m6.268s
> sys 0m0.080s
>
> [2]
> // v1 api with curl
>
> $ head -5 img1000-curl-v1.sh
> #!/bin/bash
>
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v1/images/detail?sort_key=name\&sort_dir=asc\&limit=20
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v1/images/detail?sort_key=name\&sort_dir=asc\&limit=20\&marker=1696392e-0d04-4a72-a23a-317405c7ef01
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v1/images/detail?sort_key=name\&sort_dir=asc\&limit=20\&marker=6e1c351e-77ae-4fa6-a663-e9eb9100349a
>
> $ ./img1000-curl-v1.sh 2>/dev/null | grep '"images"' | wc -l
> 51
>
> $ time ./img1000-curl-v1.sh >/dev/null 2>&1
> real 0m6.189s
> user 0m0.124s
> sys 0m0.108s
>
> // v2 api with curl
>
> $ head -10 img1000-curl-v2.sh
> #!/bin/bash
>
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/schemas/image
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/schemas/member
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/images?limit=20
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/images?marker=8b8638a4-1419-4c90-9d99-c1d8cfae6c4f\&limit=20
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/images?marker=4bf1dd82-bc88-4c62-90b7-117f0c0efb9e\&limit=20
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/images?marker=4b741769-f371-45b9-ac91-69b6a02d37a5\&limit=20
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/images?marker=35b131f4-5019-48a7-a091-af7fc7df05fc\&limit=20
> curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H
> 'Content-Type: application/json' -H 'User-Agent: python-glanceclient'
> http://10.20.0.1:9292/v2/images?marker=c9b95dfb-def8-4ff6-825d-70ba49555b5d\&limit=20
>
> $ ./img1000-curl-v2.sh 2>/dev/null | grep '"images"' | wc -l
> 51
>
> $ time ./img1000-curl-v2.sh >/dev/null 2>&1
> real 0m3.446s
> user 0m0.224s
> sys 0m0.132s
>
> zhiyan
>
> On Thu, Jun 26, 2014 at 6:39 PM, Sam Morrison <sorrison at gmail.com> wrote:
>> Hi Stuart,
>>
>> Yeah we’re using Icehouse which has this patch included. Glad to hear it’s not just our set up and to confirm that v2 API does seem to be slower.
>> I’m just worried as there is talk of deprecating v2 API but currently it doesn’t seem anywhere near ready to be a suitable replacement for v1.
>>
>> Hopefully as more people start to look at v2 we can iron out some of these issues.
>>
>> Cheers,
>> Sam
>>
>>
>>
>> On 25 Jun 2014, at 10:54 pm, stuart.mclaren at hp.com wrote:
>>
>>> Hi Sam,
>>>
>>> We're just starting to look at v2.
>>>
>>> We did see some really slow listing behaviour, backporting
>>> Ie7458044edbe1ef64871bcf5ef4996ed3cc0f957 helped a lot -- but you'll
>>> probably have that change already if you're using more recent code.
>>>
>>> Even with that it does seem slower in some cases, eg devstack listing 1000 images:
>>>
>>> v1:
>>> $ time ./.tox/py27/bin/glance --os-image-api-version 1 image-list > /dev/null
>>> real  0m6.058s
>>> user  0m1.348s
>>> sys   0m0.172s
>>>
>>> v2, no registry:
>>> $ time ./.tox/py27/bin/glance --os-image-api-version 2 image-list > /dev/null
>>> real  0m15.487s
>>> user  0m9.233s
>>> sys   0m0.120s
>>>
>>> v2, with registry:
>>> $ time glance --os-image-api-version 2 image-list > /tmp/out
>>> real  0m21.904s
>>> user  0m9.217s
>>> sys   0m0.116s
>>>
>>>
>>> -Stuart
>>>
>>> On Wed, 25 Jun 2014, Sam Morrison wrote:
>>>
>>>> We’ve recently enabled glance v2 API on our glance servers. We’ve noticed it is considerably slower than v1 to do things like glance image-list. (~4 seconds compared to ~1)
>>>> Has anyone else experienced this or run any benchmarks comparing the two APIs?
>>>>
>>>> I should note we’re using glance registry for both v1 and v2 requests too.
>>>>
>>>> Also just wondering if people are actually using v2?
>>>> For instance from what I can tell you can’t create an image with v2 via the client yet [1]
>>>>
>>>> Cheers,
>>>> Sam
>>>>
>>>>
>>>> [1] https://bugs.launchpad.net/python-glanceclient/+bug/1302351
>>>> _______________________________________________
>>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to     : openstack at lists.openstack.org
>>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack