[Openstack] Download file from swift extremely slow

Steve Martinelli s.martinelli at gmail.com
Mon Jan 9 22:27:03 UTC 2017


Thanks for the hint Clay, we merged https://review.openstack.org/#/c/417892/
- expect this issue to be fixed in OpenStackClient 3.7.0.

On Fri, Jan 6, 2017 at 2:03 PM, Clay Gerrard <clay.gerrard at gmail.com> wrote:

> Is this *really* the default chunk_size?
>
> http://docs.python-requests.org/en/master/api/#requests.
> Response.iter_content
>
> Because, that'd be like a *lot* of read calls for a large object ;)
>
> https://github.com/openstack/python-openstackclient/blob/
> master/openstackclient/api/object_store_v1.py#L383
>
> Maybe try like, idk, chunk_size=None or 64 * 2 ** 10?
>
> -Clay
>
> On Fri, Jan 6, 2017 at 9:39 AM, Rick Jones <rick.jones2 at hpe.com> wrote:
>
>> On 01/05/2017 11:52 PM, dongjh at ahope.com.cn wrote:
>>
>>> *Also, i can put/get files via dashboard/swift-CLI very quickly.*
>>> *So it is strange why 'openstack object save' so slowly...*
>>>
>>
>> Well, two additional ways to compare might be to run both the openstack
>> and swift CLI versions under an strace and compare the system calls they
>> are making:
>>
>> strace -v -f -ttt -o <something> cli command ...
>>
>> where -v tells strace to be verbose, the -ttt tells it to timestamp each
>> system call with seconds.microseconds since the epoch, the -f tells it to
>> follow forks and the -o option gives a filename into which the trace should
>> go.
>>
>> The second comparison would be to take a packet trace for each using
>> tcpdump.  So, on the client something like:
>>
>> tcpdump -s 96 -w <filename>.pcap -i <interface> "port <swift port> and
>> host <swift proxy>"
>>
>> The -s says to capture no more than 96 bytes per packet, the -w puts the
>> capture to the named file, the -i selects the network interface on the
>> client, and then the last bit is a filter expression to select only those
>> packets which are swift and to/from the proxy.
>>
>> That file can be post-processed in a number of ways, one of which is:
>>
>> tcpdump -r <filename>.pcap -n -ttt > <filename>.cooked
>>
>> where -r selects the file from which to read captured packets, the -n
>> says to disable looking up hostnames for IP addresses, and the -ttt says to
>> print the time delta for each packet compared to the one before.  I happen
>> to follow a convention of calling the resulting output a ".cooked" file -
>> as in it is a cooked version of a raw (binary) capture.
>>
>> In both cases, you would be looking for large gaps in time - particularly
>> the openstack cli traces.
>>
>> happy benchmarking,
>>
>> rick jones
>>
>>
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>>
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
> openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
> openstack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170109/4ddea15e/attachment.html>


More information about the Openstack mailing list