[Openstack] Download file from swift extremely slow

Christian Schwede cschwede at redhat.com
Tue Jan 10 16:42:26 UTC 2017


Hello Steve,

On 09.01.2017 23:27, Steve Martinelli wrote:
> Thanks for the hint Clay, we merged
> https://review.openstack.org/#/c/417892/ - expect this issue to be
> fixed in OpenStackClient 3.7.0.

unfortunately the fix didn't help - the issue is still there. I left a
comment in the bugreport, will continue to investigate.

-- Christian


> 
> On Fri, Jan 6, 2017 at 2:03 PM, Clay Gerrard <clay.gerrard at gmail.com
> <mailto:clay.gerrard at gmail.com>> wrote:
> 
>     Is this *really* the default chunk_size?
> 
>     http://docs.python-requests.org/en/master/api/#requests.Response.iter_content
>     <http://docs.python-requests.org/en/master/api/#requests.Response.iter_content>
> 
>     Because, that'd be like a *lot* of read calls for a large object ;)
> 
>     https://github.com/openstack/python-openstackclient/blob/master/openstackclient/api/object_store_v1.py#L383
>     <https://github.com/openstack/python-openstackclient/blob/master/openstackclient/api/object_store_v1.py#L383>
> 
>     Maybe try like, idk, chunk_size=None or 64 * 2 ** 10?
> 
>     -Clay
> 
>     On Fri, Jan 6, 2017 at 9:39 AM, Rick Jones <rick.jones2 at hpe.com
>     <mailto:rick.jones2 at hpe.com>> wrote:
> 
>         On 01/05/2017 11:52 PM, dongjh at ahope.com.cn
>         <mailto:dongjh at ahope.com.cn> wrote:
> 
>             *Also, i can put/get files via dashboard/swift-CLI very
>             quickly.*
>             *So it is strange why 'openstack object save' so slowly...*
> 
> 
>         Well, two additional ways to compare might be to run both the
>         openstack and swift CLI versions under an strace and compare the
>         system calls they are making:
> 
>         strace -v -f -ttt -o <something> cli command ...
> 
>         where -v tells strace to be verbose, the -ttt tells it to
>         timestamp each system call with seconds.microseconds since the
>         epoch, the -f tells it to follow forks and the -o option gives a
>         filename into which the trace should go.
> 
>         The second comparison would be to take a packet trace for each
>         using tcpdump.  So, on the client something like:
> 
>         tcpdump -s 96 -w <filename>.pcap -i <interface> "port <swift
>         port> and host <swift proxy>"
> 
>         The -s says to capture no more than 96 bytes per packet, the -w
>         puts the capture to the named file, the -i selects the network
>         interface on the client, and then the last bit is a filter
>         expression to select only those packets which are swift and
>         to/from the proxy.
> 
>         That file can be post-processed in a number of ways, one of
>         which is:
> 
>         tcpdump -r <filename>.pcap -n -ttt > <filename>.cooked
> 
>         where -r selects the file from which to read captured packets,
>         the -n says to disable looking up hostnames for IP addresses,
>         and the -ttt says to print the time delta for each packet
>         compared to the one before.  I happen to follow a convention of
>         calling the resulting output a ".cooked" file - as in it is a
>         cooked version of a raw (binary) capture.
> 
>         In both cases, you would be looking for large gaps in time -
>         particularly the openstack cli traces.
> 
>         happy benchmarking,
> 
>         rick jones
> 
> 
> 
>         _______________________________________________
>         Mailing list:
>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>         <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
>         Post to     : openstack at lists.openstack.org
>         <mailto:openstack at lists.openstack.org>
>         Unsubscribe :
>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>         <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
> 
> 
> 
>     _______________________________________________
>     Mailing list:
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>     <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
>     Post to     : openstack at lists.openstack.org
>     <mailto:openstack at lists.openstack.org>
>     Unsubscribe :
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>     <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
> 
> 
> 
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 





More information about the Openstack mailing list