<div dir="auto">My objective is to be able to download and upload from glance/computes to swift in a faster way.<div dir="auto">I was thinking that if glance could parallelizes the connections to swift for a single image (with chunks), it would be faster.</div><div dir="auto">Am I wrong ?</div><div dir="auto">Is there any other way I am not thinking of?</div><div dir="auto"><br></div><div dir="auto">Arnaud.</div></div><div class="gmail_extra"><br><div class="gmail_quote">Le 28 sept. 2017 6:30 PM, "Erno Kuvaja" <<a href="mailto:ekuvaja@redhat.com">ekuvaja@redhat.com</a>> a écrit :<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Sep 28, 2017 at 4:27 PM, Arnaud MORIN <<a href="mailto:arnaud.morin@gmail.com">arnaud.morin@gmail.com</a>> wrote:<br>
> Hey all,<br>
> So I finally tested your pull requests, it does not work.<br>
> 1 - For uploads, swiftclient is not using threads when source is given by<br>
> glance:<br>
> <a href="https://github.com/openstack/python-swiftclient/blob/master/swiftclient/service.py#L1847" rel="noreferrer" target="_blank">https://github.com/openstack/<wbr>python-swiftclient/blob/<wbr>master/swiftclient/service.py#<wbr>L1847</a><br>
><br>
> 2 - For downloads, when requesting the file from swift, it is recomposing<br>
> the chunks into one big file.<br>
><br>
><br>
> So patch is not so easy.<br>
><br>
> IMHO, for uploads, we should try to uploads chunks using multithreads.<br>
> Sounds doable.<br>
> For downloads, I need to dig a little bit more in glance store code to be<br>
> sure, but maybe we can try to download the chunks separately and recompose<br>
> them locally before sending it to the requester (compute / cli).<br>
><br>
> Cheers,<br>
><br>
<br>
So I'm still trying to understand (without success) why do we want to<br>
do this at all?<br>
<br>
- jokke<br>
<br>
><br>
> On 6 September 2017 at 21:19, Arnaud MORIN <<a href="mailto:arnaud.morin@gmail.com">arnaud.morin@gmail.com</a>> wrote:<br>
>><br>
>> Hey,<br>
>> I would love to see that reviving!<br>
>><br>
>> Cheers,<br>
>> Arnaud<br>
>><br>
>> On 6 September 2017 at 21:00, Mikhail Fedosin <<a href="mailto:mfedosin@gmail.com">mfedosin@gmail.com</a>> wrote:<br>
>>><br>
>>> Hey! As you said it's not possible now.<br>
>>><br>
>>> I implemented the support several years ago, bit unfortunately no one<br>
>>> wanted to review it: <a href="https://review.openstack.org/#/c/218993" rel="noreferrer" target="_blank">https://review.openstack.org/#<wbr>/c/218993</a><br>
>>> If you want, we can revive it.<br>
>>><br>
>>> Best,<br>
>>> Mike<br>
>>><br>
>>> On Wed, Sep 6, 2017 at 9:05 PM, Clay Gerrard <<a href="mailto:clay.gerrard@gmail.com">clay.gerrard@gmail.com</a>><br>
>>> wrote:<br>
>>>><br>
>>>> I'm pretty sure that would only be possible with a code change in glance<br>
>>>> to move the consumption of the swiftclient abstraction up a layer from the<br>
>>>> client/connection objects to swiftclient's service objects [1].  I'm not<br>
>>>> sure if that'd be something that would make a lot of sense to the Image<br>
>>>> Service team.<br>
>>>><br>
>>>> -Clay<br>
>>>><br>
>>>> 1. <a href="https://docs.openstack.org/python-swiftclient/latest/service-api.html" rel="noreferrer" target="_blank">https://docs.openstack.org/<wbr>python-swiftclient/latest/<wbr>service-api.html</a><br>
>>>><br>
>>>> On Wed, Sep 6, 2017 at 9:02 AM, Arnaud MORIN <<a href="mailto:arnaud.morin@gmail.com">arnaud.morin@gmail.com</a>><br>
>>>> wrote:<br>
>>>>><br>
>>>>> Hi all,<br>
>>>>><br>
>>>>> Is there any chance that glance can use the multiprocessing from<br>
>>>>> swiftclient library (equivalent of xxx-threads options from cli)?<br>
>>>>> If yes, how to enable it?<br>
>>>>> I did not find anything useful in the glance configuration options.<br>
>>>>> And looking at glance_store code make me think that it's not<br>
>>>>> possible...<br>
>>>>> Am I wrong?<br>
>>>>><br>
>>>>> Regards,<br>
>>>>> Arnaud<br>
>>>>><br>
>>>>><br>
>>>>> ______________________________<wbr>______________________________<wbr>______________<br>
>>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>>> Unsubscribe:<br>
>>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
>>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
>>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> ______________________________<wbr>______________________________<wbr>______________<br>
>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>> Unsubscribe:<br>
>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
>>>><br>
>>><br>
>>><br>
>>><br>
>>> ______________________________<wbr>______________________________<wbr>______________<br>
>>> OpenStack Development Mailing List (not for usage questions)<br>
>>> Unsubscribe:<br>
>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
>>><br>
>><br>
><br>
><br>
> ______________________________<wbr>______________________________<wbr>______________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
><br>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
</blockquote></div></div>