[openstack-dev] [glance] multi threads with swift backend

Erno Kuvaja ekuvaja at redhat.com
Thu Sep 28 16:30:03 UTC 2017


On Thu, Sep 28, 2017 at 4:27 PM, Arnaud MORIN <arnaud.morin at gmail.com> wrote:
> Hey all,
> So I finally tested your pull requests, it does not work.
> 1 - For uploads, swiftclient is not using threads when source is given by
> glance:
> https://github.com/openstack/python-swiftclient/blob/master/swiftclient/service.py#L1847
>
> 2 - For downloads, when requesting the file from swift, it is recomposing
> the chunks into one big file.
>
>
> So patch is not so easy.
>
> IMHO, for uploads, we should try to uploads chunks using multithreads.
> Sounds doable.
> For downloads, I need to dig a little bit more in glance store code to be
> sure, but maybe we can try to download the chunks separately and recompose
> them locally before sending it to the requester (compute / cli).
>
> Cheers,
>

So I'm still trying to understand (without success) why do we want to
do this at all?

- jokke

>
> On 6 September 2017 at 21:19, Arnaud MORIN <arnaud.morin at gmail.com> wrote:
>>
>> Hey,
>> I would love to see that reviving!
>>
>> Cheers,
>> Arnaud
>>
>> On 6 September 2017 at 21:00, Mikhail Fedosin <mfedosin at gmail.com> wrote:
>>>
>>> Hey! As you said it's not possible now.
>>>
>>> I implemented the support several years ago, bit unfortunately no one
>>> wanted to review it: https://review.openstack.org/#/c/218993
>>> If you want, we can revive it.
>>>
>>> Best,
>>> Mike
>>>
>>> On Wed, Sep 6, 2017 at 9:05 PM, Clay Gerrard <clay.gerrard at gmail.com>
>>> wrote:
>>>>
>>>> I'm pretty sure that would only be possible with a code change in glance
>>>> to move the consumption of the swiftclient abstraction up a layer from the
>>>> client/connection objects to swiftclient's service objects [1].  I'm not
>>>> sure if that'd be something that would make a lot of sense to the Image
>>>> Service team.
>>>>
>>>> -Clay
>>>>
>>>> 1. https://docs.openstack.org/python-swiftclient/latest/service-api.html
>>>>
>>>> On Wed, Sep 6, 2017 at 9:02 AM, Arnaud MORIN <arnaud.morin at gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Is there any chance that glance can use the multiprocessing from
>>>>> swiftclient library (equivalent of xxx-threads options from cli)?
>>>>> If yes, how to enable it?
>>>>> I did not find anything useful in the glance configuration options.
>>>>> And looking at glance_store code make me think that it's not
>>>>> possible...
>>>>> Am I wrong?
>>>>>
>>>>> Regards,
>>>>> Arnaud
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list