[openstack-dev] [swift] Support for unicode filename in tempurl middleware

Antonio Calanducci antonio.calanducci at ct.infn.it
Fri Jul 1 07:23:53 UTC 2016


Hi,

at the end I figured the problem out. It was the very same bug of the
python-swiftclient, missing encoding for utf-8.
I encoded the hmac_body with utf8 (but not URI encoding) and the tempURL is
now valid!

Thank you for the support!
Best
Antonio


On Wed, Jun 29, 2016 at 9:40 PM, Antonio Calanducci <
antonio.calanducci at ct.infn.it> wrote:

> Hi Tim,
>
> thank you. I have modified my local python swift client according the diff
> on your patch and now I can generate tempURL on python2.7 too.
>
> I have noticed a bizzarre thing now. Trying to use CURL to download the
> generated tempURL, using a utf8 filename ("Monè.txt"), the download works
> fine if I copy and paste the filename from "swift list" output, but if I
> type the filename with my keyboard, it doesn't work anymore. Same "è"
> character, not sure why. Maybe the key I type on the keyboard has a
> different encoding than the one from the "swift list" output?
>
> I have also a second question, related to filename that contains spaces. I
> should not need to URL encode (with %20) the path to generate the tempURL,
> but I should encode when doing the actual upload, correct?
>
> Actually the code that generates the tempURL is a Node.js script written
> in JavaScript. Probably I am not doing correctly the encodings (as far as I
> know I should not encode anything in Node.js because of the native
> support). Do you have any JS sample code for tempURL generation or any
> hints I should follow to properly generate the tempURL?
>
> thank you in advance
> Best
> Antonio
>
>
>
>
>
>
>
>
> On Wed, Jun 29, 2016 at 8:27 PM, Tim Burke <tim at swiftstack.com> wrote:
>
>> Hi Antonio,
>>
>> That sounds like a bug! Looks like we decode the temporal key coming in
>> from the terminal, but don't properly encode it when getting the HMAC. This
>> generates the UnicodeEncodeError on python 2.7, and I've submitted
>> https://review.openstack.org/#/c/335615/ to address this.
>>
>> I'm not sure about what's going on with the temporal generated on python
>> 3, though. FWIW, I'll often forget whether it's supposed to be
>> X-Container-Meta-Temp-Url-Key or X-Container-Meta-Tempurl-Key. (For
>> reference, use the first one.) Maybe that's the issue?
>>
>> Tim
>>
>> > On Jun 29, 2016, at 10:06 AM, Dr Antonio Calanducci <
>> antonio.calanducci at ct.infn.it> wrote:
>> >
>> > Hi,
>> >
>> > I hope this is the right mailing list to ask this question. In case
>> it's not, please direct me to the right one and apologise.
>> >
>> > I have some files stored in a container the use german character (and
>> in general non ascii character).
>> > Using the swift REST APIs and python-swiftclient (3.0.0)
>> download/upload command I have no problem to handle them.
>> >
>> > But when I try to generate tempurl for some of them with
>> python-swiftclient, I got errors if I use python 2.7. By the way, switching
>> to python3 I can generate properly the tempurl, but the URL is actually not
>> working generating a 401 error.
>> >
>> > Looking at the github repo it seems there is utf8 support for the
>> tempurl key, but I am not sure if you support unicode for path too. I am
>> currently using swift 2.2.0 on the server.
>> >
>> > I read in the docs that I should not encode the path while generating
>> the tempurl, but only to actually retrieve from the generate url.
>> >
>> > Could you confirm or not that filename with unicode chars are supported
>> and if so in which version?
>> >
>> > thanks a lot for the attention
>> > All the best
>> > Antonio
>> >
>> > --
>> >
>> > Dr. Antonio Calanducci, PhD
>> > INFN Catania
>> >
>> > Office:    +39 095 3785519
>> > Mobile:  +39 349 6762534
>> > Skype:   tcaland
>> >
>> > antonio.calanducci at ct.infn.it
>> > calanducci at unict.it
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Dr. *Antonio Calanducci*, PhD
> INFN Catania
> Dipartimento di Fisica e Astronomia - Università di Catania
> *Via Santa Sofia, 64*
> *95128 Catania (IT)*
>
> O : +39 095 378 5519
> M:  +39 349 6762534
> Skype: tcaland
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Dr. *Antonio Calanducci*, PhD
INFN Catania
Dipartimento di Fisica e Astronomia - Università di Catania
*Via Santa Sofia, 64*
*95128 Catania (IT)*

O : +39 095 378 5519
M:  +39 349 6762534
Skype: tcaland
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160701/b1cf2f0e/attachment.html>


More information about the OpenStack-dev mailing list