[Openstack] Struggling to get the s3 api interface to work with swift.
Kota TSUYUZAKI
tsuyuzaki.kota at lab.ntt.co.jp
Fri Jun 1 08:01:00 UTC 2018
Hi Shyam,
> s3curl: StringToSign='PUT\n\n\nFri, 01 Jun 2018 05:47:53
> +0000\n/s3server/testbucket'
It looks like your StringToSign doesn't include correct info for your request.(e.g. resource should be only "/testbucket")
See https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html for more info.
Your request should be:
> * Connected to s3server (20.20.20.229) port 8080 (#0)
>> PUT /testbucket HTTP/1.1
>> Host: s3server:8080
>> User-Agent: curl/7.58.0
>> Accept: */*
>> Date: Fri, 01 Jun 2018 05:47:53 +0000
>> Authorization: AWS
> 76498e1413284b9d961d452db608dff4:uzQs5hJOnwY4dyFk0PIxHHwmVcA=
>> Content-Length: 0
then, the string to sign should include the info corretly.
Looking at s3curl code, it looks to fall into cname case?
https://github.com/scality/S3/blob/master/tests/functional/s3curl/s3curl.pl#L311-L328
Once I played with s3curl the request with ordinary endpoint signing case worked correctly.
Again, I'm not an expert of s3curl so no idea why your setting went to the cname case tho.
Best,
Kota
(2018/06/01 14:49), Shyam Prasad N wrote:
> Hi Kota,
>
> Thanks for the response.
> When I specify the URL as http://s3server:8080/testbucket, it throws back a
> SignatureDoesNotMatch error.
>
> eightkpc at objectstore1:~/s3curl$ ./s3curl.pl --debug --id=testerks
> --createBucket -- http://s3server:8080/testbucket
> s3curl: Found the url: host=s3server; port=8080; uri=/testbucket; query=;
> s3curl: cname endpoint signing case
> s3curl: StringToSign='PUT\n\n\nFri, 01 Jun 2018 05:47:53
> +0000\n/s3server/testbucket'
> s3curl: exec curl -v -H 'Date: Fri, 01 Jun 2018 05:47:53 +0000' -H
> 'Authorization: AWS
> 76498e1413284b9d961d452db608dff4:uzQs5hJOnwY4dyFk0PIxHHwmVcA=' -L -H
> 'content-type: ' --data-binary -X PUT http://s3server:8080/testbucket
> * Trying 20.20.20.229...
> * TCP_NODELAY set
> * Connected to s3server (20.20.20.229) port 8080 (#0)
>> PUT /testbucket HTTP/1.1
>> Host: s3server:8080
>> User-Agent: curl/7.58.0
>> Accept: */*
>> Date: Fri, 01 Jun 2018 05:47:53 +0000
>> Authorization: AWS
> 76498e1413284b9d961d452db608dff4:uzQs5hJOnwY4dyFk0PIxHHwmVcA=
>> Content-Length: 0
>>
> < HTTP/1.1 403 Forbidden
> < x-amz-id-2: tx5c2ac9ea26a046ac96e4d-005b10de09
> < x-amz-request-id: tx5c2ac9ea26a046ac96e4d-005b10de09
> < Content-Type: application/xml
> < X-Trans-Id: tx5c2ac9ea26a046ac96e4d-005b10de09
> < X-Openstack-Request-Id: tx5c2ac9ea26a046ac96e4d-005b10de09
> < Date: Fri, 01 Jun 2018 05:47:53 GMT
> < Transfer-Encoding: chunked
> * HTTP error before end of send, stop sending
> <
> <?xml version='1.0' encoding='UTF-8'?>
> * Closing connection 0
> <Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we
> calculated does not match the signature you provided. Check your key and
> signing
> method.</Message><RequestId>tx5c2ac9ea26a046ac96e4d-005b10de09</RequestId></Error>
>
> Regards,
> Shyam
>
> On Fri, Jun 1, 2018 at 11:02 AM, Kota TSUYUZAKI <
> tsuyuzaki.kota at lab.ntt.co.jp> wrote:
>
>> Hi Shyam,
>>
>> You should specify the path starts from bucket. It will be like
>>
>> http://20.20.20.229:8080/testBucket
>>
>> assuming you didn't configure virtual-hosted style.
>> Even if your *Swift* endpoint is http://20.20.20.229:8080/v1/AUTH_
>> dc86f7d8787b46158268bd77098b6578,
>> swift3 doesn't require the account in the path so far as well as actual
>> amazon s3.
>>
>> Best,
>> Kota
>>
>> (2018/05/31 21:33), Shyam Prasad N wrote:
>>> Hi Yuxin,
>>>
>>> Thank you for sharing your configs.
>>> So I've managed to get past the Signature not matching error.
>>>
>>> Now the error is different. InvalidbucketName:
>>>
>>> eightkpc at objectstore1:~/s3curl$ ./s3curl.pl --debug --id=testerks
>>> --createBucket --
>>> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b65
>> 78/testBucket
>>> s3curl: Found the url: host=20.20.20.229; port=8080;
>>> uri=/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket; query=;
>>> s3curl: cname endpoint signing case
>>> s3curl: StringToSign='PUT\n\n\nThu, 31 May 2018 12:02:57 +0000\n/
>>> 20.20.20.229/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket'
>>> s3curl: exec curl -v -H 'Date: Thu, 31 May 2018 12:02:57 +0000' -H
>>> 'Authorization: AWS
>>> 76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=' -L -H
>>> 'content-type: ' --data-binary -X PUT
>>> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b65
>> 78/testBucket
>>> * Trying 20.20.20.229...
>>> * TCP_NODELAY set
>>> * Connected to 20.20.20.229 (20.20.20.229) port 8080 (#0)
>>>> PUT /v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket HTTP/1.1
>>>> Host: 20.20.20.229:8080
>>>> User-Agent: curl/7.58.0
>>>> Accept: */*
>>>> Date: Thu, 31 May 2018 12:02:57 +0000
>>>> Authorization: AWS
>>> 76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=
>>>> Content-Length: 0
>>>>
>>> < HTTP/1.1 400 Bad Request
>>> < x-amz-id-2: tx18266052d5044eb2a3bc7-005b0fe471
>>> < x-amz-request-id: tx18266052d5044eb2a3bc7-005b0fe471
>>> < Content-Type: application/xml
>>> < X-Trans-Id: tx18266052d5044eb2a3bc7-005b0fe471
>>> < X-Openstack-Request-Id: tx18266052d5044eb2a3bc7-005b0fe471
>>> < Date: Thu, 31 May 2018 12:02:57 GMT
>>> < Transfer-Encoding: chunked
>>> * HTTP error before end of send, stop sending
>>> <
>>> <?xml version='1.0' encoding='UTF-8'?>
>>> * Closing connection 0
>>> <Error><Code>InvalidBucketName</Code><Message>The specified bucket is
>> not
>>> valid.</Message><RequestId>tx18266052d5044eb2a3bc7-
>> 005b0fe471</RequestId><BucketName>v1</BucketName></
>> Error>eightkpc at objectstore1:~/s3curl$
>>>
>>>
>>> My specified endpoint is
>>> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578
>>> What am I doing wrong?
>>>
>>> Regards,
>>> Shyam
>>>
>>> On Wed, May 30, 2018 at 7:32 PM, Yuxin Wang <wang.yuxin at ostorage.com.cn>
>>> wrote:
>>>
>>>> Hi Shyam,
>>>>
>>>> No problem. The output of the commands is attached.
>>>>
>>>> And my test cluster is on Swift v2.15.1 with Swift3 v1.12
>>>>
>>>> Also, here is the common process when I'm creating an S3 credential and
>>>> using in s3curl. Hope it helps.
>>>>
>>>> 1. Create a user and a project, and assign a proper role.
>>>>
>>>> openstack project create testproject
>>>> openstack user create testuser --password 123
>>>> openstack role add --project testproject --user testuser _member_
>>>>
>>>> 2. Check accessibility to swift
>>>>
>>>> create a test-openrc file with above info
>>>> source test-openrc
>>>> swift list
>>>>
>>>> 3.Create a credential
>>>>
>>>> openstack credential create --type ec2 --project testproject testuser
>>>> '{"access": "testaccess", "secret": "testsecret"}'
>>>>
>>>> 4. Use it in s3curl
>>>>
>>>> add the endpoint url to `my @endpoints` in s3curl.pl
>>>> add the credential to .s3curl config file
>>>>
>>>> do `s3curl.pl -i cred_name --debug -- http://endpoint -X GET`
>>>>
>>>>
>>>>
>>>>
>>>> 在 2018年5月25日，18:17，Shyam Prasad N <nspmangalore at gmail.com> 写道：
>>>>
>>>> Hi Yuxin,
>>>>
>>>> If you don't mind, can you share the output of the following commands in
>>>> your running swift3 setup?
>>>>
>>>> openstack credential list
>>>> openstack ec2 credentials list
>>>> cat /etc/swift/proxy-server.conf
>>>>
>>>> Also, what are the access keys and secret keys that you use?
>>>> I want to make sure that I'm not missing anything in configuration.
>>>>
>>>> Regards,
>>>> Shyam
>>>>
>>>> On Fri, May 25, 2018 at 3:05 PM, Shyam Prasad N <nspmangalore at gmail.com
>>>
>>>> wrote:
>>>>
>>>>> Tried that. Unfortunately same error.
>>>>> Is there anything I can do to troubleshoot this?
>>>>>
>>>>> On Fri, May 25, 2018 at 2:56 PM, Yuxin Wang <
>> wang.yuxin at ostorage.com.cn>
>>>>> wrote:
>>>>>
>>>>>> They can be any strings.
>>>>>>
>>>>>> Replace them with whatever you want.
>>>>>>
>>>>>> - Yuxin
>>>>>>
>>>>>> 在 2018年5月25日，14:57，Shyam Prasad N <nspmangalore at gmail.com> 写道：
>>>>>>
>>>>>> Thanks. I'll try this.
>>>>>> But what values do I use in place of ak and sk? I want to use some
>>>>>> command to get those values, right?
>>>>>>
>>>>>> On Fri, May 25, 2018 at 9:52 AM, Yuxin Wang <
>> wang.yuxin at ostorage.com.cn>
>>>>>> wrote:
>>>>>>
>>>>>>> I created ec2 credentials using command `openstack credential
>> create`.
>>>>>>>
>>>>>>> i.e.
>>>>>>>
>>>>>>> openstack credential create --type ec2 --project proj user
>> '{"access":
>>>>>>> "ak", "secret": "sk”}'
>>>>>>>
>>>>>>>
>>>>>>> It seems the two credentials are not the same thing.
>>>>>>>
>>>>>>> Ref:
>>>>>>>
>>>>>>> https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.1.1/
>>>>>>> com.ibm.spectrum.scale.v4r11.adv.doc/bl1adv_ConfigureOpensta
>>>>>>> ckEC2credentials.htm
>>>>>>> <https://nam05.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fwww.ibm.com%2Fsupport%2Fknowledgecenter%
>> 2Fen%2FSTXKQY_4.1.1%2Fcom.ibm.spectrum.scale.v4r11.adv.doc%2Fbl1adv_
>> ConfigureOpenstackEC2credentials.htm&data=02%7C01%7C%
>> 7Cc6d4af73a0fd4208f9ac08d5c20f0a30%7C84df9e7fe9f640afb435aaaaaaaa
>> aaaa%7C1%7C0%7C636628292198191185&sdata=cBwi02p2zc3db3%2Bp8PDvrgAZ2%
>> 2FkqjB6wnuW%2FEt0zWPA%3D&reserved=0>
>>>>>>>
>>>>>>> 在 2018年5月25日，10:32，Shyam Prasad N <nspmangalore at gmail.com> 写道：
>>>>>>>
>>>>>>> Yes, I did.
>>>>>>> I don't think this is s3curl related issue, because I tried with
>> python
>>>>>>> AWS SDK, and got the same error.
>>>>>>>
>>>>>>> On Fri, May 25, 2018, 07:42 Yuxin Wang <wang.yuxin at ostorage.com.cn>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Did you add 127.0.0.1 to the endpoint list in s3curl.pl
>>>>>>>> <https://nam05.safelinks.protection.outlook.com/?url=
>> http%3A%2F%2Fs3curl.pl%2F&data=02%7C01%7C%7Cc6d4af73a0fd4208f9ac08d5c20f
>> 0a30%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%
>> 7C636628292198191185&sdata=nr9HL8HskZJ8OwCfpuU2pKg7%
>> 2FPnzWhTD1b4wEi02o5A%3D&reserved=0>
>>>>>>>> ?
>>>>>>>>
>>>>>>>> i.e.
>>>>>>>>
>>>>>>>> my @endpoints = (‘127.0.0.1’);
>>>>>>>>
>>>>>>>> 在 2018年5月24日，19:48，Shyam Prasad N <nspmangalore at gmail.com> 写道：
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I've been trying to get swift3 to work for several days now. But I
>>>>>>>> haven't managed to get it running.
>>>>>>>> Both with tempauth and keystoneauth, I'm getting the same error:
>>>>>>>>
>>>>>>>> eightkpc at objectstore1:~/s3curl$ ./s3curl.pl
>>>>>>>> <https://eur03.safelinks.protection.outlook.com/?url=
>> http%3A%2F%2Fs3curl.pl&data=02%7C01%7C%7C39742b8c6bf847ee381508d5c16d1b21%
>> 7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636627596701206160&sdata=
>> lZmKF41unLVD9J1ALYUv8Vkab448NOeGmcWcR56Rgxs%3D&reserved=0>
>>>>>>>> --id=testerks -- http://127.0.0.1:8080/
>>>>>>>> <https://eur03.safelinks.protection.outlook.com/?url=
>> http%3A%2F%2F127.0.0.1%3A8080%2F&data=02%7C01%7C%
>> 7C39742b8c6bf847ee381508d5c16d1b21%7C84df9e7fe9f640afb435aaaaaaaa
>> aaaa%7C1%7C0%7C636627596701206160&sdata=CEHEVR1HSpK0ehQ8mmOBA7fmjQsu7v
>> jqKZda3NYVDWQ%3D&reserved=0>
>>>>>>>> <?xml version='1.0' encoding='UTF-8'?>
>>>>>>>> <Error><Code>SignatureDoesNotMatch</Code><Message>The request
>>>>>>>> signature we calculated does not match the signature you provided.
>> Check
>>>>>>>> your key and signing method.</Message><RequestId>tx
>>>>>>>> a691e7ca97a44d56bc4c2-005b06a292</RequestId></Error>
>>>>>>>>
>>>>>>>> May 24 11:31:30 localhost proxy-server: 127.0.0.1 127.0.0.1
>>>>>>>> 24/May/2018/11/31/30 GET / HTTP/1.0 403 - curl/7.58.0 - - 277 -
>>>>>>>> txa691e7ca97a44d56bc4c2-005b06a292 - 0.0200 - -
>> 1527161490.543112040
>>>>>>>> 1527161490.563107014 -
>>>>>>>> May 24 11:31:30 localhost proxy-server: STDERR: 127.0.0.1 - -
>>>>>>>> [24/May/2018 11:31:30] "GET / HTTP/1.1" 403 621 0.021979 (txn:
>>>>>>>> txa691e7ca97a44d56bc4c2-005b06a292)
>>>>>>>>
>>>>>>>> eightkpc at objectstore1:~$ cat .s3curl
>>>>>>>> %awsSecretAccessKeys = (
>>>>>>>> tester => {
>>>>>>>> id => 'test:tester',
>>>>>>>> key => 'testing',
>>>>>>>> },
>>>>>>>> testerks => {
>>>>>>>> id => 'e6289a1b5692461388d0597a4873d054',
>>>>>>>> key => '88bb706887094696b082f008ba133ad7',
>>>>>>>> },
>>>>>>>> );
>>>>>>>>
>>>>>>>> eightkpc at objectstore1:~$ openstack ec2 credentials show
>>>>>>>> e6289a1b5692461388d0597a4873d054
>>>>>>>> +------------+----------------------------------------------
>>>>>>>> ------------------------------------------------------------
>>>>>>>> --------------------------+
>>>>>>>> | Field | Value
>>>>>>>>
>>>>>>>> |
>>>>>>>> +------------+----------------------------------------------
>>>>>>>> ------------------------------------------------------------
>>>>>>>> --------------------------+
>>>>>>>> | access | e6289a1b5692461388d0597a4873d0
>>>>>>>> 54
>>>>>>>> |
>>>>>>>> | links | {u'self': u'http://controller:5000/v3/us
>>>>>>>> ers/d7df7b56343b4ea988869fc30efeda09/credentials/OS-EC2/e628
>>>>>>>> 9a1b5692461388d0597a4873d054'} |
>>>>>>>> | project_id | dc86f7d8787b46158268bd77098b65
>>>>>>>> 78
>>>>>>>> |
>>>>>>>> | secret | 88bb706887094696b082f008ba133a
>>>>>>>> d7
>>>>>>>> |
>>>>>>>> | trust_id | None
>>>>>>>>
>>>>>>>> |
>>>>>>>> | user_id | d7df7b56343b4ea988869fc30efeda
>>>>>>>> 09
>>>>>>>> |
>>>>>>>> +------------+----------------------------------------------
>>>>>>>> ------------------------------------------------------------
>>>>>>>> --------------------------+
>>>>>>>>
>>>>>>>> Can someone please let me know what is going on?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Shyam
>>>>>>>> <proxy-server.conf>_______________________________________________
>>>>>>>> Mailing list: https://eur03.safelinks.protection.outlook.com/?url=
>>>>>>>> http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%
>>>>>>>> 2Flistinfo%2Fopenstack&data=02%7C01%7C%7C39742b8c6bf847ee3
>>>>>>>> 81508d5c16d1b21%7C84df9e7fe9f640afb435aaaaaaaa
>>>>>>>> aaaa%7C1%7C0%7C636627596701206160&sdata=KI%2F2T2FhVQJTeX1KbI
>>>>>>>> ObDZVDiUA3SbTq6Pplo1bc7ak%3D&reserved=0
>>>>>>>> Post to : openstack at lists.openstack.org
>>>>>>>> Unsubscribe : https://eur03.safelinks.protection.outlook.com/?url=
>>>>>>>> http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%
>>>>>>>> 2Flistinfo%2Fopenstack&data=02%7C01%7C%7C39742b8c6bf847ee3
>>>>>>>> 81508d5c16d1b21%7C84df9e7fe9f640afb435aaaaaaaa
>>>>>>>> aaaa%7C1%7C0%7C636627596701206160&sdata=KI%2F2T2FhVQJTeX1KbI
>>>>>>>> ObDZVDiUA3SbTq6Pplo1bc7ak%3D&reserved=0
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -Shyam
>>>>>> _______________________________________________
>>>>>> Mailing list: https://nam05.safelinks.protection.outlook.com/?url=
>>>>>> http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%
>>>>>> 2Flistinfo%2Fopenstack&data=02%7C01%7C%7Cc6d4af73a0fd4208f
>>>>>> 9ac08d5c20f0a30%7C84df9e7fe9f640afb435aaaaaaaa
>>>>>> aaaa%7C1%7C0%7C636628292198347486&sdata=tGhHmhX%2By9RVFjl%
>>>>>> 2B31%2BVgRiN1mD%2Fc%2B7QLiImlGnCv98%3D&reserved=0
>>>>>> Post to : openstack at lists.openstack.org
>>>>>> Unsubscribe : https://nam05.safelinks.protection.outlook.com/?url=
>>>>>> http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%
>>>>>> 2Flistinfo%2Fopenstack&data=02%7C01%7C%7Cc6d4af73a0fd4208f
>>>>>> 9ac08d5c20f0a30%7C84df9e7fe9f640afb435aaaaaaaa
>>>>>> aaaa%7C1%7C0%7C636628292198347486&sdata=tGhHmhX%2By9RVFjl%
>>>>>> 2B31%2BVgRiN1mD%2Fc%2B7QLiImlGnCv98%3D&reserved=0
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -Shyam
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> -Shyam
>>>> _______________________________________________
>>>> Mailing list: https://nam05.safelinks.protection.outlook.com/?url=
>>>> http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%
>>>> 2Flistinfo%2Fopenstack&data=02%7C01%7C%7C7bf3d25a540b4d402d3d08d5c22a
>> 5c90%
>>>> 7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636628409546910462&sdata=%
>>>> 2FhXb5bRIU0kCZSDj4pJXfEkVYEtBApcA6OjhKHw1fMQ%3D&reserved=0
>>>> Post to : openstack at lists.openstack.org
>>>> Unsubscribe : https://nam05.safelinks.protection.outlook.com/?url=
>>>> http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%
>>>> 2Flistinfo%2Fopenstack&data=02%7C01%7C%7C7bf3d25a540b4d402d3d08d5c22a
>> 5c90%
>>>> 7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636628409546910462&sdata=%
>>>> 2FhXb5bRIU0kCZSDj4pJXfEkVYEtBApcA6OjhKHw1fMQ%3D&reserved=0
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
>> openstack
>>> Post to : openstack at lists.openstack.org
>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
>> openstack
>>>
>>
>>
>>
>>
>
>
More information about the Openstack
mailing list