Open Stack

On 2013-10-31 00:49, Jamie Lennox wrote:
> Keystone signs the information in auth token with a certificate that 
> in
> most setups was generated for that instance of keystone. Swift will 
> use
> auth_token middleware to fetch the certificates of keystone so that 
> it
> can verify that the tokens are correct.
>
> My guess is that the two keystone instances are using different
> certificates and you are trying to validate a token with the other
> keystone instance (other certificates) and it won't work.
>
> If you are using the same keystone instance then it is possible that 
> the
> auth_token middleware in swift has cached the certificates for the 
> other
> keystone instance, so even though you have updated the values in 
> swift
> it is using the old certificates.
>
> Try deleting the certificates found in the folder specified by
> signing_dir in the swift setup and make sure you are issuing the 
> tokens
> from the keystone instance you are validating them against.
>
>
> Jamie
>
> On Wed, 2013-10-30 at 18:47 +0100, thorfinn at poivron.org wrote:
>> Hi all.
>>
>> * Hypervisor 1 : 192.168.1.120
>> 	- Keystone 1	: 192.168.3.141
>> 	- Swift-proxy 1	: 192.168.3.111
>> * Hypervisor 2 : 192.168.1.122
>> 	- Keystone 2	: 192.168.3.241
>> 	- Swift-proxy 2	: 192.168.3.211
>>
>> Keystone servers have the same mysql server, database and
>> configuration, so it's not a data issue.
>> Every server can ping and talk to all the other ones.
>>
>> When I talk to Swift-proxy 1, connected to Keystone 1 it works.
>> Same to Swift-proxy 2, connected to Keystone 2.
>>
>> If I connect Swift-proxy 1 to Keystone 2, it doesn't work anymore.
>> Same for Swift-proxy 2 to Keystone 1.
>>
>> All the servers are using Ubuntu 12.04.3/Havana and are up-to-date.
>>
>> When it works, I have this (keystone 2 connected to swift-proxy 2) :
>> # swift -V 2 -v -A http://192.168.3.241:5000/v2.0 -U service:swift 
>> -K
>> swift stat
>> StorageURL:
>> http://192.168.3.211:8080/v1/AUTH_5becb4a93e7f498bbe83534f4481dc0d
>> Auth Token:
>> 
>> MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0xMC0zMFQxNzozMzowMC4zNTI4MzAiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjMzOjAwWiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImF
 !
>  kbW
>>  luVVJMIj
>> 
>> ogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIxNjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidXNlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAsICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQASAXjP5gHiUWfl0e8gfh2O4n7IEoerLDmmTR07tQw1ibqyxyyhdNAVuThrceu0z3-VrHyeiuYcWAlJZzI6okeo02CDc7SRK4CjHLG9m8q80UrLOfT1-PbKes16ULKbGJpsBYykVXTV8ts+wQVAYcS73f2bwp6+Ki0Cygtfqkmvq
 !
>  Net
>>  7hDtSsvT
>> 
>> yAGKiLo0TbOYOpF96NelPwuzGAm2y-bcOhCCdJKo8iFEotTXK0SQzUQ78r3Mh1fsd6asoHRZxKKc0oXWm3KgJy1X-isnqopCMutDPPQCAXABOFb-OSovLMmmOS8ZZbII7RTd1e1z1sFYv3d67b0oc2A4e8DWAaVj
>>     Account: AUTH_5becb4a93e7f498bbe83534f4481dc0d
>> Containers: 4
>>     Objects: 11
>>       Bytes: 158989835
>> Accept-Ranges: bytes
>> X-Timestamp: 1382628587.87452
>> Content-Type: text/plain; charset=utf-8
>>
>> Oct 30 18:32:59 dev-api-002 proxy-server Verify error: Command
>> 'openssl' returned non-zero exit status 4
>> Oct 30 18:32:59 dev-api-002 proxy-server Authorization failed for 
>> token
>> 
>> MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj
>> 
>> AxMy0xMC0zMFQxNzozMjo1OC44NTY3MzEiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjMyOjU4WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl
>> 
>> Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD
>> 
>> M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1
>> 
>> ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2
>> 
>> lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx
>> 
>> NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX
>> 
>> Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs
>> 
>> ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT
>> 
>> AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCzitgoJ4ltBsCNN8xnLy3GlopgV5OlVRBa4fbHXcNT6expAdTYtx4I8q1cIF279NPVJO9T8hsedMSHwOxZvxJKskwFuuwUWT+cTBzkxlrY11Njmg9dGwQiJ1Pbb8oA3YZcgWjz6aY+1RajN-Lq9ugCidsY5tzFrHTwPed1VOcu
>> Wq2MKcMIqmt2m5b
>> Oct 30 18:32:59 dev-api-002 proxy-server Invalid user token - 
>> deferring
>> reject downstream
>>
>> Why the error if it works ?
>>
>> When it doesn't work, I have this (keystone 2 connected to 
>> swift-proxy
>> 1) :
>> # swift -V 2 -v -A http://192.168.3.241:5000/v2.0 -U service:swift 
>> -K
>> swift stat
>> Account HEAD failed:
>> http://192.168.3.111:8080/v1/AUTH_5becb4a93e7f498bbe83534f4481dc0d 
>> 401
>> Unauthorized
>>
>> Oct 30 18:34:53 dev-api-001 proxy-server Verify error: Command
>> 'openssl' returned non-zero exit status 4
>> Oct 30 18:34:53 dev-api-001 proxy-server Authorization failed for 
>> token
>> 
>> MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj
>> 
>> AxMy0xMC0zMFQxNzozNDo1My42NTY0NTMiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjM0OjUzWiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl
>> 
>> Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD
>> 
>> M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1
>> 
>> ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMTExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2
>> 
>> lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx
>> 
>> NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX
>> 
>> Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs
>> 
>> ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT
>> 
>> AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQAZRHKSulq-73euRy9HrYxMTU-HtRizyySVYkoo3CTCOgxFZz3CzelBIcp6HJySC6DVAW4Uz-xcTmtp1hju3vx3yAVstWtCczO-YZX4bUy4XFmfNje2ydJl5M2sSAUZ8160Vn3QnajesaRIvnu9w8WcpWsmaYjbx15ou2CzWnvH
>> j0V1lLTgA28dh90
>> Oct 30 18:34:53 dev-api-001 proxy-server Invalid user token - 
>> deferring
>> reject downstream
>> Oct 30 18:34:55 dev-api-001 proxy-server Verify error: Command
>> 'openssl' returned non-zero exit status 4
>> Oct 30 18:34:55 dev-api-001 proxy-server Authorization failed for 
>> token
>> 
>> MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj
>> 
>> AxMy0xMC0zMFQxNzozNDo1NS4xNTA5MjUiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjM0OjU1WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl
>> 
>> Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD
>> 
>> M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1
>> 
>> ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMTExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2
>> 
>> lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx
>> 
>> NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX
>> 
>> Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs
>> 
>> ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT
>> 
>> AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCXUDhH+Q8xUS6upUb8TtF2Uk-h2m-w84CmeJxKc-n7qGuozrZe7KPcrKp002ojDIY+CmGulWtXQD-IJ6V4hcjaVbmoxMIIVmMulxt1G2dLLIrtQCIUwnNFsOaaBiEZTus8DlpjIHGrLfcBRtzjewQXUA5PuRXC-ebtgE7wphMv
>> ETodRWB5zKixqmL
>> Oct 30 18:34:55 dev-api-001 proxy-server Invalid user token - 
>> deferring
>> reject downstream
>>
>> I have the same kind of logs entry than the working example but 
>> twice.
>>
>> _______________________________________________
>> Mailing list: 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


Perfect
The problem was much the ssl folder. It wasn't the same.
My problem is resolved

Thank you