Open Stack

Hi all.

* Hypervisor 1 : 192.168.1.120
	- Keystone 1	: 192.168.3.141
	- Swift-proxy 1	: 192.168.3.111
* Hypervisor 2 : 192.168.1.122
	- Keystone 2	: 192.168.3.241
	- Swift-proxy 2	: 192.168.3.211

Keystone servers have the same mysql server, database and 
configuration, so it's not a data issue.
Every server can ping and talk to all the other ones.

When I talk to Swift-proxy 1, connected to Keystone 1 it works.
Same to Swift-proxy 2, connected to Keystone 2.

If I connect Swift-proxy 1 to Keystone 2, it doesn't work anymore.
Same for Swift-proxy 2 to Keystone 1.

All the servers are using Ubuntu 12.04.3/Havana and are up-to-date.

When it works, I have this (keystone 2 connected to swift-proxy 2) :
# swift -V 2 -v -A http://192.168.3.241:5000/v2.0 -U service:swift -K 
swift stat
StorageURL: 
http://192.168.3.211:8080/v1/AUTH_5becb4a93e7f498bbe83534f4481dc0d
Auth Token: 
MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0xMC0zMFQxNzozMzowMC4zNTI4MzAiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjMzOjAwWiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbW
 luVVJMIj
ogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIxNjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidXNlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAsICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQASAXjP5gHiUWfl0e8gfh2O4n7IEoerLDmmTR07tQw1ibqyxyyhdNAVuThrceu0z3-VrHyeiuYcWAlJZzI6okeo02CDc7SRK4CjHLG9m8q80UrLOfT1-PbKes16ULKbGJpsBYykVXTV8ts+wQVAYcS73f2bwp6+Ki0CygtfqkmvqNet
 7hDtSsvT
yAGKiLo0TbOYOpF96NelPwuzGAm2y-bcOhCCdJKo8iFEotTXK0SQzUQ78r3Mh1fsd6asoHRZxKKc0oXWm3KgJy1X-isnqopCMutDPPQCAXABOFb-OSovLMmmOS8ZZbII7RTd1e1z1sFYv3d67b0oc2A4e8DWAaVj
    Account: AUTH_5becb4a93e7f498bbe83534f4481dc0d
Containers: 4
    Objects: 11
      Bytes: 158989835
Accept-Ranges: bytes
X-Timestamp: 1382628587.87452
Content-Type: text/plain; charset=utf-8

Oct 30 18:32:59 dev-api-002 proxy-server Verify error: Command 
'openssl' returned non-zero exit status 4
Oct 30 18:32:59 dev-api-002 proxy-server Authorization failed for token 
MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj
AxMy0xMC0zMFQxNzozMjo1OC44NTY3MzEiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjMyOjU4WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl
Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD
M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1
ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2
lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx
NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX
Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs
ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT
AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCzitgoJ4ltBsCNN8xnLy3GlopgV5OlVRBa4fbHXcNT6expAdTYtx4I8q1cIF279NPVJO9T8hsedMSHwOxZvxJKskwFuuwUWT+cTBzkxlrY11Njmg9dGwQiJ1Pbb8oA3YZcgWjz6aY+1RajN-Lq9ugCidsY5tzFrHTwPed1VOcu
Wq2MKcMIqmt2m5b
Oct 30 18:32:59 dev-api-002 proxy-server Invalid user token - deferring 
reject downstream

Why the error if it works ?

When it doesn't work, I have this (keystone 2 connected to swift-proxy 
1) :
# swift -V 2 -v -A http://192.168.3.241:5000/v2.0 -U service:swift -K 
swift stat
Account HEAD failed: 
http://192.168.3.111:8080/v1/AUTH_5becb4a93e7f498bbe83534f4481dc0d 401 
Unauthorized

Oct 30 18:34:53 dev-api-001 proxy-server Verify error: Command 
'openssl' returned non-zero exit status 4
Oct 30 18:34:53 dev-api-001 proxy-server Authorization failed for token 
MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj
AxMy0xMC0zMFQxNzozNDo1My42NTY0NTMiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjM0OjUzWiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl
Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD
M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1
ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMTExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2
lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx
NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX
Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs
ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT
AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQAZRHKSulq-73euRy9HrYxMTU-HtRizyySVYkoo3CTCOgxFZz3CzelBIcp6HJySC6DVAW4Uz-xcTmtp1hju3vx3yAVstWtCczO-YZX4bUy4XFmfNje2ydJl5M2sSAUZ8160Vn3QnajesaRIvnu9w8WcpWsmaYjbx15ou2CzWnvH
j0V1lLTgA28dh90
Oct 30 18:34:53 dev-api-001 proxy-server Invalid user token - deferring 
reject downstream
Oct 30 18:34:55 dev-api-001 proxy-server Verify error: Command 
'openssl' returned non-zero exit status 4
Oct 30 18:34:55 dev-api-001 proxy-server Authorization failed for token 
MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj
AxMy0xMC0zMFQxNzozNDo1NS4xNTA5MjUiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjM0OjU1WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl
Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD
M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1
ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMTExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2
lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx
NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX
Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs
ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT
AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCXUDhH+Q8xUS6upUb8TtF2Uk-h2m-w84CmeJxKc-n7qGuozrZe7KPcrKp002ojDIY+CmGulWtXQD-IJ6V4hcjaVbmoxMIIVmMulxt1G2dLLIrtQCIUwnNFsOaaBiEZTus8DlpjIHGrLfcBRtzjewQXUA5PuRXC-ebtgE7wphMv
ETodRWB5zKixqmL
Oct 30 18:34:55 dev-api-001 proxy-server Invalid user token - deferring 
reject downstream

I have the same kind of logs entry than the working example but twice.