Open Stack

Interesting,

Looking at my keystone log I'm seeing:

2016-10-21 18:52:43.313 1098 INFO eventlet.wsgi.server 
[req-652cc096-9df2-45db-84c7-cc6296c0d3ac - - - - -] 192.168.122.61 - - 
[21/Oct/2016 18:52:43] "POST /v2.0/tokens HTTP/1.1" 200 1920 0.269968
2016-10-21 18:52:43.360 1098 INFO keystone.common.wsgi 
[req-a531a97e-02c8-444f-af5a-91de28c85d88 
1682c148592e4b0baa3b6d2bf0cbb02c 9d9d844d97704c038abe15818d9953a9 - 
default default] GET http://192.168.122.61:35357/v3/auth/tokens
2016-10-21 18:52:43.476 1098 INFO eventlet.wsgi.server 
[req-a531a97e-02c8-444f-af5a-91de28c85d88 
1682c148592e4b0baa3b6d2bf0cbb02c 9d9d844d97704c038abe15818d9953a9 - 
default default] 192.168.122.61 - - [21/Oct/2016 18:52:43] "GET 
/v3/auth/tokens HTTP/1.1" 200 3085 0.162838


So you have progress of a kind - 1/2 works, need to see why that GET is 
failing. Some similar errors I've seen are due to using ip addresses in 
some parts of the config file(s) and hostnames in others (I'm using ips 
everywhere). Also  see if you have any ip filters/firewalls blocking 
traffic one way!

regards

Mark

On 21/10/16 16:04, Yu Watanabe wrote:
> I am still tackling with this issue...
>
> I appreciate if someone can help me out.
>
> I figured out that token issue(POST request)  succeeds but GET request 
> comes after is not proxied to keystone.
>
> [POST request]
> 1. Swift client log
> [root at opstack-controller src 2016.10.21 11:28:16]# swift list --debug
> DEBUG:keystoneclient.auth.identity.v3.base:Making authentication 
> request to http://opstack-cont-manage:5000/v3/auth/tokens
> INFO:requests.packages.urllib3.connectionpool:Starting new HTTP 
> connection (1): opstack-cont-manage
> DEBUG:requests.packages.urllib3.connectionpool:"POST /v3/auth/tokens 
> HTTP/1.1" 201 6420
>
> 2. Keystone access log
> 10.1.1.6 - - [21/Oct/2016:11:28:26 +0900] "POST /v3/auth/tokens 
> HTTP/1.1" 201 6420 "-" "python-keystoneclient"
>
> [GET request]
> 1. Swift client log
> INFO:requests.packages.urllib3.connectionpool:Starting new HTTP 
> connection (1): opstack-cont-manage
> DEBUG:requests.packages.urllib3.connectionpool:"GET 
> /v1/AUTH_a31f7e8e9ecc4d0e9d0d6c1de6c4c004?format=json HTTP/1.1" 503 100
> INFO:swiftclient:REQ: curl -i 
> http://opstack-cont-manage:8080/v1/AUTH_a31f7e8e9ecc4d0e9d0d6c1de6c4c004?format=json 
> -X GET -H "X-Auth-Token: 
> gAAAAABYCX1KbdDk_Fl45pyHbfOwsPPVgjuAi7nnvF1-Qone51N4S8Z7B_S56RR0a78NjT1C-gJgngtkmiCYrlmUCAkQQIoPG0V4heRUjbAHYBSy22L2EvXexzK_f-hlcnBmqrNKcMUHFGEWgWMvwC6QrUVYP8dP332Sg5mOASEv5sCW9t9_VxA"
> INFO:swiftclient:RESP STATUS: 503 Service Unavailable
>
> 2. Keyston access log
> Nothing logged
>
> Instead errors are seen in syslog.
>
> Oct 21 11:29:15 opstack-controller keystoneauth.identity.base: Failed 
> to contact the endpoint at http://opstack-cont-manage:5000/v3 for 
> discovery. Fallback to using that endpoint as the base url.
> Oct 21 11:29:15 opstack-controller keystoneauth.session: Failure: 
> Unable to establish connection to 
> http://opstack-cont-manage:5000/v3/auth/tokens. Retrying in 0.5s.
> Oct 21 11:29:15 opstack-controller keystoneauth.session: Failure: 
> Unable to establish connection to 
> http://opstack-cont-manage:5000/v3/auth/tokens. Retrying in 1.0s.
> Oct 21 11:29:16 opstack-controller keystoneauth.session: Failure: 
> Unable to establish connection to 
> http://opstack-cont-manage:5000/v3/auth/tokens. Retrying in 2.0s.
>
> May I ask which setting should  I look at either on keystone or 
> swift-proxy?
>
>
> On Fri, Oct 14, 2016 at 2:52 PM, Mark Kirkwood 
> <mark.kirkwood at catalyst.net.nz <mailto:mark.kirkwood at catalyst.net.nz>> 
> wrote:
>
>
>
>     On 14/10/16 18:07, Yu Watanabe wrote:
>
>         At the same time , I see connection errors in the proxy server.
>
>         Oct 14 14:05:08 opstack-controller proxy-server: Deferring
>         reject downstream
>         Oct 14 14:05:08 opstack-controller proxy-server: - -
>         14/Oct/2016/05/05/08 HEAD
>         /v1/AUTH_a31f7e8e9ecc4d0e9d0d6c1de6c4c004 HTTP/1.0 204 - Swift
>         - - - - tx60f54bfed6d94357a10c0-0058006784 - 0.0268 RL -
>         1476421508.123720884 1476421508.150515079 -
>         Oct 14 14:05:08 opstack-controller keystoneauth.identity.base:
>         Failed to contact the endpoint at
>         http://opstack-cont-manage:5000/v3
>         <http://opstack-cont-manage:5000/v3> for discovery. Fallback
>         to using that endpoint as the base url.
>         Oct 14 14:05:08 opstack-controller keystoneauth.identity.base:
>         Failed to contact the endpoint at
>         http://opstack-cont-manage:5000/v3
>         <http://opstack-cont-manage:5000/v3> for discovery. Fallback
>         to using that endpoint as the base url.
>         Oct 14 14:05:08 opstack-controller keystoneauth.session:
>         Failure: Unable to establish connection to
>         http://opstack-cont-manage:5000/v3/auth/tokens
>         <http://opstack-cont-manage:5000/v3/auth/tokens>. Retrying in
>         0.5s.
>         Oct 14 14:05:08 opstack-controller keystoneauth.session:
>         Failure: Unable to establish connection to
>         http://opstack-cont-manage:5000/v3/auth/tokens
>         <http://opstack-cont-manage:5000/v3/auth/tokens>. Retrying in
>         1.0s.
>         Oct 14 14:05:09 opstack-controller keystoneauth.session:
>         Failure: Unable to establish connection to
>         http://opstack-cont-manage:5000/v3/auth/tokens
>         <http://opstack-cont-manage:5000/v3/auth/tokens>. Retrying in
>         2.0s.
>
>         Does this mean that problem lies on proxy server side?
>
>
>     Looks like the swift proxy - keystone connection is not happy.
>     Have a look at what your keystone logs are saying.
>
>     Cheers
>
>     Mark
>
>
>
>
> -- 
> Yu Watanabe
> 渡辺 裕
>
> LinkedIn : jp.linkedin.com/in/ <http://jp.linkedin.com/in/>yuwatanabe1