<div dir="ltr">I am still tackling with this issue...<div><br></div><div>I appreciate if someone can help me out.</div><div><br></div><div>I figured out that token issue(POST request)  succeeds but GET request comes after is not proxied to keystone.</div><div><br></div><div>[POST request]</div><div>1. Swift client log</div><div>[root@opstack-controller src 2016.10.21 11:28:16]# swift list --debug</div><div>DEBUG:keystoneclient.auth.identity.v3.base:Making authentication request to <a href="http://opstack-cont-manage:5000/v3/auth/tokens">http://opstack-cont-manage:5000/v3/auth/tokens</a></div><div>INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): opstack-cont-manage</div><div>DEBUG:requests.packages.urllib3.connectionpool:"POST /v3/auth/tokens HTTP/1.1" 201 6420</div><div><br></div><div>2. Keystone access log</div><div>10.1.1.6 - - [21/Oct/2016:11:28:26 +0900] "POST /v3/auth/tokens HTTP/1.1" 201 6420 "-" "python-keystoneclient"<br></div><div><br></div><div>[GET request]</div><div>1. Swift client log</div><div>INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): opstack-cont-manage</div><div>DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/AUTH_a31f7e8e9ecc4d0e9d0d6c1de6c4c004?format=json HTTP/1.1" 503 100</div><div>INFO:swiftclient:REQ: curl -i <a href="http://opstack-cont-manage:8080/v1/AUTH_a31f7e8e9ecc4d0e9d0d6c1de6c4c004?format=json">http://opstack-cont-manage:8080/v1/AUTH_a31f7e8e9ecc4d0e9d0d6c1de6c4c004?format=json</a> -X GET -H "X-Auth-Token: gAAAAABYCX1KbdDk_Fl45pyHbfOwsPPVgjuAi7nnvF1-Qone51N4S8Z7B_S56RR0a78NjT1C-gJgngtkmiCYrlmUCAkQQIoPG0V4heRUjbAHYBSy22L2EvXexzK_f-hlcnBmqrNKcMUHFGEWgWMvwC6QrUVYP8dP332Sg5mOASEv5sCW9t9_VxA"</div><div>INFO:swiftclient:RESP STATUS: 503 Service Unavailable</div><div><br></div><div>2. Keyston access log</div><div>Nothing logged</div><div><br></div><div>Instead errors are seen in syslog.</div><div><br></div><div>Oct 21 11:29:15 opstack-controller keystoneauth.identity.base: Failed to contact the endpoint at <a href="http://opstack-cont-manage:5000/v3">http://opstack-cont-manage:5000/v3</a> for discovery. Fallback to using that endpoint as the base url.<br></div><div>Oct 21 11:29:15 opstack-controller keystoneauth.session: Failure: Unable to establish connection to <a href="http://opstack-cont-manage:5000/v3/auth/tokens">http://opstack-cont-manage:5000/v3/auth/tokens</a>. Retrying in 0.5s.</div><div>Oct 21 11:29:15 opstack-controller keystoneauth.session: Failure: Unable to establish connection to <a href="http://opstack-cont-manage:5000/v3/auth/tokens">http://opstack-cont-manage:5000/v3/auth/tokens</a>. Retrying in 1.0s.</div><div>Oct 21 11:29:16 opstack-controller keystoneauth.session: Failure: Unable to establish connection to <a href="http://opstack-cont-manage:5000/v3/auth/tokens">http://opstack-cont-manage:5000/v3/auth/tokens</a>. Retrying in 2.0s.</div><div> <br></div><div>May I ask which setting should  I look at either on keystone or swift-proxy?</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 14, 2016 at 2:52 PM, Mark Kirkwood <span dir="ltr"><<a href="mailto:mark.kirkwood@catalyst.net.nz" target="_blank">mark.kirkwood@catalyst.net.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
On 14/10/16 18:07, Yu Watanabe wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
At the same time , I see connection errors in the proxy server.<br>
<br>
Oct 14 14:05:08 opstack-controller proxy-server: Deferring reject downstream<br>
Oct 14 14:05:08 opstack-controller proxy-server: - - 14/Oct/2016/05/05/08 HEAD /v1/AUTH_a31f7e8e9ecc4d0e9d0d6<wbr>c1de6c4c004 HTTP/1.0 204 - Swift - - - - tx60f54bfed6d94357a10c0-005800<wbr>6784 - 0.0268 RL - 1476421508.123720884 1476421508.150515079 -<br>
Oct 14 14:05:08 opstack-controller keystoneauth.identity.base: Failed to contact the endpoint at <a href="http://opstack-cont-manage:5000/v3" rel="noreferrer" target="_blank">http://opstack-cont-manage:500<wbr>0/v3</a> for discovery. Fallback to using that endpoint as the base url.<br>
Oct 14 14:05:08 opstack-controller keystoneauth.identity.base: Failed to contact the endpoint at <a href="http://opstack-cont-manage:5000/v3" rel="noreferrer" target="_blank">http://opstack-cont-manage:500<wbr>0/v3</a> for discovery. Fallback to using that endpoint as the base url.<br>
Oct 14 14:05:08 opstack-controller keystoneauth.session: Failure: Unable to establish connection to <a href="http://opstack-cont-manage:5000/v3/auth/tokens" rel="noreferrer" target="_blank">http://opstack-cont-manage:500<wbr>0/v3/auth/tokens</a>. Retrying in 0.5s.<br>
Oct 14 14:05:08 opstack-controller keystoneauth.session: Failure: Unable to establish connection to <a href="http://opstack-cont-manage:5000/v3/auth/tokens" rel="noreferrer" target="_blank">http://opstack-cont-manage:500<wbr>0/v3/auth/tokens</a>. Retrying in 1.0s.<br>
Oct 14 14:05:09 opstack-controller keystoneauth.session: Failure: Unable to establish connection to <a href="http://opstack-cont-manage:5000/v3/auth/tokens" rel="noreferrer" target="_blank">http://opstack-cont-manage:500<wbr>0/v3/auth/tokens</a>. Retrying in 2.0s.<br>
<br>
Does this mean that problem lies on proxy server side?<br>
<br>
</blockquote>
<br>
Looks like the swift proxy - keystone connection is not happy. Have a look at what your keystone logs are saying.<br>
<br>
Cheers<span class="HOEnZb"><font color="#888888"><br>
<br>
Mark<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Yu Watanabe<div>渡辺 裕</div><div><br></div><div>LinkedIn : <span style="margin:0px;padding:0px;border:0px;font-size:13px;font-family:Arial,sans-serif;vertical-align:baseline;font-stretch:inherit;line-height:17px;color:rgb(102,102,102)"><a href="http://jp.linkedin.com/in/" target="_blank">jp.linkedin.com/in/</a></span><span style="margin:0px;padding:0px;border:0px;font-size:13px;font-family:Arial,sans-serif;vertical-align:baseline;font-stretch:inherit;line-height:17px;color:rgb(51,51,51)">yuwatanabe1</span></div></div></div></div></div>
</div>