<div dir="ltr">Thank you for answer.<div><br></div><div style>Chmouel, do you mean the auth_token on Keystone or swift proxy server?</div><div style><br></div><div style>from /etc/keystone/keystone.conf</div><div style><br>
</div><div style><div>[filter:token_auth]</div><div>paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory</div><div><br></div><div style>from /etc/swift/proxy-server.conf</div><div style><br></div><div style>
<div>[filter:authtoken]</div><div>paste.filter_factory = keystone.middleware.auth_token:filter_factory</div><div>...</div><div>memcache_servers = <a href="http://127.0.0.1:11211">127.0.0.1:11211</a><br></div><div><br></div>
<div style>Further debugging proved that hosts without memcached don't return the error 403. I'm still investigating what service can return such error body message.</div><div><br></div></div></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Tue, Jun 4, 2013 at 12:55 PM, Chmouel Boudjnah <span dir="ltr"><<a href="mailto:chmouel@enovance.com" target="_blank">chmouel@enovance.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I have seen this when keystone is too busy for validating tokens.<br>
getting keystone behind apache or scaling up keystone make things a<br>
better (and make sure you are using swift memcache connection in<br>
auth_token).<br>
<br>
<br>
Chmouel.<br>
<div><div class="h5"><br>
On Mon, Jun 3, 2013 at 8:15 PM, Andrii Loshkovskyi<br>
<<a href="mailto:loshkovskyi@gmail.com">loshkovskyi@gmail.com</a>> wrote:<br>
> Hello,<br>
><br>
> I would appreciate if you help me to troubleshoot the following issue:<br>
><br>
> I am having error 403 intermittenly when listing containers in swift.<br>
> Sometimes the error appears a few times per hour, sometimes once per day.<br>
> Basically, it's possible to reproduce the error with a simple curl command:<br>
><br>
> curl --get -v -H 'X-Auth-Token: ef644...'<br>
> <a href="http://swift-proxy.example.com:8080/v1/AUTH_323d0." target="_blank">http://swift-proxy.example.com:8080/v1/AUTH_323d0.</a>..<br>
> <body><br>
> <h1>403 Forbidden</h1><br>
> Access was denied to this resource.<br /><br /><br>
> </body><br>
><br>
> The token and swift proxy endpoint are all correct as most of the time the<br>
> command works.<br>
><br>
> A few words about infrastructure: I use swift 1.7.4 and several swift<br>
> proxies. Users are authenticated via Keystone. Tokens are cached with<br>
> memcached on swift proxy servers.<br>
><br>
> I did a lot of tests to figure out what service generates such error:<br>
><br>
> - same issue happens with each swift proxy server, with or without memcached<br>
> enabled<br>
> - it happens with different users and in different tenants<br>
> - I downloaded sources of swift and Keystone and grepped on that error.<br>
> There are some HTTPForbidden values returned in code but no one with the<br>
> body 'Access denied to this resource'<br>
> - I tried monitoring traffic with tcpdump to catch the error and understand<br>
> who's sending it but with no success yet<br>
> - the issue might be related to swift ACL rules but I haven't set any<br>
> read/write permissions for containers<br>
> - set debug logs for swift proxy but nothing has been found yet<br>
><br>
> Please help me to understand how this error is returned. Thank you for your<br>
> time.<br>
><br>
><br>
> --<br>
> Kind regards,<br>
> Andrii Loshkovskyi<br>
><br>
</div></div>> _______________________________________________<br>
> Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
> Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Kind regards,<div>Andrii Loshkovskyi</div>
</div>