[Openstack-operators] keystone is throwing Authorization Failed: 'module' object is not callable errors

Fischer, Matt matthew.fischer at twcable.com
Tue Aug 5 01:48:57 UTC 2014


Glad it's working for you.

As for 35357 vs 5000, 5000 is the standard endpoint for public requests. 35357 is the service endpoint that I usually use for bootstrapping and is used for admin and internal requests (from other services for example). If you get the service token from the keystone config file you can use the 35357 endpoint without a password or username. It's how you bootstrap your system or recover if you forgot the admin password.

You can use the service token like this without a password, username, or tenant set.

SERVICE_TOKEN=<value from file> SERVICE_ENDPOINT=http://foo:35357/v2.0 keystone user-list



From: Jeff Silverman <jeff at sweetlabs.com<mailto:jeff at sweetlabs.com>>
Date: Monday, August 4, 2014 6:23 PM
To: Abel Lopez <alopgeek at gmail.com<mailto:alopgeek at gmail.com>>, "openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>" <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Subject: Re: [Openstack-operators] keystone is throwing Authorization Failed: 'module' object is not callable errors

Abel,

I tried that.  Unfortunately, the argument to the curl command is -d '{ "key1": "value", "password", "comp'cated" }' .  The ' character is significant in " delimited strings.  The \ is not significant in " delimited strings.  I spent about an hour playing with the echo command working this out.

The solution that I came up with is -d @f.txt which means get the data from file f.txt.  Then I put

{ "key1": "value", "password", "comp'cated" }

In the file f.txt.  Because f.txt isn't parsed by the shell, I can use special characters.  In fact, I can use binary characters, so if I want to pass a .jpeg file to the API (I have no idea why I'd want to do that, but work with me here), I could do that.


keystone has mysteriously started working.  I don't know why.  I added some debugging code to it, and it started working.  So then I took the debugging code out of it, and it's still working.  I don't believe that my changes made a difference.  Something else has changed, but I don't know what that might be.  I am going to do some more testing.  Very frustrating.


Everybody who has helped me: thank you so very much.  I really appreciate it.


Jeff




On Mon, Aug 4, 2014 at 5:15 PM, Abel Lopez <alopgeek at gmail.com<mailto:alopgeek at gmail.com>> wrote:
You made reference to a complex password in the configs, IIRC, ! ? $ may be interpreted by the shell, if you have those, escape them like this
pa\$\$word


On Monday, August 4, 2014, Jeff Silverman <jeff at sweetlabs.com<mailto:jeff at sweetlabs.com>> wrote:
Abel,

Sticking a \ in front of what, exactly, please?   I'm still a newbie.

Thank you


Jeff



On Mon, Aug 4, 2014 at 3:48 PM, Abel Lopez <alopgeek at gmail.com> wrote:
I’ve seen similar before, especially with $ and !, try sticking a \ in front, see if that helps

On Aug 4, 2014, at 2:51 PM, Jeff Silverman <jeff at sweetlabs.com> wrote:

Matt,

The --debug switch was most helpful.  Unfortunately, my co-worker picked a very secure password with special characters, and since the curl command -d switch has its arguments enclosed by ' and " I couldn't figure out how to escape the special characters that were tripping up the shell.

However, I read the curl man page to see how it handled binary data (for example, if I wanted to upload a JPEG using curl) and I found an interesting wrinkle with the -d switch: if the next character is an @ character, then -d interpreters the string as a filename to get the data from.  So I created a file f.txt which contains

{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "XXXXX>'MA/#Z9e?_T9_XXXX}}}


Then I used:

# curl -i -X POST http://controller1-prod.sea.opencandy.com:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d @f.txt

and got

HTTP/1.1 200 OK
Vary: X-Auth-Token
Content-Type: application/json
Date: Mon, 04 Aug 2014 21:26:32 GMT
Transfer-Encoding: chunked

{"access": {"token": {"expires": "2014-08-05T21:26:32Z", ...}}}


# curl -i -X POST http://controller1-prod.sea.opencandy.com:35357/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d @f.txt
HTTP/1.1 200 OK
Vary: X-Auth-Token
Content-Type: application/json
Date: Mon, 04 Aug 2014 21:29:31 GMT
Transfer-Encoding: chunked

{"access": {"token": {"expires": "2014-08-05T21:29:31Z", ....}}}

Insofar as I can tell the outputs are the same except for some trivial changes in time stamps.  So what is supposed to be the difference between going through port 5000 and going through port 35357 ?  Obviously, there must be a difference or else 1) you wouldn't have brought it to my attention and 2) the programmer that created the API wouldn't have gone to the trouble of using two ports when one would do.

Many thanks,



Jeff





On Fri, Aug 1, 2014 at 5:27 PM, Fischer, Matt <matthew.fischer at twcable.com> wrote:
The keystone client does indeed hide failures from you and wrap them, which makes it annoying to debug, see https://bugs.launchpad.net/python-keystoneclient/+bug/1210625. If you do a —debug however you can see the exact call you are attempting and how to repro it with curl. To get a token, you need to POST, I figure the default action for curl is a GET which may be why you are having issues with your curl command.

Here is a curl request to get a token.

keystone --debug token-get
DEBUG:keystoneclient.session:REQ: curl -i -X POST http://example.com:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "myPassword"}}}'


More debugging hints:

If you still have problems the server-side logs are generally way more useful. You can enable debug in the config file and then run keystone by hand (after stopping it) by doing /usr/bin/keystone-all. That will generally provide better feedback.

Also :35357 is the service endpoint for which I usually use a service token, is there a reason you're using that and not the standard :5000?



From: Jeff Silverman <jeff at sweetlabs.com>
Date: Friday, August 1, 2014 3:35 PM
To: "openstack-operators at lists.openstack.org" <openstack-operators at lists.openstack.org>
Subject: [Openstack-operators] keystone is throwing Authorization Failed: 'module' object is not callable errors

I did something to keystone, I'm not sure what.

root at controller1-prod.controller1-prod:~# keystone role-list
Authorization Failed: 'module' object is not callable
root at controller1-prod.controller1-prod:~#
root at controller1-prod.controller1-prod:~# keystone role-get admin
Authorization Failed: 'module' object is not callable
root at controller1-prod.controller1-prod:~#


I have envars OS_USERNAME, OS_PASSWORD, OS_TENANT defined.  OS_AUTH_URL has a URL:
root at controller1-prod.controller1-prod:~# curl -i http://controller1-prod.sea.opencandy.com:35357/v2.0
HTTP/1.1 200 OK
Vary: X-Auth-Token
Content-Type: application/json
Date: Fri, 01 Aug 2014 21:10:47 GMT
Transfer-Encoding: chunked

{"version": {"status": "stable", "updated": "2012-10-13T17:42:56Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}, {"base": "application/xml", "type": "application/vnd.openstack.identity-v2.0+xml"}], "id": "v2.0", "links": [{"href": "http://controller1-prod.sea.opencandy.com:35357/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/api/openstack-identity-service/2.0/content/", "type": "text/html", "rel": "describedby"}, {"href": "http://docs.openstack.org/api/openstack-identity-service/2.0/identity-dev-guide-2.0.pdf", "type": "application/pdf", "rel": "describedby"}]}}root at controller1-prod.controller1-prod:~#


I have been poking at keystone with pdb to try find the point where the exception is raised, with little success.  Maybe I am incompetent as a python programmer.

I have discovered that keystoneclient does a call to the identity server to get a token - I think.  I tried to simulate the call using curl.


root at controller1-prod.controller1-prod:~# curl -i http://controller1-prod.sea.opencandy.com:35357/v2.0/tokens




HTTP/1.1 404 Not Found
Vary: X-Auth-Token
Content-Type: application/json
Date: Fri, 01 Aug 2014 20:26:00 GMT
Transfer-Encoding: chunked

{"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}

One of the things I find frustrating is the code assumes that any error is an authorization problem, which means that any bug is handled and doesn't percolate up the stack.  There seems to be no way to get the debugger to halt on a handled exception.  In client.py, there is
        except Exception as e:
            raise exceptions.AuthorizationFailure("Authorization Failed: "
which makes debugging a challenge..

I think that the exception is in the call to a.get_auth_ref(self.session).  I think that the problem is that a, a Password object, is not callable.

(Pdb) print callable(a)
False
(Pdb)
(Pdb) list
168                                       token=token,
169                                       trust_id=trust_id,
170                                       tenant_id=project_id or tenant_id,
171                                       tenant_name=project_name or tenant_name)
172
173  ->           return a.get_auth_ref(self.session)
174         except (exceptions.AuthorizationFailure, exceptions.Unauthorized):
175             _logger.debug("Authorization Failed.")
176             raise
177         except exceptions.EndpointNotFound:
178             msg = 'There was no suitable authentication url for this request'


(Pdb) pp vars(a)
{'auth_ref': None,
 'auth_url': 'http://controller1-prod.sea.opencandy.com:35357/v2.0',
 'password': "XXXXXXXXXXX",
 'tenant_id': None,
 'tenant_name': 'admin',
 'token': None,
 'trust_id': None,
 'username': 'admin'}
(Pdb)

I instrumented the code to see if I could get a better handle on the exception getting thrown:

(Pdb) list 165,184
165             a = v2_auth.Auth._factory(auth_url,
166                                       username=username,
167                                       password=password,
168                                       token=token,
169                                       trust_id=trust_id,
170                                       tenant_id=project_id or tenant_id,
171                                       tenant_name=project_name or tenant_name)
172
173             try:
174                 return a.get_auth_ref(self.session)
175             except Exception as e:
176                 print "Hit an exception %s" % e
177                 pdb.set_trace()
178  ->               raise
179         except (exceptions.AuthorizationFailure, exceptions.Unauthorized):
180             _logger.debug("Authorization Failed.")
181             raise
182         except exceptions.EndpointNotFound:
183             msg = 'There was no suitable authentication url for this request'
184             raise exceptions.AuthorizationFailure(msg)

(Pdb) c
Hit an exception 'module' object is not callable
> /usr/lib/python2.6/site-packages/keystoneclient/v2_0/client.py(178)get_raw_token_from_identity_service()
-> raise


Not sure what to do next.


Jeff



--
Jeff Silverman
Systems Engineer
(253) 459-2318<tel:%28253%29%20459-2318> (c)
[https://dl.dropboxusercontent.com/u/16943296/SweetLabs-Signatures/New_2014/signature-logo.png]

________________________________
This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.



--
Jeff Silverman
Systems Engineer
(253) 459-2318<tel:%28253%29%20459-2318> (c)
[https://dl.dropboxusercontent.com/u/16943296/SweetLabs-Signatures/New_2014/signature-logo.png]
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators




--
Jeff Silverman
Systems Engineer
(253) 459-2318<tel:%28253%29%20459-2318> (c)
[https://dl.dropboxusercontent.com/u/16943296/SweetLabs-Signatures/New_2014/signature-logo.png]



--
Jeff Silverman
Systems Engineer
(253) 459-2318 (c)
[https://dl.dropboxusercontent.com/u/16943296/SweetLabs-Signatures/New_2014/signature-logo.png]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140804/d83a3e65/attachment.html>


More information about the OpenStack-operators mailing list