[Openstack-operators] Horizon failing in self.auth_url of neutronclient.

Miguel A Diaz Corchero miguelangel.diaz at externos.ciemat.es
Thu Mar 27 10:59:52 UTC 2014


Using command line, all is fine. The error only appears using horizon, 
when the neutron info is going to be loaded. I've seen in the logs how 
horizon starts to reach the nova details sucessfully (but not the 
neutron one).


[Thu Mar 27 10:18:37 2014] [error] REQ: curl -i 
http://controller:8774/v2/demo/limits -X GET -H "X-Auth-Project-Id: 
demo" -H "User-Agent: python-novaclient" -H "Accept: application/json" 
-H "X-Auth-Token: *0de5eed32b09c3d7a357cd3391f13051*"
[Thu Mar 27 10:18:37 2014] [error]
[Thu Mar 27 10:18:37 2014] [error] INFO:urllib3.connectionpool:Starting 
new HTTP connection (1): controller
[Thu Mar 27 10:18:37 2014] [error] DEBUG:urllib3.connectionpool:"GET 
/v2/demo/limits HTTP/1.1" 200 437
[Thu Mar 27 10:18:37 2014] [error] RESP: [200] {'date': 'Thu, 27 Mar 
2014 10:18:37 GMT', 'content-length': '437', 'content-type': 
'application/json', 'x-compute-request-id': 
'req-d80a8b6a-3a77-40c0-9e12-ddabf8871447'}
[Thu Mar 27 10:18:37 2014] [error] RESP BODY: {"limits": {"rate": [], 
"absolute": {"maxServerMeta": 128, "maxPersonality": 5, "maxImageMeta": 
128, "maxPersonalitySize": 10240, "maxSecurityGroupRules": 20, 
"maxTotalKeypairs": 100, "totalRAMUsed": 18432, "totalInstancesUsed": 9, 
"maxSecurityGroups": 10, "totalFloatingIpsUsed": 0, "maxTotalCores": 
100, "totalSecurityGroupsUsed": 0, "maxTotalFloatingIps": 10, 
"maxTotalInstances": 100, "totalCoresUsed": 9, "maxTotalRAMSize": 51200}}}
[Thu Mar 27 10:18:37 2014] [error]
[Thu Mar 27 10:18:37 2014] [error] Internal Server Error: 
/dashboard/project/
[Thu Mar 27 10:18:37 2014] [error] Traceback (most recent call last):
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/django/core/handlers/base.py", line 
111, in get_response
[Thu Mar 27 10:18:37 2014] [error]     response = callback(request, 
*callback_args, **callback_kwargs)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/decorators.py", line 38, in dec
[Thu Mar 27 10:18:37 2014] [error]     return view_func(request, *args, 
**kwargs)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/decorators.py", line 54, in dec
[Thu Mar 27 10:18:37 2014] [error]     return view_func(request, *args, 
**kwargs)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/decorators.py", line 38, in dec
[Thu Mar 27 10:18:37 2014] [error]     return view_func(request, *args, 
**kwargs)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/django/views/generic/base.py", line 
48, in view
[Thu Mar 27 10:18:37 2014] [error]     return self.dispatch(request, 
*args, **kwargs)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/django/views/generic/base.py", line 
69, in dispatch
[Thu Mar 27 10:18:37 2014] [error]     return handler(request, *args, 
**kwargs)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 155, in get
[Thu Mar 27 10:18:37 2014] [error]     handled = self.construct_tables()
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 146, in 
construct_tables
[Thu Mar 27 10:18:37 2014] [error]     handled = self.handle_table(table)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 118, in 
handle_table
[Thu Mar 27 10:18:37 2014] [error]     data = self._get_data_dict()
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 182, in 
_get_data_dict
[Thu Mar 27 10:18:37 2014] [error]     self._data = 
{self.table_class._meta.name: self.get_data()}
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/dashboards/project/overview/views.py", 
line 56, in get_data
[Thu Mar 27 10:18:37 2014] [error]     super(ProjectOverview, 
self).get_data()
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/usage/views.py", 
line 29, in get_data
[Thu Mar 27 10:18:37 2014] [error]     self.usage.get_limits()
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/usage/base.py", 
line 178, in get_limits
[Thu Mar 27 10:18:37 2014] [error] self.get_neutron_limits()
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/usage/base.py", 
line 147, in get_neutron_limits
[Thu Mar 27 10:18:37 2014] [error] 
api.neutron.is_security_group_extension_supported(self.request)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/neutron.py", 
line 761, in is_security_group_extension_supported
[Thu Mar 27 10:18:37 2014] [error]     return 
is_extension_supported(request, 'security-group')
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/utils/memoized.py", line 33, 
in __call__
[Thu Mar 27 10:18:37 2014] [error]     value = self.func(*args)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/neutron.py", 
line 741, in is_extension_supported
[Thu Mar 27 10:18:37 2014] [error]     extensions = list_extensions(request)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/horizon/utils/memoized.py", line 33, 
in __call__
[Thu Mar 27 10:18:37 2014] [error]     value = self.func(*args)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/neutron.py", 
line 732, in list_extensions
[Thu Mar 27 10:18:37 2014] [error]     extensions_list = 
neutronclient(request).list_extensions()
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 
108, in with_params
[Thu Mar 27 10:18:37 2014] [error]     ret = self.function(instance, 
*args, **kwargs)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 
286, in list_extensions
[Thu Mar 27 10:18:37 2014] [error]     return 
self.get(self.extensions_path, params=_params)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 
1183, in get
[Thu Mar 27 10:18:37 2014] [error]     headers=headers, params=params)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 
1168, in retry_request
[Thu Mar 27 10:18:37 2014] [error]     headers=headers, params=params)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 
1103, in do_request
[Thu Mar 27 10:18:37 2014] [error]     resp, replybody = 
self.httpclient.do_request(action, method, body=body)
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/neutronclient/client.py", line 190, in 
do_request
[Thu Mar 27 10:18:37 2014] [error]     self.authenticate()
[Thu Mar 27 10:18:37 2014] [error]   File 
"/usr/lib/python2.6/site-packages/neutronclient/client.py", line 228, in 
authenticate
*[Thu Mar 27 10:18:37 2014] [error]     token_url = self.auth_url + 
"/tokens"**
**[Thu Mar 27 10:18:37 2014] [error] TypeError: unsupported operand 
type(s) for +: 'NoneType' and 'str'*
[Thu Mar 27 10:36:37 2014] [error]

Then in neutron server logs appears how the same token used by nova is 
rejected:

2014-03-27 11:18:37.252 14131 ERROR keystoneclient.middleware.auth_token 
[-] Unable to get version info from keystone: 404
2014-03-27 11:18:37.252 14131 DEBUG keystoneclient.middleware.auth_token 
[-] Token validation failure. _validate_user_token 
/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:820
2014-03-27 11:18:37.252 14131 TRACE keystoneclient.middleware.auth_token 
Traceback (most recent call last):
2014-03-27 11:18:37.252 14131 TRACE 
keystoneclient.middleware.auth_token   File 
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", 
line 811, in _validate_user_token
2014-03-27 11:18:37.252 14131 TRACE 
keystoneclient.middleware.auth_token     data = 
self.verify_uuid_token(user_token, retry)
2014-03-27 11:18:37.252 14131 TRACE 
keystoneclient.middleware.auth_token   File 
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", 
line 1088, in verify_uuid_token
2014-03-27 11:18:37.252 14131 TRACE 
keystoneclient.middleware.auth_token     self.auth_version = 
self._choose_api_version()
2014-03-27 11:18:37.252 14131 TRACE 
keystoneclient.middleware.auth_token   File 
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", 
line 509, in _choose_api_version
2014-03-27 11:18:37.252 14131 TRACE keystoneclient.middleware.auth_token 
versions_supported_by_server = self._get_supported_versions()
2014-03-27 11:18:37.252 14131 TRACE 
keystoneclient.middleware.auth_token   File 
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", 
line 536, in _get_supported_versions
2014-03-27 11:18:37.252 14131 TRACE 
keystoneclient.middleware.auth_token     raise ServiceError('Unable to 
get version info from keystone')
2014-03-27 11:18:37.252 14131 TRACE keystoneclient.middleware.auth_token 
ServiceError: Unable to get version info from keystone
2014-03-27 11:18:37.253 14131 DEBUG keystoneclient.middleware.auth_token 
[-] Marking token 0de5eed32b09c3d7a357cd3391f13051 as unauthorized in 
memcache _cache_store_invalid 
/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py:1068
2014-03-27 11:18:37.253 14131 WARNING 
keystoneclient.middleware.auth_token [-] Authorization failed for token 
*0de5eed32b09c3d7a357cd3391f13051*

Any idea?

Regards
Miguel.

El 26/03/14 19:18, matt escribió:
> hop onto the box with horizon running on it and try to authenticate to 
> nova CLI with the --debug flag set.  see what that gives you.
>
>
> On Wed, Mar 26, 2014 at 2:09 PM, Miguel Angel Diaz Corchero 
> <miguelangel.diaz at externos.ciemat.es 
> <mailto:miguelangel.diaz at externos.ciemat.es>> wrote:
>
>     Hi,
>
>     Horizon (of Havana) is showing problems when accessing:
>
>     The error message is:
>
>     unsupported operand type(s) for +: 'NoneType' and 'str'
>
>
>     and is produced in:
>
>     |/usr/lib/python2.6/site-packages/neutronclient/client.py| in
>     |authenticate|
>
>     token_url = self.auth_url + "/tokens"
>
>     I've applied the patch https://review.openstack.org/#/c/52954/
>     but the self.auth_url of the code above is still None
>
>     Commands such as nova net-list and nova floating-ip-list work fine.
>
>     Any idea?
>
>     Thanks!
>     Miguel.
>
>
>     -- 
>     /Miguel Angel Díaz Corchero/
>     /*System Administrator / Researcher*/
>     /c/ Sola nº 1; 10200 TRUJILLO, SPAIN/
>     /Tel: +34 927 65 93 17 Fax: +34 927 32 32 37
>     <tel:%2B34%20927%2032%2032%2037>/
>
>     CETA-Ciemat logo <http://www.ceta-ciemat.es/>
>
>     ---------------------------- Confidencialidad: Este mensaje y sus
>     ficheros adjuntos se dirige exclusivamente a su destinatario y
>     puede contener información privilegiada o confidencial. Si no es
>     vd. el destinatario indicado, queda notificado de que la
>     utilización, divulgación y/o copia sin autorización está prohibida
>     en virtud de la legislación vigente. Si ha recibido este mensaje
>     por error, le rogamos que nos lo comunique inmediatamente
>     respondiendo al mensaje y proceda a su destrucción. Disclaimer:
>     This message and its attached files is intended exclusively for
>     its recipients and may contain confidential information. If you
>     received this e-mail in error you are hereby notified that any
>     dissemination, copy or disclosure of this communication is
>     strictly prohibited and may be unlawful. In this case, please
>     notify us by a reply and delete this email and its contents
>     immediately. ----------------------------
>
>     _______________________________________________
>     OpenStack-operators mailing list
>     OpenStack-operators at lists.openstack.org
>     <mailto:OpenStack-operators at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140327/5170887b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 26213 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140327/5170887b/attachment-0001.png>


More information about the OpenStack-operators mailing list