<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Hello Folks.</div>
<div><br>
</div>
<div>I have a problem after grizzly->havana migration where i’m unable to rescue myself.</div>
<div>When I open the "Admin -> Resource-Usage View i get no results – only a red error box with the message "<strong>Error: </strong>Unable to retrieve tenant list.“.</div>
<div><br>
</div>
<div>Horizon log:</div>
<div>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] Creating a new keystoneclient connection to http://$controller:35357/v2.0.</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET http://$controller:35357/v2.0/tenants?marker=tenant_marker&limit=21 -H "User-Agent: python-keystoneclient" -H "Forwarded: for=131.234.5.178;by=python-keystoneclient"
-H "X-Auth-Token: 82[…]f46"</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET http://$controller:35357/v2.0/tenants?marker=tenant_marker&limit=21 -H "User-Agent: python-keystoneclient" -H "Forwarded: for=131.234.5.178;by=python-keystoneclient"
-H "X-Auth-Token: 82[…]46"</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] INFO:urllib3.connectionpool:Starting new HTTP connection (1): $controller</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] DEBUG:urllib3.connectionpool:"GET /v2.0/tenants?marker=tenant_marker&limit=21 HTTP/1.1" 400 88</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] RESP: [400] CaseInsensitiveDict({'date': 'Thu, 31 Oct 2013 11:39:47 GMT', 'vary': 'X-Auth-Token', 'content-length': '88', 'content-type': 'application/json'})</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] RESP BODY: {"error": {"message": "Marker could not be found", "code": 400, "title": "Bad Request"}}</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] </p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] RESP: [400] CaseInsensitiveDict({'date': 'Thu, 31 Oct 2013 11:39:47 GMT', 'vary': 'X-Auth-Token', 'content-length': '88', 'content-type': 'application/json'})</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] RESP BODY: {"error": {"message": "Marker could not be found", "code": 400, "title": "Bad Request"}}</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] </p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">[Thu Oct 31 11:39:44 2013] [error] Recoverable error: Marker could not be found (HTTP 400)</p>
</div>
<div><br>
</div>
<div>Keystone Log:</div>
<div><span style="font-family: Menlo; font-size: 11px;">2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Matched GET /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100</span></div>
<div>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Route path: '{path_info:.*}', defaults: {'controller': <keystone.contrib.ec2.routers.Ec2Extension object at 0x4156f10>} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Match dict: {'controller': <keystone.contrib.ec2.routers.Ec2Extension object at 0x4156f10>, 'path_info': '/tenants'} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Matched GET /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Route path: '{path_info:.*}', defaults: {'controller': <keystone.contrib.s3.core.S3Extension object at 0x4156cd0>} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Match dict: {'controller': <keystone.contrib.s3.core.S3Extension object at 0x4156cd0>, 'path_info': '/tenants'} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Matched GET /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Route path: '{path_info:.*}', defaults: {'controller': <keystone.contrib.admin_crud.core.CrudExtension object at 0x41517d0>} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Match dict: {'controller': <keystone.contrib.admin_crud.core.CrudExtension object at 0x41517d0>, 'path_info': '/tenants'} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Matched GET /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Route path: '{path_info:.*}', defaults: {'controller': <keystone.common.wsgi.ComposingRouter object at 0x4151e50>} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Match dict: {'controller': <keystone.common.wsgi.ComposingRouter object at 0x4151e50>, 'path_info': '/tenants'} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Matched GET /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.357 17187 DEBUG routes.middleware [-] Route path: '/tenants', defaults: {'action': u'get_all_projects', 'controller': <keystone.identity.controllers.Tenant object at 0x4151ed0>}
__call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.357 17187 DEBUG routes.middleware [-] Match dict: {'action': u'get_all_projects', 'controller': <keystone.identity.controllers.Tenant object at 0x4151ed0>} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.357 17187 DEBUG dogpile.core.dogpile [-] NeedRegenerationException _enter /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:94</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.358 17187 DEBUG dogpile.core.dogpile [-] no value, waiting for create lock _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:127</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.358 17187 DEBUG dogpile.core.dogpile [-] value creation lock <dogpile.cache.region._LockWrapper object at 0x4fc77d0> acquired _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:131</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.358 17187 DEBUG dogpile.core.dogpile [-] Calling creation function _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:148</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.362 17187 DEBUG dogpile.core.dogpile [-] Released creation lock _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:154</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.362 17187 DEBUG dogpile.core.dogpile [-] NeedRegenerationException _enter /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:94</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.362 17187 DEBUG dogpile.core.dogpile [-] no value, waiting for create lock _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:127</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.362 17187 DEBUG dogpile.core.dogpile [-] value creation lock <dogpile.cache.region._LockWrapper object at 0x4fc7bd0> acquired _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:131</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.363 17187 DEBUG dogpile.core.dogpile [-] Calling creation function _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:148</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.365 17187 DEBUG dogpile.core.dogpile [-] Released creation lock _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:154</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.366 17187 DEBUG dogpile.core.dogpile [-] NeedRegenerationException _enter /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:94</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.366 17187 DEBUG dogpile.core.dogpile [-] no value, waiting for create lock _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:127</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.366 17187 DEBUG dogpile.core.dogpile [-] value creation lock <dogpile.cache.region._LockWrapper object at 0x4fc7bd0> acquired _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:131</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.366 17187 DEBUG dogpile.core.dogpile [-] Calling creation function _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:148</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.369 17187 DEBUG dogpile.core.dogpile [-] Released creation lock _enter_create /usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:154</p>
</div>
<div>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.373 17187 WARNING keystone.common.wsgi [-] Marker could not be found</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">2013-10-31 12:39:47.373 17187 INFO access [-] 131.234.5.178 - - [31/Oct/2013:11:39:47 +0000] "GET http://$controller:35357/v2.0/tenants?marker=tenant_marker&limit=21 HTTP/1.0" 400 88</p>
</div>
<div><br>
</div>
<div>I have the following (ubuntu packaged) version of the possible involved components of openstack:</div>
<div><br>
</div>
<div>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">root@$controller:/# dpkg -l | grep keystone</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">ii <span style="color: #c33720">
<b>keystone</b></span> 1:2013.2-0ubuntu1~cloud0 OpenStack identity service - Daemons</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">ii python-<span style="color: #c33720"><b>keystone</b></span> 1:2013.2-0ubuntu1~cloud0 OpenStack identity service - Python library</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">ii python-<span style="color: #c33720"><b>keystone</b></span>client 1:0.3.2-0ubuntu1~cloud0 Client library for OpenStack Identity API</p>
</div>
<div><br>
</div>
<div>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">root@$dashboard:/# dpkg -l | grep openstack</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">ii <span style="color: #c33720">
<b>openstack</b></span>-dashboard 1:2013.2~rc3-0ubuntu1~cloud0 django web interface to Openstack</p>
<p style="margin: 0px; font-size: 11px; font-family: Menlo;">ii python-<span style="color: #c33720"><b>openstack</b></span>-auth 1.1.1-0ubuntu1~cloud0 A django authentication backend for Openstack</p>
</div>
<div><br>
</div>
<div>What could get wrong?</div>
<div>
<div><br>
</div>
<div>Thanks!</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>Sebastian</div>
<div><br>
</div>
<div>--</div>
<div>Sebastian Porombka, M.Sc. </div>
<div>Zentrum für Informations- und Medientechnologien (IMT)</div>
<div>Universität Paderborn</div>
<div><br>
</div>
<div>E-Mail: porombka@uni-paderborn.de</div>
<div>Tel.: 05251/60-5999</div>
<div>Fax: 05251/60-48-5999</div>
<div>Raum: N5.314 </div>
<div><br>
</div>
<div>--------------------------------------------</div>
<div>Q: Why is this email five sentences or less?</div>
<div>A: http://five.sentenc.es</div>
<div><br>
</div>
<div>Please consider the environment before printing this email.</div>
</div>
</body>
</html>