<div dir="ltr">Hi all,<div> I am slowly putting together my havana openstack deployment in a home lab, but I seem to have hit an issue with glance with the keystone authentication.</div><div><br></div><div>I can set my paste_deploy flavor to an empty string and glance stores and lists images just fine.</div>
<div><br></div><div> When enabling keystone auth I get the following (also happens with my test user and tenant)</div><div><div><br></div><div><div>openstack@admin:~$ glance --os-username glance --os-password glance_password --os-tenant-name services image-list</div>
<div>Request returned failure status.</div><div>Invalid OpenStack Identity credentials.</div></div><div><br></div><div>Relevant logs below:</div><div><div>2014-04-07 17:10:46.866 15375 DEBUG keystoneclient.middleware.auth_token [-] Authenticating user token __call__ /usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py:526</div>
<div>2014-04-07 17:10:46.867 15375 DEBUG keystoneclient.middleware.auth_token [-] Removing headers from request environment: X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role _remove_auth_headers /usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py:585</div>
<div>2014-04-07 17:10:46.918 15375 DEBUG keystoneclient.middleware.auth_token [-] Token expired a 2014-04-07T08:10:46Z _confirm_token_not_expired /usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py:1024</div>
<div>2014-04-07 17:10:46.919 15375 DEBUG keystoneclient.middleware.auth_token [-] Token validation failure. _validate_user_token /usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py:790</div><div>2014-04-07 17:10:46.919 15375 TRACE keystoneclient.middleware.auth_token Traceback (most recent call last):</div>
<div>2014-04-07 17:10:46.919 15375 TRACE keystoneclient.middleware.auth_token File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 782, in _validate_user_token</div><div>2014-04-07 17:10:46.919 15375 TRACE keystoneclient.middleware.auth_token expires = self._confirm_token_not_expired(data)</div>
<div>2014-04-07 17:10:46.919 15375 TRACE keystoneclient.middleware.auth_token File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 1025, in _confirm_token_not_expired</div><div>
2014-04-07 17:10:46.919 15375 TRACE keystoneclient.middleware.auth_token raise InvalidUserToken('Token authorization failed')</div><div>2014-04-07 17:10:46.919 15375 TRACE keystoneclient.middleware.auth_token InvalidUserToken: Token authorization failed</div>
<div>2014-04-07 17:10:46.919 15375 TRACE keystoneclient.middleware.auth_token </div><div>2014-04-07 17:10:46.919 15375 DEBUG keystoneclient.middleware.auth_token [-] Marking token <Token) as unauthorized in memcache _cache_store_invalid /usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py:1043</div>
<div>2014-04-07 17:10:46.920 15375 WARNING keystoneclient.middleware.auth_token [-] Authorization failed for token <token></div><div>2014-04-07 17:10:46.920 15375 INFO keystoneclient.middleware.auth_token [-] Invalid user token - deferring reject downstream</div>
<div>^C</div><div>root@api01:~# date --utc</div><div>Mon Apr 7 07:10:54 UTC 2014</div></div></div><div><br></div><div>Time on all hosts is synced to the same NTP source, my timezone an offset of UTC+10. The token above has just under an hour for its lifetime.</div>
<div><br></div><div>I can use these credentials directly with keystone</div><div><div>openstack@admin:~$ keystone --os-username glance --os-password glance_password --os-tenant-name services tenant-list</div><div>+----------------------------------+----------+---------+</div>
<div>| id | name | enabled |</div><div>+----------------------------------+----------+---------+</div><div>| e30936cb81524d8a8fa1a51991fd3acd | admin | True |</div><div>| 97b98e459c214db58dd6bfd367a53d8a | services | True |</div>
<div>| a8b1b92c004643999aad469ce7123bf4 | test | True |</div><div>+----------------------------------+----------+---------+</div></div><div><br></div><div>get-token works fine also.</div><div><br></div><div>Here is the relevant parts to my glance-api.conf</div>
<div><div>[keystone_authtoken]</div><div>auth_host = api-internal.openstack.home</div><div>auth_port = 35357</div><div>auth_protocol = http</div><div>admin_tenant_name = services</div><div>admin_user = glance</div><div>admin_password = glance_password</div>
<div>auth_uri=<a href="http://api-internal.openstack.home:5000/">http://api-internal.openstack.home:5000/</a></div><div><br></div><div>[paste_deploy]</div><div>flavor=keystone</div></div><div><br></div><div>and glance-api-paste.ini</div>
<div><div>[pipeline:glance-api-keystone]</div><div>pipeline = versionnegotiation authtoken context rootapp</div></div><div><br></div><div><div>[filter:authtoken]</div><div>paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory</div>
<div>delay_auth_decision = true</div><div>auth_host=api.openstack.home</div><div>admin_user=glance</div><div>admin_tenant_name=services</div><div>admin_password=glance_password</div></div><div><br></div><div>I added in the auth_* and admin_* directives after following the openstack documentation</div>
<div><a href="http://docs.openstack.org/havana/install-guide/install/apt/content/glance-install.html">http://docs.openstack.org/havana/install-guide/install/apt/content/glance-install.html</a><br></div><div><br></div><div>
below are all the runtime values gleaned from the log files:</div><div><div>admin_password = ****</div><div>admin_role = admin</div><div>admin_tenant_name = ****</div><div>
admin_user = ****</div><div>allow_additional_image_properties = True</div><div>allow_anonymous_access = False</div><div>api_limit_max = 1000</div><div>auth_region = None</div>
<div>auth_strategy = noauth</div><div>auth_url = None</div><div>backlog = 4096</div><div>bind_host = 0.0.0.0</div><div>bind_port = 9292</div>
<div>ca_file = None</div><div>cert_file = None</div><div>cinder_api_insecure = False</div><div>cinder_ca_certificates_file = None</div><div>cinder_catalog_info = volume:cinder:publicURL</div>
<div>cinder_endpoint_template = None</div><div>cinder_http_retries = 3</div><div>cleanup_scrubber = False</div><div>cleanup_scrubber_time = 86400</div><div>config_dir = None</div>
<div>config_file = ['/etc/glance/glance-api.conf']</div><div>container_formats = ['ami', 'ari', 'aki', 'bare', 'ovf']</div><div>data_api = glance.db.sqlalchemy.api</div>
<div>db_auto_create = False</div><div>debug = True</div><div>default_log_levels = ['amqplib=WARN', 'sqlalchemy=WARN', 'boto=WARN', 'suds=INFO', 'keystone=INFO', 'eventlet.wsgi.server=WARN']</div>
<div>default_store = rbd</div><div>delayed_delete = False</div><div>disable_process_locking = False</div><div>disk_formats = ['ami', 'ari', 'aki', 'vhd', 'vmdk', 'raw', 'qcow2', 'vdi', 'iso']</div>
<div>enable_v1_api = True</div><div>enable_v2_api = True</div><div>eventlet_hub = poll</div><div>fatal_deprecations = False</div><div>filesystem_store_datadir = /var/lib/glance/images/</div>
<div>filesystem_store_metadata_file = None</div><div>image_size_cap = 1099511627776</div><div>instance_format = [instance: %(uuid)s] </div><div>instance_uuid_format = [instance: %(uuid)s] </div>
<div>key_file = None</div><div>known_stores = ['glance.store.filesystem.Store', 'glance.store.http.Store', 'glance.store.rbd.Store', 'glance.store.s3.Store', 'glance.store.swift.Store', 'glance.store.sheepdog.Store', 'glance.store.cinder.Store']</div>
<div>limit_param_default = 25</div><div>lock_path = None</div><div>log_config = None</div><div>log_date_format = %Y-%m-%d %H:%M:%S</div><div>log_dir = /var/log/glance</div>
<div>log_file = /var/log/glance/api.log</div><div>log_format = None</div><div>logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user)s %(tenant)s] %(instance)s%(message)s</div>
<div>logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d</div><div>%(instance)s%(message)s</div><div>logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d TRACE %(name)s %(instance)s</div>
<div>memcached_servers = None</div><div>metadata_encryption_key = ****</div><div>notifier_strategy = rabbit</div><div>os_region_name = None</div><div>owner_is_tenant = True</div>
<div>policy_default_rule = default</div><div>policy_file = policy.json</div><div>property_protection_file = None</div><div>publish_errors = False</div><div>pydev_worker_debug_host = None</div>
<div>pydev_worker_debug_port = 5678</div><div>rabbit_durable_queues = False</div><div>rabbit_host = slb01.openstack.home</div><div>rabbit_max_retries = 0</div><div>rabbit_notification_exchange = glance</div>
<div>rabbit_notification_topic = notifications</div><div>rabbit_password = ******</div><div>rabbit_port = 5672</div><div>rabbit_retry_backoff = 2</div><div>rabbit_retry_max_backoff = 30</div>
<div>rabbit_use_ssl = False</div><div>rabbit_userid = rabbit</div><div>rabbit_virtual_host = /</div><div>rbd_store_ceph_conf = /etc/ceph/ceph.conf</div><div>rbd_store_chunk_size = 8</div>
<div>rbd_store_pool = images</div><div>rbd_store_user = glance</div><div>registry_client_ca_file = None</div><div>registry_client_cert_file = None</div><div>registry_client_insecure = False</div>
<div>registry_client_key_file = None</div><div>registry_client_protocol = http</div><div>registry_client_timeout = 600</div><div>registry_host = localhost</div><div>registry_port = 9191</div>
<div>s3_store_access_key = ************************</div><div>s3_store_bucket = <lowercased 20-char aws access key>glance</div><div>s3_store_bucket_url_format = subdomain</div><div>s3_store_create_bucket_on_put = False</div>
<div>s3_store_host = <a href="http://127.0.0.1:8080/v1.0/">127.0.0.1:8080/v1.0/</a></div><div>s3_store_object_buffer_dir = None</div><div>s3_store_secret_key = ************************</div>
<div>scrub_time = 43200</div><div>scrubber_datadir = /var/lib/glance/scrubber</div><div>send_identity_headers = False</div><div>sheepdog_store_address = localhost</div><div>
sheepdog_store_chunk_size = 64</div><div>sheepdog_store_port = 7000</div><div>show_image_direct_url = True</div><div>show_multiple_locations = False</div><div>sql_connection = ********************************************************</div>
<div>sql_idle_timeout = 3600</div><div>sql_max_retries = 60</div><div>sql_retry_interval = 1</div><div>sqlalchemy_debug = False</div><div>swift_enable_snet = False</div>
<div>swift_store_admin_tenants = []</div><div>swift_store_auth_address = <a href="http://127.0.0.1:5000/v2.0/">127.0.0.1:5000/v2.0/</a></div><div>swift_store_auth_insecure = False</div><div>swift_store_auth_version = 2</div>
<div>swift_store_container = glance</div><div>swift_store_create_container_on_put = False</div><div>swift_store_endpoint_type = publicURL</div><div>swift_store_key = ********************************</div>
<div>swift_store_large_object_chunk_size = 200</div><div>swift_store_large_object_size = 5120</div><div>swift_store_multi_tenant = False</div><div>swift_store_region = None</div><div>swift_store_service_type = object-store</div>
<div>swift_store_ssl_compression = True</div><div>swift_store_user = *********</div><div>syslog_log_facility = LOG_USER</div><div>tcp_keepidle = 600</div><div>use_stderr = True</div>
<div>use_syslog = False</div><div>use_tpool = False</div><div>use_user_token = True</div><div>user_storage_quota = 0</div><div>verbose = True</div>
<div>workers = 1</div><div>paste_deploy.config_file = None</div><div>paste_deploy.flavor = keystone</div><div>keystone_authtoken.admin_password = ***************</div><div>keystone_authtoken.admin_tenant_name = services</div>
<div>keystone_authtoken.admin_token = ****</div><div>keystone_authtoken.admin_user = glance</div><div>keystone_authtoken.auth_admin_prefix = </div><div>keystone_authtoken.auth_host = api-internal.openstack.home</div><div>
keystone_authtoken.auth_port = 35357</div><div>keystone_authtoken.auth_protocol = http</div><div>keystone_authtoken.auth_uri = <a href="http://api-internal.openstack.home:5000/">http://api-internal.openstack.home:5000/</a></div>
<div>keystone_authtoken.auth_version = None</div><div>keystone_authtoken.cache = None</div><div>keystone_authtoken.certfile = None</div><div>keystone_authtoken.delay_auth_decision = False</div><div>keystone_authtoken.http_connect_timeout = None</div>
<div>keystone_authtoken.http_handler = None</div><div>keystone_authtoken.keyfile = None</div><div>keystone_authtoken.memcache_secret_key = ****</div><div>keystone_authtoken.memcache_security_strategy = None</div><div>
keystone_authtoken.memcached_servers = None</div><div>keystone_authtoken.revocation_cache_time = 1</div><div>keystone_authtoken.signing_dir = None</div></div><div><br></div><div>I hope there is a clue in there somewhere.</div>
<div><br></div><div>Cheers</div><div><br></div><div>Adam</div></div>