[Openstack] glance image service: 500 Internal Server Error
Joerg Streckfuss
openstack at dirtyhack.org
Sat Nov 14 09:59:56 UTC 2015
Hi Tom,
I tested connecting to mysql via shell without problems:
<snip>
[root at controller ~]# mysql -h localhost -u glance -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 28
Server version: 5.5.44-MariaDB MariaDB Server
Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input
statement.
MariaDB [(none)]> use glance;
Database changed
MariaDB [glance]> show tables;
Empty set (0.00 sec)
<snap>
But as you can see there are no tables in the database glance. So I
tried to reinitialize it:
[root at controller ~]# su -s /bin/sh -c "glance-manage db_sync" glance
No handlers could be found for logger "oslo_config.cfg"
But this changed nothing. The glance database ist still empty. BTW. do
you know what the message "No handlers could be found for logger
"oslo_config.cfg"" mean? This also appears when initializing the
keystone database.
Any suggestions?
Am 13.11.2015 um 20:34 schrieb Tom Walsh:
> Joerg,
>
> The logs indicate that you have a problem connecting to MySQL. Verify
> that you are using the correct credentials to connect to the glance
> database. I am not sure if this a typo or just you modifying information
> from the configs to sanitize them, but you have:
>
> connection = mysql://glance:xxxxxx@controller/glance
>
> But the logs complain about:
>
> Access denied for user 'glance'@'localhost'
>
> Is your MySQL instance on the localhost?
>
> It might be possible that you are not using the correct credentials. Our
> Liberty setup uses a separate section for the database:
>
> [database]
> connection = mysql+pymysql://glance:password@192.168.254.1/glance
> <http://glance:password@192.168.254.1/glance>
> backend = sqlalchemy
>
> Double check your connectivity using the mysql CLI and passing in the
> same variables you are using in your connection string:
>
> mysql -h 192.168.254.1 -u glance -p
>
> See if that works. If it does, start working your way up from there as
> you know you have the right credentials as far as MySQL is concerned and
> start looking elsewhere.
>
> I suspect it might be the configuration of glance, but that's just a
> guess at this point.
>
> Also one thing that bit us was max_connections in MySQL. We had working
> details, but MySQL was preventing additional connections due to being
> out of connection slots. Just mentioning it that there can be other
> reasons MYSQL might reject a connection other than username/password.
>
> Good luck.
>
> Tom Walsh
> ExpressHosting.net
>
>
> On Fri, Nov 13, 2015 at 12:11 PM, Joerg Streckfuss
> <openstack at dirtyhack.org <mailto:openstack at dirtyhack.org>> wrote:
>
> Dear list,
>
> i tried to install the image service on centos7 from openstack
> liberty. When i try to create an image i got the folliwing error:
>
> <snip>
> # glance image-create --name "cirros" \
> --file cirros-0.3.4-x86_64-disk.img \
> --disk-format qcow2 --container-format bare \
> --visibility public --progress
> 500 Internal Server Error: The server has either erred or is
> incapable of performing the requested operation. (HTTP 500)
> <snap>
>
> glance-api.conf:
> <snip>
> [DEFAULT]
> connection = mysql://glance:xxxxxx@controller/glance
> notification_driver = noop
> verbose = True
>
> [keystone_authtoken]
> auth_uri = http://controller:5000
> auth_url = http://controller:35357
> auth_plugin = password
> project_domain_id = default
> user_domain_id = default
> project_name = service
> username = glance
> password = xxxxxx
>
> [paste_deploy]
> flavor = keystone
>
> [glance_store]
> default_store = file
> filesystem_store_datadir = /var/lib/glance/images/
> <snap>
>
>
> openstack-status gives me the folling output
>
> <snip>
> # openstack-status
> == Glance services ==
> openstack-glance-api: active
> openstack-glance-registry: active
> == Keystone service ==
> openstack-keystone: inactive (disabled on boot)
> == Support services ==
> mysqld: inactive (disabled on boot)
> dbus: active
> rabbitmq-server: active
> memcached: active
> == Keystone users ==
> /usr/lib/python2.7/site-packages/keystoneclient/shell.py:64:
> DeprecationWarning: The keystone CLI is deprecated in favor of
> python-openstackclient. For a Python library, continue using
> python-keystoneclient.
> 'python-keystoneclient.', DeprecationWarning)
> WARNING: unsupported identity-api-version 3, falling back to 2.0
> /usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145:
> DeprecationWarning: Constructing an instance of the
> keystoneclient.v2_0.client.Client class without a session is
> deprecated as of the 1.7.0 release and may be removed in the 2.0.0
> release.
> 'the 2.0.0 release.', DeprecationWarning)
> /usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147:
> DeprecationWarning: Using the 'tenant_name' argument is deprecated
> in version '1.7.0' and will be removed in version '2.0.0', please
> use the 'project_name' argument instead
> super(Client, self).__init__(**kwargs)
> /usr/lib/python2.7/site-packages/debtcollector/renames.py:43:
> DeprecationWarning: Using the 'tenant_id' argument is deprecated in
> version '1.7.0' and will be removed in version '2.0.0', please use
> the 'project_id' argument instead
> return f(*args, **kwargs)
> /usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:376:
> DeprecationWarning: Constructing an HTTPClient instance without
> using a session is deprecated as of the 1.7.0 release and may be
> removed in the 2.0.0 release.
> 'the 2.0.0 release.', DeprecationWarning)
> Authorization Failed: The resource could not be found. (HTTP 404)
> (Request-ID: req-99d108ad-edb3-4589-a7c1-85dc8e97f63a)
> == Glance images ==
> 500 Internal Server Error: The server has either erred or is
> incapable of performing the requested operation. (HTTP 500)
> <snap>
>
> in /var/log/glance/api.log i got the folling exception
>
> <snip>
> 2015-11-13 18:00:18.383 1458 INFO eventlet.wsgi.server
> [req-7e4bee11-b8d7-4dcf-a0a0-b2e98cf3593c
> 058f089c31e64370af9cb268c75da386 ff2149063f074c90bae7ea0f1dfcdfaf -
> - -] 10.11.12.230 - - [13/Nov/2015 18:00:18] "GET
> /v2/schemas/imag[45/89]
> 1.1" 200 4117 2.556846
> 2015-11-13 18:00:18.389 1458 INFO eventlet.wsgi.server
> [req-b0894879-3658-4bd7-ac23-73818a607628
> 058f089c31e64370af9cb268c75da386 ff2149063f074c90bae7ea0f1dfcdfaf -
> - -] 10.11.12.230 - - [13/Nov/2015 18:00:18] "GET /v2/schemas/image
> HTTP/
> 1.1" 200 4117 0.003295
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> [req-76502a62-8c24-4b0a-828e-a0bf07733023
> 058f089c31e64370af9cb268c75da386 ff2149063f074c90bae7ea0f1dfcdfaf -
> - -] Caught error: (_mysql_exceptions.OperationalError) (1045,
> "Access den
> ied for user 'glance'@'localhost' (using password: YES)")
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi Traceback
> (most recent call last):
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 879,
> in __call__
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi request,
> **action_args)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 907,
> in dispatch
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> method(*args, **kwargs)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 504,
> in wrapped
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> func(self, req, *args, **kwargs)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 66,
> in create
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> image_repo.add(image)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 94,
> in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi result =
> self.base.add(base_item)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/notifier.py", line 460, in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> super(ImageRepoProxy, self).add(image)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 94,
> in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi result =
> self.base.add(base_item)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 131,
> in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> super(ImageRepoProxy, self).add(image)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 94,
> in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi result =
> self.base.add(base_item)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line
> 114, in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> super(ImageRepoProxy, self).add(image)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 94,
> in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi result =
> self.base.add(base_item)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/location.py", line 63, in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi result =
> super(ImageRepoProxy, self).add(image)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 94,
> in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi result =
> self.base.add(base_item)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/db/__init__.py", line 261,
> in add
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi new_values
> = self.db_api.image_create(self.context, image_values)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line
> 128, in image_create
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> _image_update(context, values, None, purge_props=False)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> Retrying(*dargs, **dkw).call(f, *args, **kw)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/retrying.py", line 223, in call
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> attempt.get(self._wrap_exception)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> six.reraise(self.value[0], self.value[1], self.value[2])
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi attempt =
> Attempt(fn(*args, **kwargs), attempt_number, False)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 697,
> in wrapper
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> f(*args, **kwargs)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line
> 708, in _image_update
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi session =
> get_session()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line
> 101, in get_session
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi facade =
> _create_facade_lazily()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line
> 86, in _create_facade_lazily
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi _FACADE =
> session.EngineFacade.from_config(CONF)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py",
> line 1015, in from_config
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> expire_on_commit=expire_on_commit, _conf=conf)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py",
> line 943, in __init__
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> slave_connection=slave_connection)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py",
> line 338, in _start
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> engine_args, maker_args)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py",
> line 362, in _setup_for_connection
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> sql_connection=sql_connection, **engine_kwargs)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py",
> line 152, in create_engine
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi test_conn
> = _test_connection(engine, max_retries, retry_interval)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py",
> line 326, in _test_connection
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> engine.connect()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line
> 2013, in connect
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> self._connection_cls(self, **kwargs)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line
> 72, in __init__
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi if
> connection is not None else engine.raw_connection()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line
> 2099, in raw_connection
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> self.pool.unique_connection, _connection)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line
> 2073, in _wrap_pool_connect
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi e,
> dialect, self)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line
> 1399, in _handle_dbapi_exception_noconnection
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> util.raise_from_cause(newraise, exc_info)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line
> 199, in raise_from_cause
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> reraise(type(exception), exception, tb=exc_tb)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line
> 2069, in _wrap_pool_connect
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return fn()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 318,
> in unique_connection
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> _ConnectionFairy._checkout(self)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 708,
> in _checkout
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi fairy =
> _ConnectionRecord.checkout(pool)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 480,
> in checkout
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi rec =
> pool._do_get()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1049,
> in _do_get
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> self._dec_overflow()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
> line 60, in __exit__
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> compat.reraise(exc_type, exc_value, exc_tb)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1046,
> in _do_get
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> self._create_connection()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 323,
> in _create_connection
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> _ConnectionRecord(self)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 449,
> in __init__
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> self.connection = self.__connect()
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 602,
> in __connect
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi connection
> = self.__pool._invoke_creator(self)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line
> 97, in connect
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> dialect.connect(*cargs, **cparams)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py",
> line 377, in connect
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> self.dbapi.connect(*cargs, **cparams)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81,
> in Connect
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi return
> Connection(*args, **kwargs)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi File
> "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line
> 187, in __init__
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> super(Connection, self).__init__(*args, **kwargs2)
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> OperationalError: (_mysql_exceptions.OperationalError) (1045,
> "Access denied for user 'glance'@'localhost' (using password: YES)")
> 2015-11-13 18:00:18.511 1458 ERROR glance.common.wsgi
> 2015-11-13 18:00:18.688 1458 INFO eventlet.wsgi.server
> [req-76502a62-8c24-4b0a-828e-a0bf07733023
> 058f089c31e64370af9cb268c75da386 ff2149063f074c90bae7ea0f1dfcdfaf -
> - -] 10.11.12.230 - - [13/Nov/2015 18:00:18] "POST /v2/images
> HTTP/1.1" 5
> 00 454 0.222921 <tel:00%20454%200.222921>
> <snap>
>
>
> What does the entry 'ERROR glance.common.wsgi OperationalError:
> (_mysql_exceptions.OperationalError) (1045, "Access denied for user
> 'glance'@'localhost' (using password: YES)")' mean. Does it mean the
> password for the user glance in mysql does not match.
>
> I double checked it by setting it once more in the database but the
> error remains.
>
> Thanks,
>
> Joerg
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack at lists.openstack.org
> <mailto:openstack at lists.openstack.org>
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
More information about the Openstack
mailing list