[openstack-discuss][Nova][Horizon] 'Too many connections' error

Minjun Hong nickeysgo at gmail.com
Fri Dec 28 16:26:16 UTC 2018


Hi.
I'm currently trying to substitute the existing controller node with a new
server.
After I install the 'Minimal deployment for Queens' on the new server, I
connect all nodes.

However, when I made an instance on Horizon, there were some error
messages, as below link:

https://imgur.com/hSfHx4M

And, when I typed 'openstack flavor list', the result was as follows:

> # openstack flavor list
> Unexpected API Error. Please report this at
> http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
> <class 'sqlalchemy.exc.OperationalError'> (HTTP 500) (Request-ID:
> req-666d4e30-081a-429f-88ae-763e62990770)


Moreover, I found something regarding this problem in the Nova log:

2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
> [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3
> 96ad10a59d114042b8f1ee82c438649a - default default] Unexpected exception in
> API method: OperationalError: (pymysql.err.OperationalError) (1040, u'Too
> many connections')

2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi Traceback (most
> recent call last):
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 788, in
> wrapped
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> f(*args, **kwargs)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/flavors.py",
> line 50, in detail
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  limited_flavors = self._get_flavors(req)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/flavors.py",
> line 117, in _get_flavors
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  limit=limit, marker=marker)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184,
> in wrapper
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     result =
> fn(cls, context, *args, **kwargs)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/nova/objects/flavor.py", line 650, in
> get_all
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  marker=marker)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line
> 985, in wrapper
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     with
> self._transaction_scope(context):
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> self.gen.next()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line
> 1035, in _transaction_scope
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  context=context) as resource:
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> self.gen.next()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line
> 638, in _session
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  bind=self.connection, mode=self.mode)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line
> 403, in _create_session
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  self._start()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line
> 489, in _start
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  engine_args, maker_args)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line
> 513, in _setup_for_connection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  sql_connection=sql_connection, **engine_kwargs)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/debtcollector/renames.py", line 45, in
> wrapper
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> f(*args, **kwargs)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 184,
> in create_engine
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     test_conn =
> _test_connection(engine, max_retries, retry_interval)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 362,
> in _test_connection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> engine.connect()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2091, in
> connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> self._connection_cls(self, **kwargs)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 90, in
> __init__
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     if
> connection is not None else engine.raw_connection()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2177, in
> raw_connection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  self.pool.unique_connection, _connection)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2151, in
> _wrap_pool_connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     e, dialect,
> self)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1461, in
> _handle_dbapi_exception_noconnection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  util.raise_from_cause(newraise, exc_info)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in
> raise_from_cause
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  reraise(type(exception), exception, tb=exc_tb, cause=cause)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2147, in
> _wrap_pool_connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return fn()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 328, in
> unique_connection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> _ConnectionFairy._checkout(self)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2151, in
> _wrap_pool_connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     e, dialect,
> self)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1461, in
> _handle_dbapi_exception_noconnection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  util.raise_from_cause(newraise, exc_info)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in
> raise_from_cause
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  reraise(type(exception), exception, tb=exc_tb, cause=cause)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2147, in
> _wrap_pool_connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return fn()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 328, in
> unique_connection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> _ConnectionFairy._checkout(self)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 766, in
> _checkout
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     fairy =
> _ConnectionRecord.checkout(pool)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 516, in checkout
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     rec =
> pool._do_get()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1138, in _do_get
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  self._dec_overflow()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66,
> in __exit__
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  compat.reraise(exc_type, exc_value, exc_tb)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1135, in _do_get
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> self._create_connection()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 333, in
> _create_connection
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> _ConnectionRecord(self)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 461, in __init__
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  self.__connect(first_connect_check=True)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 651, in
> __connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     connection
> = pool._invoke_creator(self)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
> 105, in connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> dialect.connect(*cargs, **cparams)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 393,
> in connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> self.dbapi.connect(*cargs, **cparams)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/pymysql/__init__.py", line 90, in Connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     return
> Connection(*args, **kwargs)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 699, in
> __init__
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  self.connect()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 936, in
> connect
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  self._request_authentication()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1156, in
> _request_authentication
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     auth_packet
> = self._read_packet()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1018, in
> _read_packet
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  packet.check_error()
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 384, in
> check_error
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
>  err.raise_mysql_exception(self._data)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi   File
> "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 107, in
> raise_mysql_exception
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi     raise
> errorclass(errno, errval)
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
> OperationalError: (pymysql.err.OperationalError) (1040, u'Too many
> connections')
> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi
> 2018-12-29 00:58:52.930 3765 INFO nova.api.openstack.wsgi
> [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3
> 96ad10a59d114042b8f1ee82c438649a - default default] HTTP exception thrown:
> Unexpected API Error. Please report this at
> http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
> <class 'sqlalchemy.exc.OperationalError'>
> 2018-12-29 00:58:52.931 3765 INFO nova.osapi_compute.wsgi.server
> [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3
> 96ad10a59d114042b8f1ee82c438649a - default default] 10.150.21.195 "GET
> /v2.1/flavors/detail HTTP/1.1" status: 500 len: 638 time: 0.1569352


I already did googling with 'Too many connections' but, there were only
results about 'max_connections' of the mysql config file.
I also set the value as 4096 and I think it is enough.

Now, I do not know what to look for to resolve this problem.
Please give me any clue.

Thanks!
Regards,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20181229/6baca037/attachment-0001.html>


More information about the openstack-discuss mailing list