- openstack-discuss - lists.openstack.org

Re: [openstack-discuss][Nova][Horizon] 'Too many connections' error
by Minjun Hong 30 Dec '18

30 Dec '18

Thanks for your reply, Chris. Following your advice, I adjusted the values in 'nova.conf' file. Setting max_connections in the mysql config file to 4096, the modified values are: 1. max_pool_size in [database] section to 1000 2. max_pool_size in [api_database] section to 1000 3. max_overflow in [database] section to 1000 4. max_overflow in [api_database] section to 1000 However, the problem which some services, such as Nova or Neutron, cannot connect to mysql DB occurred again. So that, I also modified the number of workers you had mentioned as follows: 1. osapi_compute_workers in [DEFAULT] section to 128 2. metadata_workers in [DEFAULT] section to 128 3. workers in [conductor] section to 128 And then, I could not use all openstack command at all. I mean, all openstack command did not work with "An unexpected error prevented the server from fulfilling your request. (HTTP 500)" error messages. After looking into some config files of the openstack services, I found out some similar parameters with the parameters you had mentioned were in the config files. For example, in neutron.conf file, max_pool_size also exists and glance-api.conf has worker parameter, either. I'm confused for what parameters should I use. Actually, I'm not sure of functions of the parameters and I need your help more. Thanks! Regards, On Sat, Dec 29, 2018 at 2:11 PM Chris Hoge <chris(a)openstack.org> wrote: > I’ve run into something similar. By default there are a few things going > on with Nova. To begin with, Nova starts up several services multiplied > by the number of cores on your system. When you add to this that Nova > doesn't by default constrain its database connection pool and it's eager > to create new database connections, you can quickly open tens of > thousands of new connections across more than a dozen processes in a > matter of minutes when starting a complete Nova control plane. > > Thankfully, this is easy to fix by setting the max_pool_size and > max_overflow parameters, as well as constraining the number of workers > for the different processes in nova.conf. > > You’ll want to tune your mysql max-connections to match your expected > volume of connections from the control plane. > > Hope that helps. > > On Dec 28, 2018, at 8:26 AM, Minjun Hong <nickeysgo(a)gmail.com> wrote: > > Hi. > I'm currently trying to substitute the existing controller node with a new > server. > After I install the 'Minimal deployment for Queens' on the new server, I > connect all nodes. > > However, when I made an instance on Horizon, there were some error > messages, as below link: > > https://imgur.com/hSfHx4M > > And, when I typed 'openstack flavor list', the result was as follows: > >> # openstack flavor list >> Unexpected API Error. Please report this at >> http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. >> <class 'sqlalchemy.exc.OperationalError'> (HTTP 500) (Request-ID: >> req-666d4e30-081a-429f-88ae-763e62990770) > > > Moreover, I found something regarding this problem in the Nova log: > > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3 >> 96ad10a59d114042b8f1ee82c438649a - default default] Unexpected exception in >> API method: OperationalError: (pymysql.err.OperationalError) (1040, u'Too >> many connections') > > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi Traceback (most >> recent call last): >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 788, in >> wrapped >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> f(*args, **kwargs) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/flavors.py", >> line 50, in detail >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> limited_flavors = self._get_flavors(req) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/flavors.py", >> line 117, in _get_flavors >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> limit=limit, marker=marker) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, >> in wrapper >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi result = >> fn(cls, context, *args, **kwargs) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/nova/objects/flavor.py", line 650, in >> get_all >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> marker=marker) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line >> 985, in wrapper >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi with >> self._transaction_scope(context): >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> self.gen.next() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line >> 1035, in _transaction_scope >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> context=context) as resource: >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> self.gen.next() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line >> 638, in _session >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> bind=self.connection, mode=self.mode) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line >> 403, in _create_session >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> self._start() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line >> 489, in _start >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> engine_args, maker_args) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line >> 513, in _setup_for_connection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> sql_connection=sql_connection, **engine_kwargs) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/debtcollector/renames.py", line 45, in >> wrapper >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> f(*args, **kwargs) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 184, >> in create_engine >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi test_conn >> = _test_connection(engine, max_retries, retry_interval) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 362, >> in _test_connection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> engine.connect() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2091, in >> connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> self._connection_cls(self, **kwargs) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 90, in >> __init__ >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi if >> connection is not None else engine.raw_connection() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2177, in >> raw_connection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> self.pool.unique_connection, _connection) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2151, in >> _wrap_pool_connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi e, >> dialect, self) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1461, in >> _handle_dbapi_exception_noconnection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> util.raise_from_cause(newraise, exc_info) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in >> raise_from_cause >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> reraise(type(exception), exception, tb=exc_tb, cause=cause) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2147, in >> _wrap_pool_connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return fn() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 328, in >> unique_connection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> _ConnectionFairy._checkout(self) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2151, in >> _wrap_pool_connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi e, >> dialect, self) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1461, in >> _handle_dbapi_exception_noconnection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> util.raise_from_cause(newraise, exc_info) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in >> raise_from_cause >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> reraise(type(exception), exception, tb=exc_tb, cause=cause) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2147, in >> _wrap_pool_connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return fn() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 328, in >> unique_connection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> _ConnectionFairy._checkout(self) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 766, in >> _checkout >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi fairy = >> _ConnectionRecord.checkout(pool) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 516, in checkout >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi rec = >> pool._do_get() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1138, in _do_get >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> self._dec_overflow() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, >> in __exit__ >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> compat.reraise(exc_type, exc_value, exc_tb) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1135, in _do_get >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> self._create_connection() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 333, in >> _create_connection >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> _ConnectionRecord(self) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 461, in __init__ >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> self.__connect(first_connect_check=True) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 651, in >> __connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi connection >> = pool._invoke_creator(self) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line >> 105, in connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> dialect.connect(*cargs, **cparams) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 393, >> in connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> self.dbapi.connect(*cargs, **cparams) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/pymysql/__init__.py", line 90, in Connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return >> Connection(*args, **kwargs) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 699, in >> __init__ >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> self.connect() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 936, in >> connect >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> self._request_authentication() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1156, in >> _request_authentication >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> auth_packet = self._read_packet() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1018, in >> _read_packet >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> packet.check_error() >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 384, in >> check_error >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> err.raise_mysql_exception(self._data) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File >> "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 107, in >> raise_mysql_exception >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi raise >> errorclass(errno, errval) >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> OperationalError: (pymysql.err.OperationalError) (1040, u'Too many >> connections') >> 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi >> 2018-12-29 00:58:52.930 3765 INFO nova.api.openstack.wsgi >> [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3 >> 96ad10a59d114042b8f1ee82c438649a - default default] HTTP exception thrown: >> Unexpected API Error. Please report this at >> http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. >> <class 'sqlalchemy.exc.OperationalError'> >> 2018-12-29 00:58:52.931 3765 INFO nova.osapi_compute.wsgi.server >> [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3 >> 96ad10a59d114042b8f1ee82c438649a - default default] 10.150.21.195 "GET >> /v2.1/flavors/detail HTTP/1.1" status: 500 len: 638 time: 0.1569352 > > > I already did googling with 'Too many connections' but, there were only > results about 'max_connections' of the mysql config file. > I also set the value as 4096 and I think it is enough. > > Now, I do not know what to look for to resolve this problem. > Please give me any clue. > > Thanks! > Regards, > > >

1 0

Re: Openstack-ansible and HAProxy
by zud0ku 29 Dec '18

29 Dec '18

Hi, I am having same problem. Were you able to solve the situation? Please can you share any updates? Thanks in advance. -- Sent from: http://openstack.10931.n7.nabble.com/Operators-f4774.html

1 0

[tripleo] cleaning-up old reviews
by Emilien Macchi 29 Dec '18

29 Dec '18

Like once a year, we usually clean-up old reviews in Gerrit. Per policy, I've abandoned a lot of patches that were in merge conflict, or not reviewed, or WIP or not passing CI, older than 6 months. If you want to check if one of your patches were abandoned: https://review.openstack.org/#/q/status:abandoned+owner:self Please feel free to re-open it, and let me know if any trouble. I'll also spend time on our Launchpad and also remove old bugs without any activity. Thanks, -- Emilien Macchi

1 0

[I18n] No team meeting
by Frank Kloeker 29 Dec '18

29 Dec '18

Hello, due the vacation period the I18n team will also skip the next team meetings. Our next appointment will be 2019/01/10 07:00 UTC in #openstack-meeting. Enjoy the time and a Happy New Year! Frank

3 2

[Firewall][OpenStack]
by Soheil Pourbafrani 29 Dec '18

29 Dec '18

I tried to configure the firewall for OpenStack Controller and Compute node and here are the rules I added to the firewall: myZone (active) target: default icmp-block-inversion: no interfaces: enp2s0 enp7s4 sources: services: ssh dhcpv6-client ports: 80/tcp 6080/tcp 11211/tcp 9696/tcp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: rule family="ipv4" source address="192.168.0.32" accept rule family="ipv4" source address="192.168.0.31" accept The address of the Controller and the Compute nodes are 192.168.0.31 and 192.168.0.32, respectively. Using these rules I can use Horizon on the browser and the Compute node services can connect to the Controller nodes ports. The problem is when the firewall is enabled on the Controller node, instances that are running on the Controller node (I configure the Controller node as the Compute node, too) just can be pinged and all other VMs and nodes (including the Controller node) cannot connect to it (using SSH or any other connection to a specific port). - There is no firewall running on instances. - I configured an external network to connect VMs to each other - CentOS7 is running on all nodes Here are ports listening on the Controller node: Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:8775 0.0.0.0:* LISTEN 4478/python2 tcp 0 0 0.0.0.0:9191 0.0.0.0:* LISTEN 4461/python2 tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 10189/httpd tcp 0 0 0.0.0.0:8776 0.0.0.0:* LISTEN 4487/python2 tcp 0 0 0.0.0.0:25672 0.0.0.0:* LISTEN 4466/beam.smp tcp 0 0 0.0.0.0:8778 0.0.0.0:* LISTEN 10189/httpd tcp 0 0 192.168.0.31:3306 0.0.0.0:* LISTEN 4860/mysqld tcp 0 0 192.168.0.31:2379 0.0.0.0:* LISTEN 4464/etcd tcp 0 0 192.168.0.31:11211 0.0.0.0:* LISTEN 4457/memcached tcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 4457/memcached tcp 0 0 192.168.0.31:5900 0.0.0.0:* LISTEN 16844/qemu-kvm tcp 0 0 0.0.0.0:9292 0.0.0.0:* LISTEN 4500/python2 tcp 0 0 192.168.0.31:2380 0.0.0.0:* LISTEN 4464/etcd tcp 0 0 192.168.0.31:5901 0.0.0.0:* LISTEN 16982/qemu-kvm tcp 0 0 192.168.0.31:5902 0.0.0.0:* LISTEN 17339/qemu-kvm tcp 0 0 192.168.0.31:5903 0.0.0.0:* LISTEN 17621/qemu-kvm tcp 0 0 192.168.0.31:5904 0.0.0.0:* LISTEN 17840/qemu-kvm tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 10189/httpd tcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN 1/systemd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 4468/sshd tcp 0 0 192.168.0.31:3260 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:6080 0.0.0.0:* LISTEN 4458/python2 tcp 0 0 0.0.0.0:9696 0.0.0.0:* LISTEN 4473/python2 tcp 0 0 0.0.0.0:8774 0.0.0.0:* LISTEN 4478/python2 tcp6 0 0 :::5672 :::* LISTEN 4466/beam.smp tcp6 0 0 :::22 :::* LISTEN 4468/sshd So, is there any port or something to add to firewall rules for making instances reachable when the firewall is running on the Controller node?

1 0

openstack magnum kubernetes load balancer
by Ignazio Cassano 28 Dec '18

28 Dec '18

Hello All, I created a kuberneted cluster with magnum. Than I would like to create a load balancer for nginx: kubectl run nginx --image=nginx --replicas=2 --port=80 kubectl expose deployment nginx --target-port=80 --type=LoadBalancer But kubectl get service gives: kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 152m nginx LoadBalancer 10.254.161.102 <pending> 80:30648/TCP 2m58s If I use NodePort instead of LoadBalancer it works. Does kubernetes requires octavia ? I am using loadbalance v2 without octavia. Regards Ignazio

1 0

[openstack-discuss][Nova][Horizon] 'Too many connections' error
by Minjun Hong 28 Dec '18

28 Dec '18

Hi. I'm currently trying to substitute the existing controller node with a new server. After I install the 'Minimal deployment for Queens' on the new server, I connect all nodes. However, when I made an instance on Horizon, there were some error messages, as below link: https://imgur.com/hSfHx4M And, when I typed 'openstack flavor list', the result was as follows: > # openstack flavor list > Unexpected API Error. Please report this at > http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. > <class 'sqlalchemy.exc.OperationalError'> (HTTP 500) (Request-ID: > req-666d4e30-081a-429f-88ae-763e62990770) Moreover, I found something regarding this problem in the Nova log: 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3 > 96ad10a59d114042b8f1ee82c438649a - default default] Unexpected exception in > API method: OperationalError: (pymysql.err.OperationalError) (1040, u'Too > many connections') 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi Traceback (most > recent call last): > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 788, in > wrapped > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > f(*args, **kwargs) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/flavors.py", > line 50, in detail > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > limited_flavors = self._get_flavors(req) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/flavors.py", > line 117, in _get_flavors > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > limit=limit, marker=marker) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, > in wrapper > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi result = > fn(cls, context, *args, **kwargs) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/nova/objects/flavor.py", line 650, in > get_all > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > marker=marker) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line > 985, in wrapper > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi with > self._transaction_scope(context): > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > self.gen.next() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line > 1035, in _transaction_scope > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > context=context) as resource: > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > self.gen.next() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line > 638, in _session > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > bind=self.connection, mode=self.mode) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line > 403, in _create_session > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > self._start() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line > 489, in _start > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > engine_args, maker_args) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line > 513, in _setup_for_connection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > sql_connection=sql_connection, **engine_kwargs) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/debtcollector/renames.py", line 45, in > wrapper > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > f(*args, **kwargs) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 184, > in create_engine > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi test_conn = > _test_connection(engine, max_retries, retry_interval) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 362, > in _test_connection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > engine.connect() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2091, in > connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > self._connection_cls(self, **kwargs) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 90, in > __init__ > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi if > connection is not None else engine.raw_connection() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2177, in > raw_connection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > self.pool.unique_connection, _connection) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2151, in > _wrap_pool_connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi e, dialect, > self) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1461, in > _handle_dbapi_exception_noconnection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > util.raise_from_cause(newraise, exc_info) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in > raise_from_cause > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > reraise(type(exception), exception, tb=exc_tb, cause=cause) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2147, in > _wrap_pool_connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return fn() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 328, in > unique_connection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > _ConnectionFairy._checkout(self) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2151, in > _wrap_pool_connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi e, dialect, > self) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1461, in > _handle_dbapi_exception_noconnection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > util.raise_from_cause(newraise, exc_info) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in > raise_from_cause > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > reraise(type(exception), exception, tb=exc_tb, cause=cause) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2147, in > _wrap_pool_connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return fn() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 328, in > unique_connection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > _ConnectionFairy._checkout(self) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 766, in > _checkout > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi fairy = > _ConnectionRecord.checkout(pool) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 516, in checkout > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi rec = > pool._do_get() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1138, in _do_get > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > self._dec_overflow() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, > in __exit__ > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > compat.reraise(exc_type, exc_value, exc_tb) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1135, in _do_get > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > self._create_connection() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 333, in > _create_connection > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > _ConnectionRecord(self) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 461, in __init__ > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > self.__connect(first_connect_check=True) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 651, in > __connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi connection > = pool._invoke_creator(self) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line > 105, in connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > dialect.connect(*cargs, **cparams) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 393, > in connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > self.dbapi.connect(*cargs, **cparams) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/pymysql/__init__.py", line 90, in Connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi return > Connection(*args, **kwargs) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 699, in > __init__ > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > self.connect() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 936, in > connect > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > self._request_authentication() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1156, in > _request_authentication > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi auth_packet > = self._read_packet() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1018, in > _read_packet > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > packet.check_error() > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 384, in > check_error > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > err.raise_mysql_exception(self._data) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi File > "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 107, in > raise_mysql_exception > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi raise > errorclass(errno, errval) > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > OperationalError: (pymysql.err.OperationalError) (1040, u'Too many > connections') > 2018-12-29 00:58:52.928 3765 ERROR nova.api.openstack.wsgi > 2018-12-29 00:58:52.930 3765 INFO nova.api.openstack.wsgi > [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3 > 96ad10a59d114042b8f1ee82c438649a - default default] HTTP exception thrown: > Unexpected API Error. Please report this at > http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. > <class 'sqlalchemy.exc.OperationalError'> > 2018-12-29 00:58:52.931 3765 INFO nova.osapi_compute.wsgi.server > [req-666d4e30-081a-429f-88ae-763e62990770 bb1e571e4d64462bac80654b153a88c3 > 96ad10a59d114042b8f1ee82c438649a - default default] 10.150.21.195 "GET > /v2.1/flavors/detail HTTP/1.1" status: 500 len: 638 time: 0.1569352 I already did googling with 'Too many connections' but, there were only results about 'max_connections' of the mysql config file. I also set the value as 4096 and I think it is enough. Now, I do not know what to look for to resolve this problem. Please give me any clue. Thanks! Regards,

1 0

[tripleo] removing MongoDB
by Emilien Macchi 28 Dec '18

28 Dec '18

Hi, MongoDB hasn't been used anywhere and it's even unsupported in TripleO since Pike. I think it's time to clean it up, therefore I propose we remove it from TripleO: https://review.openstack.org/#/q/topic:tripleo/rm/mongodb+(status:open+OR+s… Feedback is welcome, Thanks -- Emilien Macchi

1 0

[Cyborg] Nominating Coco Gao for Cyborg Core Reviewer
by Zhipeng Huang 28 Dec '18

28 Dec '18

Hi folks, Coco has been participating in Cyborg project from Rocky design discussion and has been driving the DB refactoring work ever since. I would like to nominate her as a new core reviewer of the team to recognize her contribution as well as increase the bandwidth of cyborg core team. If you have any concerns by replying to this email thread within a week. Thank you very much ! -- Zhipeng (Howard) Huang Principle Engineer IT Standard & Patent/IT Product Line Huawei Technologies Co,. Ltd Email: huangzhipeng(a)huawei.com Office: Huawei Industrial Base, Longgang, Shenzhen

1 0

[Cyborg] Nominating Yumeng Bao for Cyborg Core Reviewer
by Li Liu 28 Dec '18

28 Dec '18

Hi Team, Yumeng Bao has high-quality reviews in the last couple of cycles so I would like to nominate her for Cyborg core Please respond with your +1 or -1 votes. We'll hold voting open for 7 days -- Thank you Regards Li

1 0