Le 24 janvier 2022 10:42:00 GMT+01:00, "Md. Hejbul Tawhid MUNNA" <munnaeebd@gmail.com> a écrit :

Hi,

Currently we have running 500+VM and total network is 383 including HA-network. 

Can you advice the appropriate value and is there any chance of service impact? 

Should we change the configuration in the neutron.conf on controller node?

Regards,

Munna

On Mon, Jan 24, 2022 at 2:47 PM Slawek Kaplonski <skaplons@redhat.com> wrote:

Hi,

On poniedziałek, 24 stycznia 2022 09:09:10 CET Md. Hejbul Tawhid MUNNA wrote:
> Hi,
>
> Suddenly we have observed few VM down . then we have found some agent are
> getting down (XXX) , agents are getting UP and down randomly. Please check
> the attachment.
>
>
/////////////////////////////////////////////////////////////////////////////
> /////////////////////// /sqlalchemy/pool.py", line 788, in _checkout\n   
> fairy =
> _ConnectionRecord.checkout(pool)\n', u'  File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 532, in
> checkout\n    rec = pool._do_get()\n', u'  File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1186, in
> _do_get\n    (self.size(), self.overflow(), self._timeout),
> code="3o7r")\n', u'TimeoutError: QueuePool limit of size 5 overflow 50
> reached, connection timed out, timeout 30 (Background on this error at:
> http://sqlalche.me/e/3o7r)\n'].
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent Traceback (most
> recent call last):
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent   File
> "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 837, in
> _report_state
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent     True)
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent   File
> "/usr/lib/python2.7/site-packages/neutron/agent/rpc.py", line 97, in
> report_state
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent     return
> method(context, 'report_state', **kwargs)
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent   File
> "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 179,
> in call
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent
> retry=self.retry)
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent   File
> "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 133,
> in _send
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent     retry=retry)
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent   File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py",
> line 645, in send
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent
> call_monitor_timeout, retry=retry)
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent   File
> "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py",
> line 636, in _send
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent     raise result
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent RemoteError:
> Remote error: TimeoutError QueuePool limit of size 5 overflow 50 reached,
> connection timed out, timeout 30 (Background on this error at:
> http://sqlalche.me/e/3o7r)
> 2022-01-24 01:05:39.592 302841 ERROR neutron.agent.l3.agent [u'Traceback
> (most recent call last):\n', u'  File
> "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163,
> in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', u'
>  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
> line 265, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt,
> args)\n', u'  File
> "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line
> 194, in _do_dispatch\n
>
>
/////////////////////////////////////////////////////////////////////////////
> ////
>
> Is there anything related with the following default configuration.
>
> /etc/neutron/neutron.conf
> #max_pool_size = 5
> #max_overflow = 50

Yes. You probably have busy environment and You need to increase those values
to have more connections from the neutron server to the database.

>
> regards,
> Munna



--
Slawek Kaplonski
Principal Software Engineer
Red Hat