Open Stack

On Thu, 18 Mar 2021 at 16:05, Adam Tomas <bkslash at poczta.onet.pl> wrote:
>
> Hi,
> I have a problem with rabbitmq heartbeats timeout (described also here: https://bugzilla.redhat.com/show_bug.cgi?id=1711794). Although I have threads=1 the problem still persists, generating a lot of messages in logs:
>
> 2021-03-18 15:17:17.482 [error] <0.122.51> closing AMQP connection <0.122.51> (x.x.x.100:60456 -> x.x.x.100:5672 - mod_wsgi:699:9a813fcb-c29f-4886-82bc-00bf478b6b64):
> missed heartbeats from client, timeout: 60s
> 2021-03-18 15:17:17.484 [info] <0.846.51> Closing all channels from connection '<<x.x.x.100:60456 -> x.x.x.100:5672">>' because it has been closed
> 2021-03-18 15:18:15.918 [error] <0.150.51> closing AMQP connection <0.150.51> (x.x.x.111:41934 -> x.x.x.100:5672 - mod_wsgi:697:b608c7b8-9644-434e-93af-c00222c0a700):
> missed heartbeats from client, timeout: 60s
> 2021-03-18 15:18:15.920 [error] <0.153.51> closing AMQP connection <0.153.51> (x.x.x.111:41936 -> x.x.x.100:5672 - mod_wsgi:697:77348197-148b-41a6-928f-c5eddfab57c9):
> missed heartbeats from client, timeout: 60s
> 2021-03-18 15:18:15.920 [info] <0.1527.51> Closing all channels from connection '<<x.x.x.111:41934 -> x.x.x.100:5672">>' because it has been closed
> 2021-03-18 15:18:15.922 [info] <0.1531.51> Closing all channels from connection '<<x.x.x.111:41936 -> x.x.x.100:5672">>' because it has been closed
> 2021-03-18 15:20:16.080 [info] <0.2196.51> accepting AMQP connection <0.2196.51> (x.x.x.111:34826 -> x.x.x.100:5672)
> 2021-03-18 15:20:16.080 [info] <0.2199.51> accepting AMQP connection <0.2199.51> (x.x.x.111:34828 -> x.x.x.100:5672)
>
> I’ve set heartbeat = 600 in rabbitmq.conf and still get disconnections after 60s timeout… How to set proper timeout to avoid disconnections?

Hi Adam,

I have seen similar messages in the past, but haven't really looked
into it. It seems to happen during some intensive processes like
encrypted cinder volume creation.

Have you tried configuring oslo.messaging? For example,
[oslo_messaging_rabbitmq] heartbeat_timeout_threshold.

https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html

Mark

>
> Best regards
> Adam
>