Open Stack

Dominic,
according to documentation:

heartbeat_rate = 2

integer value

How often times during the heartbeat_timeout_threshold we check the heartbeat.

heartbeat_timeout_threshold = 60

integer value

Number of seconds after which the Rabbit broker is considered down if heartbeat’s keep-alive fails (0 disables heartbeat).

So to avoid disconnection from service side (nova, keystone, etc.) I’ve increased Heartbeat_timeout_treshold from 60 to 720 and set heartbeat_rate to 4, so Rabbit will be considered dead after 360s not after 60s as in default. In addition I’ve increased heartbeat to 600 from 60 in rabbitmq.conf. And again - according to documentation:

The heartbeat timeout value defines after what period of time the peer TCP connection should be considered unreachable (down) by RabbitMQ and client libraries.

So disconnection from Rabbit’s side should be after 600s of no heartbeat

And I still got disconnections.

Best regards
Adam Tomas

> Wiadomość napisana przez DHilsbos at performair.com w dniu 17.09.2021, o godz. 18:38:
> 
> Adam;
> 
> If I'm reading this correctly; Rabbit is timing out, but you're increasing the heartbeat period of OpenStack.  This would make the issue worse, wouldn't it? 
> 
> It seems to me that you would want to lower the heartbeat interval of OpenStack, and raise the timeout of Rabbit.
> 
> That said; it looks like you're using Kola, and I know nothing about Kola.
> 
> Thank you,
> 
> Dominic L. Hilsbos, MBA
> Vice President – Information Technology
> Perform Air International Inc.
> DHilsbos at PerformAir.com
> www.PerformAir.com
> 
> 
> From: Adam Tomas [mailto:bkslash at poczta.onet.pl] 
> Sent: Friday, September 17, 2021 5:01 AM
> To: openstack-discuss
> Subject: RabbitMQ annoying disconnections
> 
> Hi,
> after some struggling I have almost „clear” logs (clear=error free :) ) Almost…. RabbitMQ keeps disconnecting sessions and there is a huge amount of disconnect errors in all logs (see below). I found this bug description:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1711794
> 
> in which we can read as follows: "this is a recoverable issue that is already handled by how oslo.messaging is designed. disconnection is not an error and should not be reported as such in the logs.”
> 
> 
> but… It is reported :( And produces tons of logs.
> 
> 
> I tried to modify heartbeat values - helped a little bit, but I had to increase [database] max_pool_size = 5 and that of course multiplied number of disconnection errors by 5 :( 
> 
> 
> [oslo_messaging_rabbit]
> heartbeat_timeout_threshold = 720 heartbeat_interval = 360 heartbeat_rate = 4
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210920/1a75cd94/attachment.htm>