Open Stack

Hi Mark,
this happens without any noticeable load on Openstack. Where should I put [oslo_messaging_rabbitmq] heartbeat_timeout_threshold in kolla-ansible? I can’t find any oslo.config file… Should it be in  .conf file of every service?
Best regards
Adam


> Wiadomość napisana przez Mark Goddard <mark at stackhpc.com> w dniu 22.03.2021, o godz. 10:49:
> 
> On Thu, 18 Mar 2021 at 16:05, Adam Tomas <bkslash at poczta.onet.pl <mailto:bkslash at poczta.onet.pl>> wrote:
>> 
>> Hi,
>> I have a problem with rabbitmq heartbeats timeout (described also here: https://bugzilla.redhat.com/show_bug.cgi?id=1711794). Although I have threads=1 the problem still persists, generating a lot of messages in logs:
>> 
>> 2021-03-18 15:17:17.482 [error] <0.122.51> closing AMQP connection <0.122.51> (x.x.x.100:60456 -> x.x.x.100:5672 - mod_wsgi:699:9a813fcb-c29f-4886-82bc-00bf478b6b64):
>> missed heartbeats from client, timeout: 60s
>> 2021-03-18 15:17:17.484 [info] <0.846.51> Closing all channels from connection '<<x.x.x.100:60456 -> x.x.x.100:5672">>' because it has been closed
>> 2021-03-18 15:18:15.918 [error] <0.150.51> closing AMQP connection <0.150.51> (x.x.x.111:41934 -> x.x.x.100:5672 - mod_wsgi:697:b608c7b8-9644-434e-93af-c00222c0a700):
>> missed heartbeats from client, timeout: 60s
>> 2021-03-18 15:18:15.920 [error] <0.153.51> closing AMQP connection <0.153.51> (x.x.x.111:41936 -> x.x.x.100:5672 - mod_wsgi:697:77348197-148b-41a6-928f-c5eddfab57c9):
>> missed heartbeats from client, timeout: 60s
>> 2021-03-18 15:18:15.920 [info] <0.1527.51> Closing all channels from connection '<<x.x.x.111:41934 -> x.x.x.100:5672">>' because it has been closed
>> 2021-03-18 15:18:15.922 [info] <0.1531.51> Closing all channels from connection '<<x.x.x.111:41936 -> x.x.x.100:5672">>' because it has been closed
>> 2021-03-18 15:20:16.080 [info] <0.2196.51> accepting AMQP connection <0.2196.51> (x.x.x.111:34826 -> x.x.x.100:5672)
>> 2021-03-18 15:20:16.080 [info] <0.2199.51> accepting AMQP connection <0.2199.51> (x.x.x.111:34828 -> x.x.x.100:5672)
>> 
>> I’ve set heartbeat = 600 in rabbitmq.conf and still get disconnections after 60s timeout… How to set proper timeout to avoid disconnections?
> 
> Hi Adam,
> 
> I have seen similar messages in the past, but haven't really looked
> into it. It seems to happen during some intensive processes like
> encrypted cinder volume creation.
> 
> Have you tried configuring oslo.messaging? For example,
> [oslo_messaging_rabbitmq] heartbeat_timeout_threshold.
> 
> https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html <https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html>
> 
> Mark
> 
>> 
>> Best regards
>> Adam

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210322/db449afa/attachment.html>