Thank you, Christopher. From: cmccarth@mathworks.com At: 06/04/21 11:17:23 UTC-4:00To: openstack-discuss@lists.openstack.org Subject: Re: [ops] rabbitmq queues for nova versioned notifications queues keep filling up Hi Ajay, We work around this by setting a TTL on our notifications queues via RabbitMQ policy definition. We include the following in our definitions.json for RabbitMQ: "policies":[ {"vhost": "/", "name": "notifications-ttl", "pattern": "^(notifications|versioned_notifications)\\.", "apply-to": "queues", "definition": {"message-ttl":600000}, "priority":0} ] This expires messages in the notifications and versioned_notifications queues after 10 minutes, which seems to work well for us. I believe we initially picked up this workaround from this[1] bug report. Hope this helps, - Chris -- Christopher McCarthy MathWorks cmccarth@mathworks.com [1] https://bugs.launchpad.net/charm-rabbitmq-server/+bug/1737170 Date: Wed, 2 Jun 2021 22:39:54 -0000 From: "Ajay Tikoo (BLOOMBERG/ 120 PARK)" <atikoo@bloomberg.net> To: openstack-discuss@lists.openstack.org Subject: [ops] rabbitmq queues for nova versioned notifications queues keep filling up Message-ID: <60B808BA00D0068401D80001_0_3025859@msclnypmsgsv04> Content-Type: text/plain; charset="utf-8" I am not sure if this is the right channel/format to post this question, so my apologies in advance if this is not the right place. We are using Openstack Rocky. Watcher needs versioned notifications to be enabled. However after enabling versioned notifications, the queues for versioned_notifications (info and error) keep filling up Based on the updates the the Watchers cluster data model, it appears that Watcher is consuming messages, but they still linger in these queues. So with nova versioned notifications disabled, Watcher is unable to update the cluster data model (between rebuild intervals), and with them enabled, it keeps filling up the MQ queues. What is the best way to resolve this? Thank you, Ajay Tikoo