<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi Ajay,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We work around this by setting a TTL on our notifications queues via RabbitMQ policy definition. We include the following in our definitions.json for RabbitMQ:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">"policies":[<o:p></o:p></p>
<p class="MsoNormal"> {"vhost": "/", "name": "notifications-ttl", "pattern": "^(notifications|versioned_notifications)\\.", "apply-to": "queues", "definition": {"message-ttl":600000}, "priority":0} ]<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This expires messages in the notifications and versioned_notifications queues after 10 minutes, which seems to work well for us. I believe we initially picked up this workaround from this[1] bug report.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hope this helps,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">- Chris<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-- <o:p></o:p></p>
<p class="MsoNormal">Christopher McCarthy<o:p></o:p></p>
<p class="MsoNormal">MathWorks<o:p></o:p></p>
<p class="MsoNormal">cmccarth@mathworks.com<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[1] https://bugs.launchpad.net/charm-rabbitmq-server/+bug/1737170<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Date: Wed, 2 Jun 2021 22:39:54 -0000<br>
From: "Ajay Tikoo (BLOOMBERG/ 120 PARK)" <atikoo@bloomberg.net><br>
To: openstack-discuss@lists.openstack.org<br>
Subject: [ops] rabbitmq queues for nova versioned notifications queues<br>
keep filling up<br>
Message-ID: <60B808BA00D0068401D80001_0_3025859@msclnypmsgsv04><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
I am not sure if this is the right channel/format to post this question, so my apologies in advance if this is not the right place.<br>
<br>
We are using Openstack Rocky. Watcher needs versioned notifications to be enabled. However after enabling versioned notifications, the queues for versioned_notifications (info and error) keep filling up Based on the updates the the Watchers cluster data model,
it appears that Watcher is consuming messages, but they still linger in these queues. So with nova versioned notifications disabled, Watcher is unable to update the cluster data model (between rebuild intervals), and with them enabled, it keeps filling up
the MQ queues. What is the best way to resolve this?<br>
<br>
Thank you,<br>
Ajay Tikoo<br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>