<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best Regards, Erik Olof Gunnar Andersson<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>From:</b> Satish Patel <satish.txt@gmail.com> <br>
<b>Sent:</b> Friday, March 20, 2020 9:23 AM<br>
<b>To:</b> Grant Morley <grant@civo.com><br>
<b>Cc:</b> Erik Olof Gunnar Andersson <eandersson@blizzard.com>; openstack-discuss@lists.openstack.org<br>
<b>Subject:</b> Re: Neutron RabbitMQ issues<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Oh you are right here, i have following stuff in my neutron.conf on server <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"># Notifications<br>
[oslo_messaging_notifications]<br>
driver = messagingv2<br>
topics = notifications<br>
transport_url = rabbit://<a href="https://urldefense.com/v3/__http:/neutron:5be2a043f9a93adbd@172.28.15.192:5671__;!!Ci6f514n9QsL8ck!1JtoahRvJ7-o3UTgt10NXnicoDEJzvPdT32aMejFaeSUu2ObRSEFvEfDVSWmp4HJag$">neutron:5be2a043f9a93adbd@172.28.15.192:5671</a>,<a href="https://urldefense.com/v3/__http:/neutron:5be2a043f9a93adbd@172.28.15.248:5671__;!!Ci6f514n9QsL8ck!1JtoahRvJ7-o3UTgt10NXnicoDEJzvPdT32aMejFaeSUu2ObRSEFvEfDVSXMR7XWRg$">neutron:5be2a043f9a93adbd@172.28.15.248:5671</a>,<a href="https://urldefense.com/v3/__http:/neutron:5be2a043f9a93adbd@172.28.15.22:5671/*neutron?ssl=1__;Lw!!Ci6f514n9QsL8ck!1JtoahRvJ7-o3UTgt10NXnicoDEJzvPdT32aMejFaeSUu2ObRSEFvEfDVSUgm8NHPA$">neutron:5be2a043f9a93adbd@172.28.15.22:5671//neutron?ssl=1</a><br>
<br>
# Messaging<br>
[oslo_messaging_rabbit]<br>
rpc_conn_pool_size = 30<br>
ssl = True<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Following change i am going to made let me know if anything missing. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">[DEFAULT]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">executor_thread_pool_size = 2048 <--- is this correct? i didn't see anywhere "rpc_thread_pool_size"<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">rpc_response_timeout = 3600<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">[oslo_messaging_notifications]<br>
topics = notifications<br>
driver = noop<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"># Messaging<br>
[oslo_messaging_rabbit]<br>
rpc_conn_pool_size = 300<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#500050">heartbeat_timeout_threshold = 0</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ssl = True<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Btw you might not necessarily be having RabbitMQ issues. You might also be experiencing something like this.<o:p></o:p></p>
<p class="MsoNormal"><a href="https://bugs.launchpad.net/neutron/+bug/1853071">https://bugs.launchpad.net/neutron/+bug/1853071</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best Regards, Erik Andersson<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Should i be adding this to all my compute nodes also ? <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, Mar 20, 2020 at 11:40 AM Grant Morley <<a href="mailto:grant@civo.com">grant@civo.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p>If you tune rabbit then for: <o:p></o:p></p>
<p><span style="color:#500050">heartbeat_timeout_threshold = 0</span><o:p></o:p></p>
<p>That should help with the error message you are getting. <o:p></o:p></p>
<p>That is a lot of messages queued. We had the same because we were not using ceilometer but had the "notifications" still turned on for it for services.<o:p></o:p></p>
<p>Are all of the ready messages for "<a href="https://urldefense.com/v3/__http:/notifications.info__;!!Ci6f514n9QsL8ck!1JtoahRvJ7-o3UTgt10NXnicoDEJzvPdT32aMejFaeSUu2ObRSEFvEfDVSXCYHosug$" target="_blank">notifications.info</a>" for the various services ( Nova,
Neutron, Keystone etc )<o:p></o:p></p>
<p>If that is the case you can disable those messages in your config files for those services. Look for:<o:p></o:p></p>
<p><span style="color:#500050"># Notifications<br>
[oslo_messaging_notifications]<br>
notification_topics = notifications<br>
driver = noop</span><o:p></o:p></p>
<p><span style="color:black">Make sure the driver option is set to "noop" by default it will be set too "messagingv2" and then restart the service and that should stop sending messages to the queue. You can then purge the "<a href="https://urldefense.com/v3/__http:/notifications.info__;!!Ci6f514n9QsL8ck!1JtoahRvJ7-o3UTgt10NXnicoDEJzvPdT32aMejFaeSUu2ObRSEFvEfDVSXCYHosug$" target="_blank">notifications.info</a>"
queue for Nova or Neutron etc..</span><o:p></o:p></p>
<p>We only had the "message ready" when we had the setting for ceilometer set but was not using it. Also only purge a queue if it is for that reason. Do not purge the queue if it is for any other reason than that as it can cause issues.<o:p></o:p></p>
<p style="margin-left:11.55pt">Hope that <span style="font-size:10.0pt;font-family:"Helvetica",sans-serif">
<o:p></o:p></span></p>
<div>
<div>
<div>
<div style="margin-left:67.5pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>