Open Stack

Hi,

thank you both for your helpful responses.

There are no consumers for the notifications so I'm indeed wondering  
who enabled them since the default for this deployment method was alse  
"false". Anyway, I will disable notifications and purge the queues and  
hopefully see the load dropping on the control node.

Thank you very much!

Eugen


Zitat von Satish Patel <satish.txt at gmail.com>:

> Yes, notifications are biggest enemy of rabbitMQ, specially if you  
> don’t purge time to time. Also increase time of stats collection for  
> rabbitMQ monitor if you are using it.
>
> Sent from my iPhone
>
>> On Aug 27, 2021, at 10:34 PM, Mohammed Naser <mnaser at vexxhost.com> wrote:
>>
>> 
>> Do you actually need notifications to be enabled?  If not, switch  
>> the driver to noop and make sure you purge the notification queues
>>
>>> On Fri, Aug 27, 2021 at 4:56 AM Eugen Block <eblock at nde.ag> wrote:
>>> Sorry, I accidentally hit "send"...adding more information.
>>>
>>> # nova.conf
>>>
>>> [oslo_messaging_notifications]
>>> driver = messagingv2
>>> [oslo_messaging_rabbit]
>>> amqp_durable_queues = false
>>> rabbit_ha_queues = false
>>> ssl = false
>>> heartbeat_timeout_threshold = 10
>>>
>>> # neutron.conf
>>>
>>> [oslo_messaging_rabbit]
>>> pool_max_size = 5000
>>> pool_max_overflow = 2000
>>> pool_timeout = 60
>>>
>>>
>>> These are the basics in the openstack services, I don't have access to
>>> the rabbit.conf right now but I can add more information as soon as I
>>> regain access to that machine.
>>>
>>> If there's anything else I can provide to improve the user experience
>>> at least a bit please let me know. It would help a lot!
>>>
>>> Regards,
>>> Eugen
>>>
>>>
>>> Zitat von Eugen Block <eblock at nde.ag>:
>>>
>>> > Hi *,
>>> >
>>> > I would greatly appreciate if anyone could help identifying some
>>> > tweaks, if possible at all.
>>> >
>>> > This is a single control node environment with 19 compute nodes
>>> > which are heavily used. It's still running on Pike and we won't be
>>> > able to update until complete redeployment.
>>> > I was able to improve the performance a lot by increasing the number
>>> > of workers etc., but to me it seems the next bottleneck is rabbitmq.
>>> > The rabbit process shows heavy CPU usage, almost constantly around
>>> > 150% according to 'top' and I see heartbeat errors in the logs (e.g.
>>> > cinder). Since there are so many config options I haven't touched
>>> > them yet. Reading some of the tuning docs often refer to HA
>>> > environments which is not the case here.
>>> >
>>> > I can paste some of the config snippets, maybe that already helps
>>> > identifying anything:
>>> >
>>> > # nova.conf
>>> >
>>> > [oslo_messaging_rabbit]
>>> > pool_max_size = 1600
>>> > pool_max_overflow = 1600
>>> > pool_timeout = 120
>>> > rpc_response_timeout = 120
>>> > rpc_conn_pool_size = 600
>>>
>>>
>>>
>>>
>> --
>> Mohammed Naser
>> VEXXHOST, Inc.