Slow instance launch times due to RabbitMQ

Laurent Dumont laurentfdumont at gmail.com
Wed Jul 31 10:49:15 UTC 2019


Could you forward the output of the following commands on a controller
node? :

rabbitmqctl cluster_status
rabbitmqctl list_queues

You won't necessarily see a high load on a Rabbit cluster that is in a bad
state.

On Wed, Jul 31, 2019 at 5:19 AM Grant Morley <grant at civo.com> wrote:

> Hi all,
>
> We are randomly seeing slow instance launch / deletion times and it
> appears to be because of RabbitMQ. We are seeing a lot of these messages in
> the logs for Nova and Neutron:
>
> ERROR oslo.messaging._drivers.impl_rabbit [-]
> [f4ab3ca0-b837-4962-95ef-dfd7d60686b6] AMQP server on 10.6.2.212:5671 is
> unreachable: Too many heartbeats missed. Trying again in 1 seconds. Client
> port: 37098: ConnectionForced: Too many heartbeats missed
>
> The RabbitMQ cluster isn't under high load and I am not seeing any packets
> drop over the network when I do some tracing.
>
> We are only running 15 compute nodes currently and have >1000 instances so
> it isn't a large deployment.
>
> Are there any good configuration tweaks for RabbitMQ running on OpenStack
> Queens?
>
> Many Thanks,
> --
>
> Grant Morley
> Cloud Lead, Civo Ltd
> www.civo.com | Signup for an account! <https://www.civo.com/signup>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190731/6636b989/attachment.html>


More information about the openstack-discuss mailing list