[Openstack] Bottleneck of message queue

Joshua Harlow harlowja at yahoo-inc.com
Wed Oct 10 17:53:12 UTC 2012


Also some real data/graphs/metrics if u have anything would go a long way in helping others see the problem.

Without data though its hard to know what is broke, what is the limit, and what needs to be fixed.

-Josh

From: Hao Wang <hao.1.wang at gmail.com<mailto:hao.1.wang at gmail.com>>
Date: Wednesday, October 10, 2012 6:50 AM
To: Sandy Walsh <sandy.walsh at rackspace.com<mailto:sandy.walsh at rackspace.com>>, "alexis at rabbitmq.com<mailto:alexis at rabbitmq.com>" <alexis at rabbitmq.com<mailto:alexis at rabbitmq.com>>
Cc: "openstack at lists.launchpad.net<mailto:openstack at lists.launchpad.net>" <openstack at lists.launchpad.net<mailto:openstack at lists.launchpad.net>>
Subject: Re: [Openstack] Bottleneck of message queue

Thanks guys for your insightful replies. I'm studying them. If I've got anything, I'll get back to you as soon as possible.

Have a nice day!

Cheers,
Howard

On Wed, Oct 10, 2012 at 7:25 PM, Sandy Walsh <sandy.walsh at rackspace.com<mailto:sandy.walsh at rackspace.com>> wrote:
Hey Howard,

Queues are generally in memory, but you may turn on persistent (disk) queues in your environment. So that's your limitation. Having rabbitmq on a different server is a good idea.

Also, Queues are only used for control, not user data, so they shouldn't be that big of a burden. Having a queue-based architecture adds some complexity for synchronization, but their benefit of giving us burst-handling capabilities far outweigh that (imho).

If your queues are filling up, you may:
1. need beefier machines processing the offending queues (or rabbit server)
2. need to add more worker nodes (more network, more scheduler, though more compute isn't appropriate)
3. think about clustering rabbit

Notifications are perhaps the chattiest queues in the system, so make sure you have suitable workers there (if you have notifications turned on)

This might help you understand the flow through the queues a little more?
http://www.sandywalsh.com/2012/04/openstack-nova-internals-pt1-overview.html
http://www.sandywalsh.com/2012/09/openstack-nova-internals-pt2-services.html

Cheers,
Sandy


From: openstack-bounces+sandy.walsh=rackspace.com at lists.launchpad.net<mailto:rackspace.com at lists.launchpad.net> [openstack-bounces+sandy.walsh=rackspace.com at lists.launchpad.net<mailto:rackspace.com at lists.launchpad.net>] on behalf of Hao Wang [hao.1.wang at gmail.com<mailto:hao.1.wang at gmail.com>]
Sent: Tuesday, October 09, 2012 11:49 PM
To: openstack at lists.launchpad.net<mailto:openstack at lists.launchpad.net>
Subject: [Openstack] Bottleneck of message queue


Hi guys,

I am trying to figure out how the internal interaction processes within different modules of OpenStack. Frankly speaking, while I'm reading the source codes I lost myself and have to jump out again to look at OpenStack from out of the box. I don't know if anybody has the similiar feeling with me. Is there any picture I can follow to see the message flows?

OpenStack is based on message queue to ensure the expansion easy. Here come my questions. Does anybody know the capacity of message queue? Would the capacity be a bottleneck of the platform?

Thanks,
Howard




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20121010/b9b62707/attachment.html>


More information about the Openstack mailing list