[Openstack] [metering] Choice of a messaging queue

Doug Hellmann doug.hellmann at dreamhost.com
Fri May 18 15:17:35 UTC 2012


On Fri, May 18, 2012 at 11:02 AM, Eric Windisch <eric at cloudscaling.com>wrote:

> The nova rpc implementation is moving into openstack common, I agree with
> using this abstraction.
>

That's a good point, I forgot about that.


>
> As per ZeroMQ, I'm the author of that plugin. There is a downloadable
> plugin for Essex and I'm preparing to make a Folsom merge prop within the
> next week or so, if all goes well.
>

Excellent!


>
> Sent from my iPad
>
> On May 18, 2012, at 7:26, Doug Hellmann <doug.hellmann at dreamhost.com>
> wrote:
>
>
>
> On Fri, May 18, 2012 at 4:42 AM, Nick Barcet <nick.barcet at canonical.com>wrote:
>
>> Hello everyone,
>>
>> Next week's irc meeting will have for goal to choose a reference
>> messaging queue service for the ceilometer project.  For this meeting to
>> be able to be successful, a discussion on the choices that we have to
>> make need to occur first right here.
>>
>> To open the discussion here are a few requirements that I would consider
>> important for the queue to support:
>>
>> a) the queue must guaranty the delivery of messages.
>> To the contrary of monitoring, loss of events may have important billing
>> impacts, it therefore cannot be an option that message be lost.
>>
>> b) the client should be able to store and forward.
>> As the load of system or traffic increases, or if the client is
>> temporarily disconnected, client element of the queue should be able to
>> hold messages in a local queue to be emitted as soon as condition permit.
>>
>> c) client must authenticate
>> Only client which hold a shared private key should be able to send
>> messages on the queue.
>>
>
> Does the username/password authentication of rabbitmq meet this
> requirement?
>
>
>>
>> d) queue may support client signing of individual messages
>> Each message should be individually signed by the agent that emits it in
>> order to guaranty non repudiability.  This function can be done by the
>> queue client or by the agent prior to en-queuing of messages.
>>
>
> We can embed the message signature in the message, so this requirement
> shouldn't have any bearing on the bus itself. Unless I'm missing something?
>
>
>>
>> d) queue must be highly available
>> the queue servers must be able to support multiple instances running in
>> parallel in order to support continuation of operations with the loss of
>> one server.  This should be achievable without the need to use complex
>> fail over systems and shared storage.
>>
>> e) queue should be horizontally scalable
>> The scalability of queue servers should be achievable by increasing the
>> number of servers.
>>
>> Not sure this list is exhaustive or viable, feel free to comment on it,
>> but the real question is: which queue should we be using here?
>>
>
> While I see the benefit of discussing requirements for the message bus
> platform in general, I'm not sure we need to dictate a specific
> implementation. If we say we are going to use the nova RPC library to
> communicate with the bus for sending and receiving messages, then we can
> use all of the tools for which there are drivers in nova -- rabbit, qpid,
> zeromq (assuming someone releases a driver, which I think is being worked
> on somewhere), etc. This leaves the decision of which bus to use up to the
> deployer, where the decision belongs. It also means we won't end up
> choosing a tool for which the other projects have no driver, leading us to
> have to create one and add a new dependency to the project.
>
> Doug
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120518/581477dd/attachment.html>


More information about the Openstack mailing list