Open Stack

On 12/09/2013 10:37 PM, Russell Bryant wrote:
> On 12/09/2013 05:16 PM, Gordon Sim wrote:
>> On 12/09/2013 07:15 PM, Russell Bryant wrote:
>> Understood. The Dispatch Router was indeed created from an understanding
>> of the limitations and drawbacks of the 'federation' feature of qpidd
>> (which was the primary mechanism for scaling beyond one broker) as well
>> learning lessons around the difficulties of message replication and
>> storage.
>
> Cool.  To make the current situation worse, AFAIK, we've never been able
> to make Qpid federation work at all for OpenStack.  That may be due to
> the way we use Qpid, though.

The federation in qpidd requires path specific configuration. I.e. for 
each topic or queu or whatever you need to explicitly enable the flow of 
messages in each direction between each pair of brokers.

The original uses cases it was designed for were much simpler than 
openstacks needs and this wasn't insurmountable. As it was used in new 
areas however the limitation became apparent. The Dispatch Router 
instances on the other hand communicate with each other to automatically 
setup the internal routing necessary to ensure publishers and 
subscribers communicate regardless of the point at which they are attached.

Another limitation of the original federation was the inability to 
handle redundant routes between brokers without duplicating messages.

[...]
>> The Dispatch Router can work for pub-sub patterns as well, though not
>> store and forward directly. In theory, for flows where store-and-forward
>> is needed, that can be supplied by an additional service e.g. a more
>> traditional broker, which would take responsibility for replaying over
>> from the publisher in order that subscribers could if needed have
>> message replayed even after the original publisher had exited.
>
> Any thoughts on what we would be recommending for notifications?

For basic pub-sub (i.e. no durable/reliable subscriptions that retain 
messages across disconnects), dispatch router itself will work fine.

For those cases where you do need store-and-forward, i.e. where 
something other than the publisher needs to reliably store notifications 
until all interested subscribers have confirmed receipt, you could use a 
broker (rabbitmq, qpidd, activemq or similar).

The plan for dispatch router is to allow such brokers to be hooked into 
the router network also. So you could have the broker(s) accept the 
messages from the publishers, and then have any subscribers needing 
reliability guarantees subscribe to the broker via the router network. 
The aim is to allow the load to be spread across multiple brokers, but 
have dispatch router control the flow of messages in and out of these, 
rather than relying on the brokers to link themselves up. This aspect 
however is not yet built (but is coming soon!).

Durable/reliable subscriptions do of course bring with them the need to 
manage potential growth of backed up, unconfirmed messages, e.g. when a 
subscriber is down or unresponsive for some time. Ensuring that there 
are limits in place that prevent this getting out of hand are essential.