[Openstack] [nova] why does notification use a "topic" exchange instead of "fanout"?

Day, Phil philip.day at hp.com
Wed May 9 08:49:49 UTC 2012


Hi Doug,

> I think you missed my main point, which was that a topic exchange does
> not impose a limitation that only one client can consume a given
> notification.  That's only true if each client is consuming from the
> same queue bound to the exchange.

So just to be clear, if I understand you correctly within the nova service/rpc abstraction layers the code is set up so that all services do bind to the same queue, and hence we get the round-robin delivery.
But, if someone wanted to write a separate notification consumer so that they didn't block anyone else from seeing the same messages then they (the consumer) should create a new queue on the existing topic exchange.

Is that correct - and is there any worked example of doing this ?

I thought within the nova code both the exchange and topic queues were set up by the consumer (so for example all compute_managers try to create the "compute" exchange and topic queue, but its only created by the first one and the others connect to the same queue).   In that context I'm finding it hard to see how to change this model to have multiple "notify.info" topic queues into the same exchange ?

Cheers,
Phil




From: openstack-bounces+philip.day=hp.com at lists.launchpad.net [mailto:openstack-bounces+philip.day=hp.com at lists.launchpad.net] On Behalf Of Doug Hellmann
Sent: 08 May 2012 23:34
To: Russell Bryant
Cc: openstack at lists.launchpad.net
Subject: Re: [Openstack] [nova] why does notification use a "topic" exchange instead of "fanout"?


On Tue, May 8, 2012 at 6:04 PM, Russell Bryant <rbryant at redhat.com<mailto:rbryant at redhat.com>> wrote:
On 05/08/2012 05:59 PM, Doug Hellmann wrote:
>     Here is a relevant section pulled out of the amqp 0-9-1 spec:
>
>        3.1.3.3 The Topic Exchange Type
>
>        The topic exchange type works as follows:
>
>            1. A message queue binds to the exchange using a routing
>               pattern, P.
>            2. A publisher sends the exchange a message with the routing
>               key R.
>            3. The message is passed to the message queue if R matches P.
>
>        The routing key used for a topic exchange MUST consist of zero or
>        more words delimited by dots. Each word may contain the letters A-Z
>        and a-z and digits 0-9.
>
>        The routing pattern follows the same rules as the routing key with
>        the addition that * matches a single word, and # matches zero or
>        more words. Thus the routing pattern *.stock.# matches the routing
>        keys usd.stock and eur.stock.db but not stock.nasdaq.
>
>     In nova, for a given topic such as 'scheduler', all of the consumers are
>     binding to the same queue on the topic exchange, resulting in
>     round-robin delivery to each of the consumers.  If instead you make a
>     new queue, you can get your own copy of each message.
>
>     There is an additional benefit of using a topic exchange here.  The
>     topic used for notifications is 'notifications.<priority>'.  That means
>     that when you create your queue, you can set it up to receive all
>     notifications, or only notifications of a certain priority.
>
>
> Topic exchanges make a lot of sense for messages that should only be
> consumed once, such as tasks. Notifications are different. Lots of
> different clients might want to know that some event happened in the
> system. The way things are in Nova today, they can't. The first client
> who consumes a notification message will prevent all of the other
> clients from seeing that message at all.
I think you missed my main point, which was that a topic exchange does
not impose a limitation that only one client can consume a given
notification.  That's only true if each client is consuming from the
same queue bound to the exchange.

Yes, that wasn't obvious from any of the kombu documentation I've seen so far. I'll keep looking.

Thanks,
Doug


> I can change Nova's notification system to use a fanout exchange (in
> impl_kombu.py changing the exchange type used by NotifyPublisher), but
> before I submit a patch I want to make sure the current implementation
> using a topic exchange wasn't selected deliberately for some reason.
I think using a fanout exchange would be a downgrade.  As I mentioned
before, a topic exchange allows you to create a queue to get all
notifications or only notifications of a specific priority.  If the
exchange type is changed to fanout, it's everybody gets everything, and
that's it.

--
Russell Bryant

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120509/90be80b3/attachment.html>


More information about the Openstack mailing list