Open Stack

On Wed, May 9, 2012 at 3:17 PM, Tihomir Trifonov <t.trifonov at gmail.com>wrote:

> Hi Doug,
>
> not sure if you've hit the same problem, but I've spent some time on that
> when I started using RabbitMQ. As I see from the example, you've provided:
>
> queue = Queue(name='notifications.info',
>
>                   exchange=Exchange(name='nova'...
>
>
>
> So you set explicitly a name for the queue. If you have two queues
> declared with the same name, when they are bound to an Exchange, actually
> each message is received only by the one queue at a time. Just leave the
> name field empty(it will be auto-generated), and each queue will receive
> its copy of the message. So the logic is that the first queue with that
> name acknowledges the message, and the other one receives nothing.
>
>
> Besides that, the topics are more powerful and handy to use than fanout.
>

Now that I see how it works, I agree. :-)


>
>
>
> On Wed, May 9, 2012 at 6:34 PM, Doug Hellmann <doug.hellmann at dreamhost.com
> > wrote:
>
>>
>>
>> On Wed, May 9, 2012 at 6:42 AM, Kiall Mac Innes <kiall at managedit.ie>wrote:
>>
>>> Kinda! The queue has a name, but that name has no bearing on the set of
>>> messages received.
>>
>>
>>>  If you create a queue called "MyCustomNotificationQueue", you can bind
>>> that to the "notifications" exchange using the "notifications.info"
>>> routing key.
>>>
>>> (I'm guessing some of the names here.. I know AMQP, and not the specific
>>> naming nova uses!)
>>>
>>
>> notifications.info is right.
>>
>>
>>>
>>> Nova just happens to use the same queue name and routing key. I believe
>>> this is causing the confusion.
>>>
>>
>> Exactly. I ended up creating a separate queue for each client that I have
>> and setting them to auto-delete when the client disconnects. That way I can
>> have as many clients connecting and listening as I want. The code is in
>> https://github.com/dhellmann/metering-prototype if you want to take a
>> look.
>>
>>
>>>
>>
>>>
>>> Does this make sense?
>>>
>>> Anyway - The RabbitMQ docs probably explain it better than I..
>>> http://www.rabbitmq.com/tutorials/tutorial-five-python.html
>>>
>>> Thanks,
>>> Kiall
>>>
>>>
>>>
>>> On Wed, May 9, 2012 at 11:30 AM, Day, Phil <philip.day at hp.com> wrote:
>>>
>>>> OK, get that so far – so both consumers need to declare and use the
>>>> same exchange.****
>>>>
>>>> ** **
>>>>
>>>> But If I understand the next step right, to get multiple consumers of
>>>>  info notification messages they would all need to create separate “
>>>> notifications.info” queues into that exchange.    And isn’t that
>>>> exactly what Nova currently does to create a shared queue ?****
>>>>
>>>> ** **
>>>>
>>>> Phil****
>>>>
>>>> ** **
>>>>
>>>> *From:* Kiall Mac Innes [mailto:kiall at managedit.ie]
>>>> *Sent:* 09 May 2012 10:51
>>>> *To:* Day, Phil
>>>> *Cc:* openstack at lists.launchpad.net; Russell Bryant; Doug Hellmann
>>>>
>>>> *Subject:* Re: [Openstack] [nova] why does notification use a "topic"
>>>> exchange instead of "fanout"?****
>>>>
>>>> ** **
>>>>
>>>> Your own queue listener should attempt to declare the exchange, using
>>>> the same settings as Nova does. ****
>>>>
>>>> If the exchange exists, its a noop. Otherwise it's created for you.****
>>>>
>>>> After that, if you start up Nova, it will do the same and reuse your
>>>> exchange.****
>>>>
>>>> Obviously this works both ways, and either nova or your code can
>>>> declare the exchange.****
>>>>
>>>> AMQP is designed to be a configuration-less protocol, where resources
>>>> are configured by the first consumer attempting to use them.****
>>>>
>>>> Thanks,
>>>> Kiall****
>>>>
>>>> Sent from my phone.****
>>>>
>>>> On May 9, 2012 9:52 a.m., "Day, Phil" <philip.day at hp.com> wrote:****
>>>>
>>>> Hi Doug,****
>>>>
>>>>  ****
>>>>
>>>> > I think you missed my main point, which was that a topic exchange does
>>>> > not impose a limitation that only one client can consume a given
>>>> > notification.  That's only true if each client is consuming from the
>>>> > same queue bound to the exchange.****
>>>>
>>>>  ****
>>>>
>>>> So just to be clear, if I understand you correctly within the nova
>>>> service/rpc abstraction layers the code is set up so that all services do
>>>> bind to the same queue, and hence we get the round-robin delivery.****
>>>>
>>>> But, if someone wanted to write a separate notification consumer so
>>>> that they didn’t block anyone else from seeing the same messages then they
>>>> (the consumer) should create a new queue on the existing topic exchange.
>>>> ****
>>>>
>>>>  ****
>>>>
>>>> Is that correct – and is there any worked example of doing this ?****
>>>>
>>>>  ****
>>>>
>>>> I thought within the nova code both the exchange and topic queues were
>>>> set up by the consumer (so for example all compute_managers try to create
>>>> the “compute” exchange and topic queue, but its only created by the first
>>>> one and the others connect to the same queue).   In that context I’m
>>>> finding it hard to see how to change this model to have multiple “
>>>> notify.info” topic queues into the same exchange ?****
>>>>
>>>>  ****
>>>>
>>>> Cheers,****
>>>>
>>>> Phil****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>> *From:* openstack-bounces+philip.day=hp.com at lists.launchpad.net[mailto:
>>>> openstack-bounces+philip.day=hp.com at lists.launchpad.net] *On Behalf Of
>>>> *Doug Hellmann
>>>> *Sent:* 08 May 2012 23:34
>>>> *To:* Russell Bryant
>>>> *Cc:* openstack at lists.launchpad.net
>>>> *Subject:* Re: [Openstack] [nova] why does notification use a "topic"
>>>> exchange instead of "fanout"?****
>>>>
>>>>  ****
>>>>
>>>>  ****
>>>>
>>>> On Tue, May 8, 2012 at 6:04 PM, Russell Bryant <rbryant at redhat.com>
>>>> wrote:****
>>>>
>>>> On 05/08/2012 05:59 PM, Doug Hellmann wrote:
>>>> >     Here is a relevant section pulled out of the amqp 0-9-1 spec:
>>>> >
>>>> >        3.1.3.3 The Topic Exchange Type
>>>> >
>>>> >        The topic exchange type works as follows:
>>>> >
>>>> >            1. A message queue binds to the exchange using a routing
>>>> >               pattern, P.
>>>> >            2. A publisher sends the exchange a message with the
>>>> routing
>>>> >               key R.
>>>> >            3. The message is passed to the message queue if R matches
>>>> P.
>>>> >
>>>> >        The routing key used for a topic exchange MUST consist of zero
>>>> or
>>>> >        more words delimited by dots. Each word may contain the
>>>> letters A-Z
>>>> >        and a-z and digits 0-9.
>>>> >
>>>> >        The routing pattern follows the same rules as the routing key
>>>> with
>>>> >        the addition that * matches a single word, and # matches zero
>>>> or
>>>> >        more words. Thus the routing pattern *.stock.# matches the
>>>> routing
>>>> >        keys usd.stock and eur.stock.db but not stock.nasdaq.
>>>> >
>>>> >     In nova, for a given topic such as 'scheduler', all of the
>>>> consumers are
>>>> >     binding to the same queue on the topic exchange, resulting in
>>>> >     round-robin delivery to each of the consumers.  If instead you
>>>> make a
>>>> >     new queue, you can get your own copy of each message.
>>>> >
>>>> >     There is an additional benefit of using a topic exchange here.
>>>>  The
>>>> >     topic used for notifications is 'notifications.<priority>'.  That
>>>> means
>>>> >     that when you create your queue, you can set it up to receive all
>>>> >     notifications, or only notifications of a certain priority.
>>>> >
>>>> >
>>>> > Topic exchanges make a lot of sense for messages that should only be
>>>> > consumed once, such as tasks. Notifications are different. Lots of
>>>> > different clients might want to know that some event happened in the
>>>> > system. The way things are in Nova today, they can't. The first client
>>>> > who consumes a notification message will prevent all of the other
>>>> > clients from seeing that message at all.****
>>>>
>>>> I think you missed my main point, which was that a topic exchange does
>>>> not impose a limitation that only one client can consume a given
>>>> notification.  That's only true if each client is consuming from the
>>>> same queue bound to the exchange.****
>>>>
>>>>  ****
>>>>
>>>> Yes, that wasn't obvious from any of the kombu documentation I've seen
>>>> so far. I'll keep looking.****
>>>>
>>>>  ****
>>>>
>>>> Thanks,****
>>>>
>>>> Doug****
>>>>
>>>>  ****
>>>>
>>>>
>>>> > I can change Nova's notification system to use a fanout exchange (in
>>>> > impl_kombu.py changing the exchange type used by NotifyPublisher), but
>>>> > before I submit a patch I want to make sure the current implementation
>>>> > using a topic exchange wasn't selected deliberately for some reason.*
>>>> ***
>>>>
>>>> I think using a fanout exchange would be a downgrade.  As I mentioned
>>>> before, a topic exchange allows you to create a queue to get all
>>>> notifications or only notifications of a specific priority.  If the
>>>> exchange type is changed to fanout, it's everybody gets everything, and
>>>> that's it.
>>>>
>>>> --
>>>> Russell Bryant****
>>>>
>>>>  ****
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to     : openstack at lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help   : https://help.launchpad.net/ListHelp****
>>>>
>>>
>>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack at lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Regards,
> Tihomir Trifonov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120509/835b018e/attachment.html>