Open Stack

On 10/15/2013 12:28 PM, Neal, Phil wrote:
> 
> 
>> -----Original Message-----
>> From: Sandy Walsh [mailto:sandy.walsh at rackspace.com]
>> Sent: Thursday, October 10, 2013 6:20 PM
>> To: openstack-dev at lists.openstack.org
>> Subject: Re: [openstack-dev] [Ceilometer] Notifications from non-local
>> exchanges
>>
>>
>> On 10/10/2013 06:16 PM, Neal, Phil wrote:
>>> Greetings all, I'm looking at how to expand the ability of our CM
>>> instance to consume notifications and have a quick question about
>>> the configuration and flow...
>>>
>>> For the notifications central agent ,  we rely on the services (i.e.
>>> glance, cinder)  to drop messages on the same messaging host as used
>>> by Ceilometer. From there the listener picks it up and cycles through
>>> the plugin logic to convert it to a sample. It's apparent that we
>>> can't pass an alternate hostname via the control_exchange values, so
>>> is there another method for harvesting messages off of other
>>> instances (e.g. another compute node)?
>>
>> Hey Phil,
>>
>> You don't really need to specify the exchange name to consume
>> notifications. It will default to the control-exchange if not specified
>> anyway.
>>
>> How it works isn't so obvious.
>>
>> Depending on the priority of then notification the oslo notifier will
>> publish on <topic>.<priority> using the service's control-exchange. If
>> that queue doesn't exist it'll create it and bind the control-exchange
>> to it. This is so we can publish even if there are no consumers yet.
> 
> I think the common default is notifications.info, yes?
> 
>>
>> Oslo.rpc creates a 1:1 mapping of routing_key and queue to topic (no
>> wildcards). So we get
>>
>> <exchange:<service>> -> binding: routing_key "<topic>.<priority>" ->
>> queue "<topic>.<priority>"
>>
>> (essentially, 1 queue per priority)
>>
>> Which is why, if you want to enable services to generate notifications,
>> you just have to set the driver and the topic(s) to publish on. Exchange
>> is implied and routing key/queue are inferred from topic.
> 
> Yep, following up to this point: Oslo takes care of the setup of exchanges on behalf of the 
> services. When, say, Glance wants to push notifications onto the message bus, they can set 
> the control_exchange value and the driver ("rabbit", for example) and voila! An exchange is
> set up with a default queue bound to the key. 

Correct.

>>
>> Likewise we only have to specify the queue name to consume, since we
>> only need an exchange to publish.
> 
> Here's where my gap is: the notification plugins seem to assume that Ceilometer 
> is sitting on the same messaging node/endpoint as the service. The config file allows
> us to specify the exchange names for the services , but not endpoints, so if Glance 
> is publishing to "notifications.info" on "rabbit.glance.hpcloud.net", and ceilometer
>  is  publishing/consuming from the "rabbit.ceil.hpcloud.net" node then the Glance
>  notifications won't be collected.

Hmm, I think I see your point. All the rabbit endpoints are determined
by these switches:
https://github.com/openstack/nova/blob/master/etc/nova/nova.conf.sample#L1532-L1592

We will need a way in CM to pull from multiple rabbits.

>  I took another look at the Ceilometer config options..."rabbit_hosts"
> takes multiple hosts (i.e. rabbit.glance.hpcloud.net:0000, rabbit.ceil.hpcloud.net:0000) 
> but it's not clear whether that's for publishing, collection, or both?  The impl_kombu
> module does cycle through that list to create the connection pool, but it's not
> clear to me how it all comes together in the plugin instantiation...

Nice catch. I'll have a look at that as well.

Regardless, I think CM should have separate switches for each collector
we run and break out the consume rabbit from the service rabbit.

I may be in a position to work on this shortly if that's needed.

> 
>>
>> I have a bare-bones oslo notifier consumer and client here if you want
>> to mess around with it (and a bare-bones kombu version in the parent).
> 
> Will take a look! 
> 
>>
>> https://github.com/SandyWalsh/amqp_sandbox/tree/master/oslo
>>
>> Not sure if that answered your question or made it worse? :)
>>
>> Cheers
>> -S
>>
>>
>>>
>>>
>>> - Phil
>>>
>>> _______________________________________________ OpenStack- dev mailing
>>> list OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>