[openstack-dev] [OSLO][RPC] AMQP / ZeroMQ control_exchange vs port numbers

Doug Hellmann doug.hellmann at dreamhost.com
Mon Apr 29 13:28:42 UTC 2013


On Mon, Apr 29, 2013 at 6:47 AM, Mark McLoughlin <markmc at redhat.com> wrote:

> On Wed, 2013-04-24 at 10:42 -0400, Russell Bryant wrote:
> > On 04/24/2013 10:35 AM, Julien Danjou wrote:
> > > On Wed, Apr 24 2013, Russell Bryant wrote:
> > >
> > >> On 04/24/2013 10:10 AM, Eric Windisch wrote:
> > >>> Sending aside sending, Ceilometer is already doing consumption
> > >>> cross-project, receiving notifications from multiple
> > >>> control_exchanges (at least where amqp is concerned).
> > >>
> > >> Yes, I'm aware that it's consuming notifications from multiple
> > >> projects. The patch in question is about enabling *sending* cross
> > >> project. That's what I'm curious about.
> > >
> > > The use case here is to send metrics to the Ceilometer collector
> > > directly from other projects using openstack.common.rpc. To do
> > > that, you have to send a message… on another control exchange.
> > >
> > > As Eric points out, this is typically the same use case we already
> > > have, just put in the opposite way: not having the Ceilometer
> > > collector joining a bunch of other control exchange (that it may
> > > not know about in advance), but having each project send its meters
> > > to the right place directly (Ceilometer's control exchange).
> >
> > That seems like a step backwards to me.  Right now we have a nice
> > generic notification mechanism that can be consumed by ceilometer or
> > something else.  This would be a move toward coupling all projects
> > with ceilometer specifically.  Is there something problematic with
> > consuming notifications as they are today?
>
> Yeah, this is an interesting subject. I'm seeing a few different points:
>
>  * RPCs should be for intra project communication
>
>  * Notifications should be used for where you want to make event
>    notifications available for third parties
>
>  * Just because the RPC notification driver needs to be able to do
>    something, doesn't mean we also allow other users of the RPC API to
>    do the same
>
>  * A ceilometer notifications driver is part of the ceilometer project,
>    so it sending RPCs back to ceilometer-collector is fine
>
>  * There are probably reasonable use cases (like a ceilometer
>    notifications plugin used by nova, but also cells) for a service to
>    connect to another broker and/or use a different exchange. Even
>    without those use cases, though, I think it would be lame for the
>    API to only allow such things to be specified through global
>    configuration ... global configuration should be a convenience
>    default only.
>

Agreed. I would prefer it if we could establish an API that didn't use the
global configuration object at all.


>
> However, I'd really like to understand this ceilometer use case better.
> See:
>
>
> https://wiki.openstack.org/wiki/Oslo/Messaging#Ceilometer_Metering_Messages
>
> Ceilometer is sending record_metering_data casts on the 'metering'
> topic. The ceilometer-collector service consumes these, but it uses
> create_worker() so that it doesn't use the default queue to receive
> them. It's like we're supporting the existence of third party consumers
> of these messages which would be peers with ceilometer-collector. Maybe
> ceilometer-collector consuming them is intended to be optional?
>

Yes, we originally designed ceilometer so that someone could tie into it at
any point for integrating with their existing system. I don't know how
practical that is in reality, but at the time we were very focused on
flexibility.


>
> I'm mostly curious why 'record_metering_data' isn't a notification if we
> want to support third party consumers of it.
>

It was, originally, but create_worker() uses the RPC dispatcher so we had
to change. Now that we have join_consumer_pool() in the public API, we
could change back.

Doug


>
> Cheers,
> Mark.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130429/6281bb9f/attachment.html>


More information about the OpenStack-dev mailing list