Open Stack

On Tue, Jan 8, 2013 at 2:59 AM, Jiang, Yunhong <yunhong.jiang at intel.com>wrote:

> >               There are two type potential relationship between counters.
> > One is counters from the same pollster, another one is counters from
> different
> > pollsters. I don't think we need consider counters from notification
> event (am I
> > right?).
> >
> >
> >
> > We will want all of the data published by ceilometer to be processed in
> the
> > same way, so the notification data will need to go through the
> pipelines, too.
>
> I mean we don't need consider relationship from different notifications,
> sorry for not clear.
>

Ah, yes, you're right.


>
> >
> > Aggregation of data between counters is an implementation detail of the
> > transformer or publisher that needs the aggregation, so I don't think we
> want
> > to handle it by grouping data in our main loop. For one thing, we can't
> predict
> > all of the ways the different transformers will want to aggregate data
> (by
> > instance, by project, by something like the public IP address of a load
> balanced
> > service, etc.). A more general solution would be to give the pipeline a
> signal
> > that there is no more data coming -- a flush() call, or something
> similar -- and
> > let the transformer handle caching data internally until it sees the end
> of the
> > data.
>
> For this NetPollster specific issue, I'd take Eglyn's suggestion to
> calculate it in the source pollster.
>

That moves the information about the aggregation very far from the source
that is going to consume it. It also means we have to do something similar
for all of the other cases. We have the information we need to let the
transformers aggregate the data, we should do it there.


>
> Generally, I think we can add or something like notification() or flulsh()
> to the pipeline interface in future. The notification can be information
> like instance closed with instance_id, IP address released etc. The
> pipeline pass such information down to the transformers. By default, the
> transformer will simply ignore such information.
>

In order to be generic, we won't want to include any extra details. Think
about it like a commit() call on a database handle or flush() on a file
handle. Each pipeline should be flushed after all counter data has been
sent to it. We wake up on the timer event, collect the data we need for all
pipelines, push the data into the pipelines, then call flush to say "no
more data is coming right now." Managing any contextual information used to
aggregate the data (like the instance id) is the responsibility of the
transformer or publisher.

Doug


>
> Thanks
> --jyh
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130108/c79d1966/attachment.html>