Open Stack

One pipeline to multiple publisher was discussed before, and the reason is possibly there are multiple similar publisher, and the only difference is different conduit (udp, rpc etc). I'm not sure if this is really required, but seems quite reasonable.

But I agree that the interval is a property of pipeline, and all counters flow through the pipeline should have the same interval. If we do have requirement for per-counter interval, that can be achieved through a separated pipeline. And if two similar publisher want different interval, create separated pipeline. I will update the pipeline framework to support interval in the pipeline config file.

And I will work as your suggestion for the main loop.

BTW, this in fact requires an interface to get the counters supported by a pollster, and I hope it's ok for everyone.

Thanks
--jyh

From: Doug Hellmann [mailto:doug.hellmann at dreamhost.com]
Sent: Tuesday, January 08, 2013 1:40 AM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [ceilometer] Counter frequency bp discussion


On Sat, Jan 5, 2013 at 8:03 AM, Jiang, Yunhong <yunhong.jiang at intel.com<mailto:yunhong.jiang at intel.com>> wrote:
Hi, Doug
        As a follow--up with you on the counter frequency discussion, I create a wiki page at http://wiki.openstack.org/Ceilometer/blueprints/publisher-counters-frequency . Please have a look on it.

This feels a little more complicated than should be necessary to meet the requirements we have.

First, lets constrain each pipeline to go to a single publisher. The transformers have mostly been described as publisher-specific anyway (formatting the data in a particular way, for example), so I think in practice we are likely to only have one publisher per pipeline anyway. If this proves to be wrong, we can change it later to allow more than one publisher, but starting with one will give us an easier target to hit now.

Second, the interval should be a property of the pipeline rather than a transformer or counter. There are two reasons for that change: (1) each pipeline must have an interval, but if we leave it up to the transformer a user may configure a pipeline without that type of transformer in place and (2) all of the data flowing through the pipeline (the counters) should be collected at the same interval to give the pipeline a simple operating model for the user to understand it ("every N seconds this pipeline collects data from these sources and publishes it to this destination").

If we make those changes, then we only need to schedule timers for each unique interval value. When a timer goes off, data for the counters for all of the pipelines with that interval can be collected and sent into the appropriate pipelines. We can do some caching to avoid collecting the same data over and over if multiple pipelines want the same counters, but that's an optimization we can make after we have the basic logic working correctly.

Doug

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130108/3ad0279d/attachment.html>