Open Stack

On Fri, Oct 26, 2012 at 9:59 AM, Julien Danjou <julien at danjou.info> wrote:

> On Fri, Oct 26 2012, Eoghan Glynn wrote:
>
> Hi Eoghan,
>
> > So for example the compute.libvirt.DiskIOPollster would provide a
> > transformer to map the raw disk stats reported by libvirt into
> > DiskWriteOps, DiskWriteBytes, DiskReadOps & DiskReadBytes metric
> > datapoints for CW, also indicating that this should be published
> > very minute. This CWPublisher would handle batching these data
> > as a single PutMetricData call.
> >
> > The DiskIOPollster would also provide a second Transformer to
> > mediate between the raw stats format and the ceilo Counter
> > representation for metering.  Additionally, this second Publisher
> > could be configured with a longer period (as befits metering data
> > likely to be consumed by a billing engine).
>
> I don't think it's up to the Pollster to provide Transformers. If we go
> on that road, I'm afraid it complicates a lot the Pollster writing.
>
> Why wouldn't each publisher be in charge to convert the Counters yielded
> by the Pollsters using whatever their own Transformer(s)?
>

I agree, pollsters and publishers shouldn't have to know about each other.
We should try to make Counter() objects the common data object used on both
sides of the transformation. Notification listeners and pollsters should
emit Counter instances and Publishers should take Counters and send out
formatted data. If the current definition of a Counter doesn't support all
of our use cases, we should change it.


>
> > Now the frequency aspect is a bit awkward. For notifications,
> > the frequency of the "exists" style of usage audit is out of our
> > hands, depending usually on the configuration of the emitter
> > (e.g. the nova instance_usage_audit_period config). Whereas ceilo
>

I want to, eventually, change the auditing stuff from of the other projects
because it doesn't support our use cases. I haven't had a chance to study
what use cases it really does support to ensure that Ceilometer would meet
those needs in the future, so for now, we can mostly ignore them. Quantum
didn't have anything like that, so we added it in a way that works for
us. Nova and Cinder share the same old implementation, but the polling the
Ceilometer agent  does makes up for the sparse event generation.


>
>
> config direct controls only the frequency of polling. So one idea
> > would be for a publisher's frequency only to impact on the periods
> > under ceilo control - i.e. we'd accept and publish notifications
> > whenever we get them, whereas the polling cycle would be determined
> > by the greatest common divisor of all the configured publisher
> > periods for each set of related pollsters (the config would probably
> > have to be restricted so to ensure a sane GCD).
>
> That makes sense. Basically, your polling frequence should be
> automatically computed to be the min() of the frequency requested by all
> enabled Publishers.
>

Do we need to protect against having the polling performed too frequently
and consuming too many resources on the server?


>
> For notifications, the code rounting Counter to Publishers may/should
> filter out the Counters if they are likely to be emitted wrt with what a
> Publisher asked for.
>
> --
> Julien Danjou
> # Free Software hacker & freelance
> # http://julien.danjou.info
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121026/d1c55f79/attachment.html>