<br><br><div class="gmail_quote">On Fri, Oct 26, 2012 at 9:59 AM, Julien Danjou <span dir="ltr"><<a href="mailto:julien@danjou.info" target="_blank">julien@danjou.info</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Fri, Oct 26 2012, Eoghan Glynn wrote:<br>
<br>
Hi Eoghan,<br>
<div class="im"><br>
> So for example the compute.libvirt.DiskIOPollster would provide a<br>
> transformer to map the raw disk stats reported by libvirt into<br>
> DiskWriteOps, DiskWriteBytes, DiskReadOps & DiskReadBytes metric<br>
> datapoints for CW, also indicating that this should be published<br>
> very minute. This CWPublisher would handle batching these data<br>
> as a single PutMetricData call.<br>
><br>
> The DiskIOPollster would also provide a second Transformer to<br>
> mediate between the raw stats format and the ceilo Counter<br>
> representation for metering. Additionally, this second Publisher<br>
> could be configured with a longer period (as befits metering data<br>
> likely to be consumed by a billing engine).<br>
<br>
</div>I don't think it's up to the Pollster to provide Transformers. If we go<br>
on that road, I'm afraid it complicates a lot the Pollster writing.<br>
<br>
Why wouldn't each publisher be in charge to convert the Counters yielded<br>
by the Pollsters using whatever their own Transformer(s)?<br></blockquote><div><br></div><div>I agree, pollsters and publishers shouldn't have to know about each other. We should try to make Counter() objects the common data object used on both sides of the transformation. Notification listeners and pollsters should emit Counter instances and Publishers should take Counters and send out formatted data. If the current definition of a Counter doesn't support all of our use cases, we should change it.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> Now the frequency aspect is a bit awkward. For notifications,<br>
> the frequency of the "exists" style of usage audit is out of our<br>
> hands, depending usually on the configuration of the emitter<br>
> (e.g. the nova instance_usage_audit_period config). Whereas ceilo</div></blockquote><div><br></div><div>I want to, eventually, change the auditing stuff from of the other projects because it doesn't support our use cases. I haven't had a chance to study what use cases it really does support to ensure that Ceilometer would meet those needs in the future, so for now, we can mostly ignore them. Quantum didn't have anything like that, so we added it in a way that works for us. Nova and Cinder share the same old implementation, but the polling the Ceilometer agent does makes up for the sparse event generation.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im"> </div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">
> config direct controls only the frequency of polling. So one idea<br>
> would be for a publisher's frequency only to impact on the periods<br>
> under ceilo control - i.e. we'd accept and publish notifications<br>
> whenever we get them, whereas the polling cycle would be determined<br>
> by the greatest common divisor of all the configured publisher<br>
> periods for each set of related pollsters (the config would probably<br>
> have to be restricted so to ensure a sane GCD).<br>
<br>
</div>That makes sense. Basically, your polling frequence should be<br>
automatically computed to be the min() of the frequency requested by all<br>
enabled Publishers.<br></blockquote><div><br></div><div>Do we need to protect against having the polling performed too frequently and consuming too many resources on the server?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
For notifications, the code rounting Counter to Publishers may/should<br>
filter out the Counters if they are likely to be emitted wrt with what a<br>
Publisher asked for.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Julien Danjou<br>
# Free Software hacker & freelance<br>
# <a href="http://julien.danjou.info" target="_blank">http://julien.danjou.info</a><br>
</font></span><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br>