Open Stack

On Fri, Oct 26, 2012 at 11:58 AM, Eoghan Glynn <eglynn at redhat.com> wrote:

>
>
> > I agree, pollsters and publishers shouldn't have to know about each
> > other. We should try to make Counter() objects the common data
> > object used on both sides of the transformation. Notification
> > listeners and pollsters should emit Counter instances and Publishers
> > should take Counters and send out formatted data. If the current
> > definition of a Counter doesn't support all of our use cases, we
> > should change it.
>
> Well, what I was trying to avoid there was bloating up the Counter
> with fields that are only of interest to a subset of publishers.
> So the idea was to just do a direct transformation from the raw data
> to the representation most appropriate for the associated publisher.
> It seemed to introduce less "impedance" to just go directly from
> raw data to say a type like boto.ec2.cloudwatch.datapoint.Datapoint,
> instead of using the ceilo Counter type as an intermediate step.
>
> The reason for avoiding building that logic into the publisher is
> to split off knowledge (say in the cloudwatch case) of the metric
> naming, units etc. specific to each type of raw data produced by a
> pollster or notification handler (so for ex. a CWPublisher wouldn't
> need to be concerned with the differences between CPU/network/disk/
> whatever stats).
>

I also don't want the sources of the data to have to be concerned about the
output format differences used by the different publishers, because I want
to make it as easy as possible to add new meters.

As you point out later in this thread, sources such as instrumented RPC
calls may not have all of the same information as ceilometer's existing
data sources. We may need different publishing APIs for sources that have
different levels of data (starting from a Counter vs. a "raw" event).

> I want to, eventually, change the auditing stuff from of the other
> > projects because it doesn't support our use cases. I haven't had a
> > chance to study what use cases it really does support to ensure that
> > Ceilometer would meet those needs in the future, so for now, we can
> > mostly ignore them. Quantum didn't have anything like that, so we
> > added it in a way that works for us. Nova and Cinder share the same
> > old implementation, but the polling the Ceilometer agent does makes
> > up for the sparse event generation.
>
> Cool enough.
>
>
> > Do we need to protect against having the polling performed too
> > frequently and consuming too many resources on the server?
>
> So the idea is to mitigate this by sharing polling cycles where possible.
>
> Say the metering publisher needs to sample every 10 minutes, whereas a
> metrics publisher needs a more fine-grain, say every 2 minutes. In that
> case, the yield from every fifth polling cycle will be shared and published
> by both.
>

Sure, that makes sense. What do we do if the user configures a pollster to
run every 10 seconds?

Doug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121029/9f8a65ca/attachment.html>