Open Stack

As this patch is only for discussion and direction check, so I didn't make effort to make Jekins success. Hope to get more feedback from community before I continue on it.

Thanks
--jyh

> -----Original Message-----
> From: Jiang, Yunhong [mailto:yunhong.jiang at intel.com]
> Sent: Tuesday, November 20, 2012 8:17 PM
> To: OpenStack Development Mailing List
> Subject: Re: [openstack-dev] [ceilometer] alternative publication conduits
> 
> I create a draft patch on https://review.openstack.org/#/c/16522/ .
> 
> Can anyone (Julien, Doug, Eoghan, or anyone else) have a look on it? It still an
> initial patch, and if the method is acceptable for you, I will continue the effort to
> make it a final one.
> 
> Thanks
> --jyh
> 
> > -----Original Message-----
> > From: Eoghan Glynn [mailto:eglynn at redhat.com]
> > Sent: Friday, October 26, 2012 9:34 PM
> > To: OpenStack Development Mailing List
> > Subject: [openstack-dev] [ceilometer] alternative publication conduits
> >
> >
> > Hi Ceilo Folks,
> >
> > TL;DR: just wanted to get feedback on some initial ideas on how
> > we might accommodate different conduits of publication from the
> > ceilo pollsters and notification handlers.
> >
> > Currently, we're obviously hard-coded to always publish via the
> > message bus, so the idea is to support alternative/additional
> > publication routes, according to the following outline:
> >
> >  - a ceilo notification handler or pollster can be associated
> >    via config with a chain of zero or more Publishers
> >
> >  - each Publisher knows now to emit their representation over some
> >    conduit (AMQP notification, CW PutMetricData call, statsd UDP
> >    packet, or whatever)
> >
> >  - a ceilometer notification handler or pollster provides a
> >    Transformer corresponding to each of its Publishers, which
> >    knows how to transform domain-specific raw data into the
> >    relevant representation
> >
> >  - the batching of transformer output is up to the Publisher
> >
> >  - each Publisher has an independent frequency, so not all
> >    Publishers are called to publish every measurement
> >
> > So for example the compute.libvirt.DiskIOPollster would provide a
> > transformer to map the raw disk stats reported by libvirt into
> > DiskWriteOps, DiskWriteBytes, DiskReadOps & DiskReadBytes metric
> > datapoints for CW, also indicating that this should be published
> > very minute. This CWPublisher would handle batching these data
> > as a single PutMetricData call.
> >
> > The DiskIOPollster would also provide a second Transformer to
> > mediate between the raw stats format and the ceilo Counter
> > representation for metering.  Additionally, this second Publisher
> > could be configured with a longer period (as befits metering data
> > likely to be consumed by a billing engine).
> >
> > Now the frequency aspect is a bit awkward. For notifications,
> > the frequency of the "exists" style of usage audit is out of our
> > hands, depending usually on the configuration of the emitter
> > (e.g. the nova instance_usage_audit_period config). Whereas ceilo
> > config direct controls only the frequency of polling. So one idea
> > would be for a publisher's frequency only to impact on the periods
> > under ceilo control - i.e. we'd accept and publish notifications
> > whenever we get them, whereas the polling cycle would be determined
> > by the greatest common divisor of all the configured publisher
> > periods for each set of related pollsters (the config would probably
> > have to be restricted so to ensure a sane GCD).
> >
> > Also, I'm assuming that an alternative publication route would
> > generally be used when the target endpoint is not the ceilo collector
> > & metering store ... i.e. non-standard publisher would most often
> > be used in order to hive off the data to an alternative endpoint
> > or store, such as a CW metrics store, or say to be dumped as RRD
> > files to be consumed by graphite.
> >
> > Thoughts?
> >
> > Cheers,
> > Eoghan
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev