[openstack-dev] [Openstack] [Ceilometer][Architecture] Transformers in Kilo vs Liberty(and Mitaka)
cdent+os at anticdent.org
Tue Apr 12 14:21:14 UTC 2016
This discussion needs to be happening on openstack-dev too, so
cc'ing that list in as well. The top of the thread is at
On Tue, 12 Apr 2016, Chris Dent wrote:
> On Tue, 12 Apr 2016, Nadya Shakhat wrote:
>> I'd like to discuss one question with you. Perhaps, you remember that
>> in Liberty we decided to get rid of transformers on polling agents . I'd
>> like to describe several issues we are facing now because of this decision.
>> 1. pipeline.yaml inconsistency.
> The original idea with centralizing the transformers to just the
> notification agents was to allow a few different things, only one of
> which has happened:
> * Make the pollster code less complex with few dependencies,
> easing deployment options (this has happened), maintenance
> and custom pollsters.
> With the transformers in the pollsters they must maintain a
> considerable amount of state that makes effective use of eventlet
> (or whatever the chosen concurrent solution is) more difficult.
> The ideal pollster is just something that spits out a dumb piece
> of identified data every interval. And nothing else.
> * Make it far easier to use and conceptualize the use of pollsters
> outside of the ceilometer environment as simple data collectors.
> In that context transformation would occur only close to the data
> consumption not at the data production.
> This, following the good practice of services doing one thing
> * Migrate away from the pipeline.yaml that conflated sources and
> sinks to a model that is good both for computers and humans:
> * sources over here
> * sinks over here
> That these other things haven't happened means we're in an awkward
> Are the options the following?
> * Do what you suggest and pull transformers back into the pollsters.
> Basically revert the change. I think this is the wrong long term
> solution but might be the best option if there's nobody to do the
> other options.
> * Implement a pollster.yaml for use by the pollsters and consider
> pipeline.yaml as the canonical file for the notification agents as
> there's where the actual _pipelines_ are. Somewhere in there kill
> interval as a concept on pipeline side.
> This of course doesn't address the messaging complexity. I admit
> that I don't understand all the issues there but it often feels
> like we are doing that aspect of things completely wrong, so I
> would hope that before we change things there we consider all the
> What else?
> One probably crazy idea: What about figuring out the desired end-meters
> of common transformations and making them into dedicated pollsters?
> Encapsulating that transformation not at the level of the polling
> manager but at the individual pollster.
Chris Dent (╯°□°)╯︵┻━┻ http://anticdent.org/
freenode: cdent tw: @anticdent
More information about the OpenStack-dev