Open Stack

This discussion needs to be happening on openstack-dev too, so
cc'ing that list in as well. The top of the thread is at
http://lists.openstack.org/pipermail/openstack/2016-April/015864.html

On Tue, 12 Apr 2016, Chris Dent wrote:

> On Tue, 12 Apr 2016, Nadya Shakhat wrote:
>
>>    I'd like to discuss one question with you. Perhaps, you remember that
>> in Liberty we decided to get rid of transformers on polling agents [1]. I'd
>> like to describe several issues we are facing now because of this decision.
>> 1. pipeline.yaml inconsistency.
>
> The original idea with centralizing the transformers to just the
> notification agents was to allow a few different things, only one of
> which has happened:
>
> * Make the pollster code less complex with few dependencies,
>  easing deployment options (this has happened), maintenance
>  and custom pollsters.
>
>  With the transformers in the pollsters they must maintain a
>  considerable amount of state that makes effective use of eventlet
>  (or whatever the chosen concurrent solution is) more difficult.
>
>  The ideal pollster is just something that spits out a dumb piece
>  of identified data every interval. And nothing else.
>
> * Make it far easier to use and conceptualize the use of pollsters
>  outside of the ceilometer environment as simple data collectors.
>  In that context transformation would occur only close to the data
>  consumption not at the data production.
>
>  This, following the good practice of services doing one thing
>  well.
>
> * Migrate away from the pipeline.yaml that conflated sources and
>  sinks to a model that is good both for computers and humans:
>
>  * sources over here
>  * sinks over here
>
> That these other things haven't happened means we're in an awkward
> situation.
>
> Are the options the following?
>
> * Do what you suggest and pull transformers back into the pollsters.
>  Basically revert the change. I think this is the wrong long term
>  solution but might be the best option if there's nobody to do the
>  other options.
>
> * Implement a pollster.yaml for use by the pollsters and consider
>  pipeline.yaml as the canonical file for the notification agents as
>  there's where the actual _pipelines_ are. Somewhere in there kill
>  interval as a concept on pipeline side.
>
>  This of course doesn't address the messaging complexity. I admit
>  that I don't understand all the issues there but it often feels
>  like we are doing that aspect of things completely wrong, so I
>  would hope that before we change things there we consider all the
>  options.
>
> What else?
>
> One probably crazy idea: What about figuring out the desired end-meters
> of common transformations and making them into dedicated pollsters?
> Encapsulating that transformation not at the level of the polling
> manager but at the individual pollster.
>
>

-- 
Chris Dent               (╯°□°)╯︵┻━┻            http://anticdent.org/
freenode: cdent                                         tw: @anticdent