Open Stack

[...] 
> > The pollster or notification handler would basically just stuff all
> > the available raw data into the sample kwargs. The transformers would
> > then have to know which named args to expect and how to interpret the
> > resource_obj.
> 
> 
> 
> In previous API designs I have always found it easier to have
> well-defined classes passing data between the layers, rather than
> accepting variable, undefined, arguments like this. The new
> developer coming along to add a feature has far less work to do when
> figuring out what data to emit as output or take as input to a new
> plugin, and it almost always turns out to be easier to create
> components that can be recombined in unexpected ways because of the
> standard data structures. Using a class also means fewer changes
> when new fields are added (because you only have to find where they
> are constructed, not every call to a plugin) or removed (because you
> can provide a backwards-compatibility @property method).

That's fair enough, hopefully we can achieve the strongly-typed
interaction between pollster and transformer without an undue
proliferation of new types. 


> All of the notification and pollsters should continue to emit a
> common object (either Counter instances or some updated thing that
> meets our needs better). All of the transformers should accept
> instances of those objects as input. Each publisher should define a
> class representing what it wants as inputs -- no dictionaries,
> tuples, etc. Use classes that can be documented clearly (even if
> just as namedtuples). Since each transformer will be bound to a
> publisher, it will know which type of object(s) to emit (I assume a
> transformation may cause one Counter instance to become several
> inputs to a publisher).

Yes, agreed, the publisher and transformer relationship is an
intimate one, and the transformer will know what type the publisher
expects (which may be a type defined by the publisher itself, or
by some third party library such as boto if that's more convenient).


[...]
> > Yeah, that's an idea. Wouldn't transformers have to be chained in
> > that case? So for example the relevant CW transformer chain would be:
> >
> > generic-transformer-calculating-cpu-util-from-cumulative-time -->
> > CW-specific-transformer-outputting-as-CPUUtilization-datapoint
> >
> > That would be fine, just wanted to call out a potential extra
> > bit of complexity to capture in the pipeline config.
> 
> Would we expect users to configure that pipeline?

Rarely, it should be more of a side-effect of deployment (e.g.
rolling out a CW service would require corresponding updates to
the ceilo compute agent config).

Cheers,
Eoghan