Open Stack

On Fri, Nov 23, 2012 at 11:26 AM, Eoghan Glynn <eglynn at redhat.com> wrote:

>
>
> > > The pollster or notification handler would basically just
> > > stuff all the available raw data into the sample kwargs. The
> > > transformers would then have to know which named args to
> > > expect and how to interpret the resource_obj.
> >
> > I don't like the idea of having the transformer to guess which args
> > is going to get.
>
> Well I was thinking that the transformer would be written
> with the inbuilt knowledge of what the corresponding pollster
> is going to emit (as opposed to guessing).
>
> > To me, the pollster or notifications handler should be
> > responsible to emit the maximum amount of "counters" it can
> > from what it gets. And for each counter, you would pass it
> > through a transformer, mangling the value to some over thing.
>
> Isn't the Counter just a tuple of name-value pairs?
>
> I'm not sure how it would help to have the raw data passed via
> a tuple as compared to kwargs?
>

The Counter is implemented as a namedtuple because that was enough for what
we were doing before but gave us the ability to switch to a more flexible
class later when our requirements changed (like now, where I envision more
complex documentation requirements).


>
> > The solution I'd imagine for that is to have the all pollster to emit
> > something equivalent (but probably simpler) to Counter like for
> > example
> > in this case:
> >
> >   { resource_name = 'disk.io',
> >     resource_id = 'instanceid.vda',
> >     user_id = 'qwerty789',
> >     tenant_id = 'abcdef123',
> >     value = 123456 }
> >
> > So for CW you wouldn't transform the value, but the resource_name to
> > some other thing. This could be achieved via a transformer named
> > "RenameResourceName" which could be "configured" with a map:
> >
> >   { "disk.io": "DiskWriteBytes",
> >      … }
> >
> > So it's kind of generic and you can even use it to do some other
> > stuff.
> >
> > Does that make sense?
>
> Interesting thought. I think the generic renaming approach could work
> for the simplest cases, but say the CW transformer also had to construct
> a few different dimensions for the metric?
>
> For example for an instance metric, it would set the ImageId dimension
> to the image UUID, the InstanceId dimension to the instance UUID, the
> AutoScalingGroupName dimension to some other value taken from the instance
> metadata.
>
> In that case, the logic in the CW transformer would probably have to
> be hand-crafted instead of being generic.
>

We can provide a library of generic functions for building transformers so
everyone doesn't have to do it from scratch. I don't think we want
deployers to have to understand or deal with all of that, though.


>
> > > Yeah, that's an idea. Wouldn't transformers have to be chained in
> > > that
> > > case? So for example the relevant CW transformer chain would be:
> > >
> > >   generic-transformer-calculating-cpu-util-from-cumulative-time -->
> > >     CW-specific-transformer-outputting-as-CPUUtilization-datapoint
> > >
> > > That would be fine, just wanted to call out a potential extra
> > > bit of complexity to capture in the pipeline config.
> >
> > Yes, I think we'll need to chain them. Don't know how we can do that
> > easily in our configuration file.
>
> Yeah, the common cfg format has only limited support for collection
> types.
>
> > OTOH, I imagine that most of the transformers configuration is
> > going to be pretty generic, like for CW. So we could define and
> > provide a default JSON file with all the default pipelining
> > set-up correctly for most counters and publishers.
>
> Yep, a JSON file might be better option here than the common cfg
> format.
>
> Cheers,
> Eoghan
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121126/dd2975db/attachment.html>