<br><br><div class="gmail_quote">On Fri, Nov 23, 2012 at 11:26 AM, Eoghan Glynn <span dir="ltr"><<a href="mailto:eglynn@redhat.com" target="_blank">eglynn@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
<br>
> > The pollster or notification handler would basically just<br>
> > stuff all the available raw data into the sample kwargs. The<br>
> > transformers would then have to know which named args to<br>
> > expect and how to interpret the resource_obj.<br>
><br>
> I don't like the idea of having the transformer to guess which args<br>
> is going to get.<br>
<br>
</div>Well I was thinking that the transformer would be written<br>
with the inbuilt knowledge of what the corresponding pollster<br>
is going to emit (as opposed to guessing).<br>
<div class="im"><br>
> To me, the pollster or notifications handler should be<br>
> responsible to emit the maximum amount of "counters" it can<br>
> from what it gets. And for each counter, you would pass it<br>
> through a transformer, mangling the value to some over thing.<br>
<br>
</div>Isn't the Counter just a tuple of name-value pairs?<br>
<br>
I'm not sure how it would help to have the raw data passed via<br>
a tuple as compared to kwargs?<br></blockquote><div><br></div><div>The Counter is implemented as a namedtuple because that was enough for what we were doing before but gave us the ability to switch to a more flexible class later when our requirements changed (like now, where I envision more complex documentation requirements).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> The solution I'd imagine for that is to have the all pollster to emit<br>
> something equivalent (but probably simpler) to Counter like for<br>
> example<br>
> in this case:<br>
><br>
> { resource_name = '<a href="http://disk.io" target="_blank">disk.io</a>',<br>
> resource_id = 'instanceid.vda',<br>
> user_id = 'qwerty789',<br>
> tenant_id = 'abcdef123',<br>
> value = 123456 }<br>
><br>
> So for CW you wouldn't transform the value, but the resource_name to<br>
> some other thing. This could be achieved via a transformer named<br>
> "RenameResourceName" which could be "configured" with a map:<br>
><br>
> { "<a href="http://disk.io" target="_blank">disk.io</a>": "DiskWriteBytes",<br>
> … }<br>
><br>
> So it's kind of generic and you can even use it to do some other<br>
> stuff.<br>
><br>
> Does that make sense?<br>
<br>
</div>Interesting thought. I think the generic renaming approach could work<br>
for the simplest cases, but say the CW transformer also had to construct<br>
a few different dimensions for the metric?<br>
<br>
For example for an instance metric, it would set the ImageId dimension<br>
to the image UUID, the InstanceId dimension to the instance UUID, the<br>
AutoScalingGroupName dimension to some other value taken from the instance<br>
metadata.<br>
<br>
In that case, the logic in the CW transformer would probably have to<br>
be hand-crafted instead of being generic.<br></blockquote><div><br></div><div>We can provide a library of generic functions for building transformers so everyone doesn't have to do it from scratch. I don't think we want deployers to have to understand or deal with all of that, though.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> > Yeah, that's an idea. Wouldn't transformers have to be chained in<br>
> > that<br>
> > case? So for example the relevant CW transformer chain would be:<br>
> ><br>
> > generic-transformer-calculating-cpu-util-from-cumulative-time --><br>
> > CW-specific-transformer-outputting-as-CPUUtilization-datapoint<br>
> ><br>
> > That would be fine, just wanted to call out a potential extra<br>
> > bit of complexity to capture in the pipeline config.<br>
><br>
> Yes, I think we'll need to chain them. Don't know how we can do that<br>
> easily in our configuration file.<br>
<br>
</div>Yeah, the common cfg format has only limited support for collection<br>
types.<br>
<div class="im"><br>
> OTOH, I imagine that most of the transformers configuration is<br>
> going to be pretty generic, like for CW. So we could define and<br>
> provide a default JSON file with all the default pipelining<br>
> set-up correctly for most counters and publishers.<br>
<br>
</div>Yep, a JSON file might be better option here than the common cfg<br>
format.<br>
<br>
Cheers,<br>
Eoghan<br>
<div class="HOEnZb"><div class="h5"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br>