Open Stack

On 01/02/2014 09:44 PM, Clark Boylan wrote:
<snip>
> This is great stuff. Out of curiousity is doing the graphing with
> pandas and ES vs graphite so that we can graph things in a more ad hoc
> fashion?

So, we need to go to ES for the fingerprints anyway (because that's 
where we mine them from), which means we need a way to process ES data 
into TimeSeries. In order to calculate frequencies we need largely 
equivalent TimeSeries that are base lines for # of jobs run of 
particular types. Given that we can get that with an ES query, it 
prevents the need of having to have a different data transformation 
process to get to the same kind of TimeSeries.

It also lets us bulk query. With 1 ~20second ES query we get all states, 
of all jobs, across all queues, over the last 7 days (as well as 
information on review). And the transform to slice is super easy because 
it's 10s of thousands of records that are dictionaries, which makes for 
good input. You'd need to do a bunch of unbinning and transforms to 
massage the graphite data to pair with what we have in the fingerprint data.

Eventually having tools to do the same thing with graphite is probe ably 
a good thing, largely for other analysis people want to do on that (I 
think long term having some data kits for our bulk data to let people 
play with it is goodness). I'd just put it after a 1.0 as I think it's 
not really needed.

> Also, for the dashboard, Kibana3 does a lot more stuff than
> Kibana2 which we currently use. I have been meaning to get Kibana3
> running alongside Kibana2 and I think it may be able to do multi line
> queries (I need to double check that but it has a lot more query and
> graphing capability). I think Kibana3 is worth looking into as well
> before we go too far down the road of custom UI.

Absolutely. There is a reason that's all the way at the bottom of the 
list, and honestly, something I almost didn't put in there. But I 
figured we needed to understand the implications of multi line matches 
with our current UI, and the fact that they will make some things 
better, but discovering those matches will be harder with the existing UI.

If Kibana3 solves it, score. One less thing to do. Because I'd really 
like to not be in the business of maintaining a custom web UI just for 
discovery of fingerprints.

	-Sean

-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net