[openstack-dev] os-loganalyze, project log parsing, or ...

Clint Byrum clint at fewbar.com
Wed Sep 28 12:45:05 UTC 2016


Excerpts from Andrew Laski's message of 2016-09-27 11:36:07 -0400:
> Hello all,
> 
> Recently I noticed that people would look at logs from a Zuul née
> Jenkins CI run and comment something like "there seem to be more
> warnings in here than usual." And so I thought it might be nice to
> quantify that sort of thing so we didn't have to rely on gut feelings.
> 
> So I threw together https://review.openstack.org/#/c/376531 which is a
> script that lives in the Nova tree, gets called from a devstack-gate
> post_test_hook, and outputs an n-stats.json file which can be seen at
> http://logs.openstack.org/06/375106/8/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/e103612/logs/n-stats.json.
> This provides just a simple way to compare two runs and spot large
> changes between them. Perhaps later things could get fancy and these
> stats could be tracked over time. I am also interested in adding stats
> for things that are a bit project specific like how long (max, min, med)
> it took to boot an instance, or what's probably better to track is how
> many operations that took for some definition of an operation.
> 
> I received some initial feedback that this might be a better fit in the
> os-loganalyze project so I took a look over there. So I cloned the
> project to take a look and quickly noticed
> http://git.openstack.org/cgit/openstack-infra/os-loganalyze/tree/README.rst#n13.
> That makes me think it would not be a good fit there because what I'm
> looking to do relies on parsing the full file, or potentially multiple
> files, in order to get useful data.
> 
> So my questions: does this seem like a good fit for os-loganalyze? If
> not is there another infra/QA project that this would be a good fit for?
> Or would people be okay with a lone project like Nova implementing this
> in tree for their own use?
> 

I wonder if we could combine forces on this. You want to report log
stats. I want to report query/messaging stats:

http://specs.openstack.org/openstack/qa-specs/specs/devstack/counter-inspection.html

I've never finished this one completely, but basically I wrote a thing
that inspects performance counters before and after the tempest run,
and outputs JSON which gets picked up and inserted into the subunit
stream as a subunit attachment.

>From there, the part that isn't finished, is a job that picks up that
attachment and sprays the numbers into statsd/graphite, allowing trend
analysis. One can also of course just go look at the json file to find
out useful things like how many SQL queries were run, or how many rows
were read out of the database.

Wherever your code lands, it feels like it would be useful to have those
numbers graphed.



More information about the OpenStack-dev mailing list