Open Stack

On Tue, Sep 27, 2016, at 04:43 PM, Matthew Treinish wrote:
<snip>
> > > 
> > > I definitely can see the value in having machine parsable log stats in
> > > our
> > > artifacts, but I'm not sure where project specific pieces would come
> > > from. But,
> > > given that hypothetical I would say as long as you made those pieces
> > > configurable (like a yaml syntax to search for patterns by log file or
> > > something) and kept a generic framework/tooling for parsing the log stats
> > > I
> > > think it's still a good fit for a QA or Infra project. Especially if you
> > > think
> > > whatever pattern you're planning to use is something other projects would
> > > want
> > > to reuse.
> > 
> > My concern here is that I want to go beyond simple pattern matching. I
> > want to be able to maintain state while parsing to associate log lines
> > with events that came before. The project specific bits I envision are
> > the logic to handle that, but I don't think yaml is expressive enough
> > for it. I came up with a quick example at
> > http://paste.openstack.org/show/583160/ . That's Nova specific and
> > beyond my capability to express in yaml or elastic-recheck.
> 
> That's pretty simple to do with yaml too. Especially since it's tied to a
> single
> regex. For example, something roughly like:
> 
> http://paste.openstack.org/show/583165/
> 
> would use yaml to make it a generic framework. 

Okay, I see that it can be done for this example. But I look at that
yaml and all I see is that after a couple of definitions:

count_api_stuffs:
          - name: Instance Boots
            regex = '(req-\S+).*Starting instance'
            log_file = n-cpu.txt
          - name: Volume something
            regex = '(req-\S+).*Random Cinder Event to Count'
            log_file = c-api.txt
time_api_event:
           -name: Foo
           regex = <bleh>
another_thing:
yet_another:

there's no context to what each of these does and the logic is competely
decoupled. I guess I'm just personally biased to wanting to see it done
in code to help my understanding.

If yaml is what makes this acceptable then maybe I'll rework to do it
that way eventually. What I was hoping to do was start small, prove the
concept in a project while maintaining flexibility, and then look at
expanding for general use. Essentially the oslo model. I'm a little
surprised that there's so much resistance to that.


> 
> That's what I was trying to get across before, what might seem project
> specific
> is just a search pattern or combination of them for a log file. These
> patterns
> are generic enough we can extract out the project specific pieces and it
> could
> be made configurable to enable reuse between projects. Especially since
> most of
> the logging format should be standardized. (this would further expose
> where
> things differ too)
> 
> So yeah I still think this makes sense as a QA or Infra project. Although
> I'm
> wondering if there is a better way to harvest this info from a run.
> 
> As an aside it does feel like we should have a better mechanism for most
> of
> this. Like for counting booted instances I normally use the qemu log
> files, for
> example:
> 
> http://logs.openstack.org/49/373249/1/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/b0159bf/logs/libvirt/qemu/
> 
> which shows how many instances we booted during that run. But I guess it
> really depends on what we're looking for. So there probably isn't an
> easier
> answer.
> 
> -Matt Treinish
> 
> > > > > 
> > > > > I would caution against doing it as a one off in a project repo doesn't
> > > > > seem
> > > > > like the best path forward for something like this. We actually tried to
> > > > > do
> > > > > something similar to that in the past inside the tempest repo:
> > > > > 
> > > > > http://git.openstack.org/cgit/openstack/tempest/tree/tools/check_logs.py
> > > > > 
> > > > > and
> > > > > 
> > > > > http://git.openstack.org/cgit/openstack/tempest/tree/tools/find_stack_traces.py
> > > > > 
> > > > > all it did was cause confusion because no one knew where the output was
> > > > > coming
> > > > > from. Although, the output from those tools was also misleading, which
> > > > > was
> > > > > likely a bigger problm. So this probably won't be an issue if you add a
> > > > > json
> > > > > output to the jobs.
> > > > > 
> > > > > I also wonder if the JSONFormatter from oslo.log:
> > > > > 
> > > > > http://docs.openstack.org/developer/oslo.log/api/formatters.html#oslo_log.formatters.JSONFormatter
> > > > > 
> > > > > would be useful here. We can proabbly turn that on if it makes things
> > > > > easier.
> > > > > 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> Email had 1 attachment:
> + signature.asc
>   1k (application/pgp-signature)