[kolla] [kolla-ansible] fluentd doesn't forward OpenStack logs to Elasticsearch
Mark Goddard
mark at stackhpc.com
Thu Jun 3 08:53:28 UTC 2021
On Thu, 3 Jun 2021 at 09:24, Mark Goddard <mark at stackhpc.com> wrote:
>
> On Sat, 29 May 2021 at 11:24, Bernd Bausch <berndbausch at gmail.com> wrote:
> >
> > I might have found a bug in Kolla-Ansible (Victoria version) but don't know where to file it.
>
> Hi Bernd, you can file kolla-ansible bugs on Launchpad [1].
>
> [1] https://bugs.launchpad.net/kolla-ansible/+filebug
>
> >
> > This is about central logging. In my installation, none of the interesting logs (Nova, Cinder, Neutron...) are sent to Elasticsearch. I confirmed that using tcpdump.
> >
> > I found that fluentd's config file /etc/kolla/fluentd/td-agent.conf tags these logs with "kolla.*". But later in the file, one finds filters like this:
> >
> > # Included from conf/filter/01-rewrite-0.14.conf.j2:
> > <match kolla.var.log.kolla.*.*.log>
> > @type rewrite_tag_filter
> > capitalize_regex_backreference yes
> > ...
> > <rule>
> > key programname
> > pattern ^(nova-api|nova-compute|nova-compute-ironic|nova-conductor|nova-manage|nova-novncproxy|nova-scheduler|nova-placement-api|placement-api|privsep-helper)$
> > tag openstack_python
> > </rule>
> >
> > If I understand this right, this basically re-tags all nova logs with "openstack_python".
> >
> > The same config file has an output rule at the very end. I think the intention is to make this a catch-all rule (or "match anything else"):
> >
> > # Included from conf/output/01-es.conf.j2:
> > <match *.**>
> > @type copy
> > <store>
> > @type elasticsearch
> > host 192.168.122.209
> > port 9200
> > scheme http
> >
> > etc.
> >
> > Unfortunately, the openstack_python tag doesn't match *.**, since it contains no dot. I fixed this with <match **>. Now I receive all logs, but I am not sure if this is the right way to fix it.
>
> I have seen log aggregation working, although possibly haven't tried
> it with Victoria. I can't see any obviously relevant changes, so
> please file a bug.
I tried this out on a recent (CentOS) Victoria deployment. I couldn't
reproduce the issue. My test case was nova-scheduler. I restarted it
and verified that shutdown/startup logs appear in Elastic. Could you
verify whether that case also works for you, and if so, provide a
broken case.
>
> >
> > The error, if it is one, is in https://opendev.org/openstack/kolla-ansible/src/branch/master/ansible/roles/common/templates/conf/output/01-es.conf.j2.
> >
> > If you want me to file a bug, please let me know how.
> >
> > Bernd.
> >
> >
More information about the openstack-discuss
mailing list