Open Stack

Hi;

I did update another update of my platforme, and still having problems with
fluentd container (it is still restarting) but with different error :


> *Running command: '/usr/sbin/td-agent -o
> /var/log/kolla/fluentd/fluentd.log'/opt/td-agent/lib/ruby/2.7.0/rubygems/specification.rb:2247:in
> `raise_if_conflicts': Unable to activate fluent-plugin-elasticsearch-5.3.0,
> because faraday-1.10.3 conflicts with faraday (>= 2.0.0),
> faraday-excon-1.1.0 conflicts with faraday-excon (>= 2.0.0)
> (Gem::ConflictError)*
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/specification.rb:1369:in `activate'
>         from /opt/td-agent/lib/ruby/2.7.0/rubygems.rb:217:in `rescue in
> try_activate'
>         from /opt/td-agent/lib/ruby/2.7.0/rubygems.rb:210:in `try_activate'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:151:in
> `rescue in require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:147:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluent-plugin-elasticsearch-5.3.0/lib/fluent/plugin/out_elasticsearch.rb:20:in
> `<top (required)>'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:103:in
> `block in search'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:100:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:100:in
> `search'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:44:in
> `lookup'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:169:in
> `new_impl'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:114:in
> `new_output'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:108:in
> `block in configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:99:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:99:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/out_copy.rb:39:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:187:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:132:in
> `add_match'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:74:in
> `block in configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:64:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:64:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/root_agent.rb:149:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/engine.rb:105:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/engine.rb:80:in
> `run_configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/supervisor.rb:571:in
> `run_supervisor'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/command/fluentd.rb:352:in
> `<top (required)>'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/bin/fluentd:15:in
> `<top (required)>'
>         from /opt/td-agent/bin/fluentd:23:in `load'
>         from /opt/td-agent/bin/fluentd:23:in `<top (required)>'
>         from /usr/sbin/td-agent:15:in `load'
>         from /usr/sbin/td-agent:15:in `<main>'
> */opt/td-agent/lib/ruby/2.7.0/rubygems/specification.rb:2247:in
> `raise_if_conflicts': Unable to activate fluent-plugin-elasticsearch-5.3.0,
> because faraday-1.10.3 conflicts with faraday (>= 2.0.0),
> faraday-excon-1.1.0 conflicts with faraday-excon (>= 2.0.0)
> (Gem::ConflictError)*
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/specification.rb:1369:in `activate'
>         from /opt/td-agent/lib/ruby/2.7.0/rubygems.rb:211:in `try_activate'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:151:in
> `rescue in require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:147:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluent-plugin-elasticsearch-5.3.0/lib/fluent/plugin/out_elasticsearch.rb:20:in
> `<top (required)>'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:103:in
> `block in search'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:100:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:100:in
> `search'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:44:in
> `lookup'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:169:in
> `new_impl'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:114:in
> `new_output'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:108:in
> `block in configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:99:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:99:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/out_copy.rb:39:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:187:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:132:in
> `add_match'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:74:in
> `block in configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:64:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:64:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/root_agent.rb:149:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/engine.rb:105:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/engine.rb:80:in
> `run_configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/supervisor.rb:571:in
> `run_supervisor'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/command/fluentd.rb:352:in
> `<top (required)>'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/bin/fluentd:15:in
> `<top (required)>'
>         from /opt/td-agent/bin/fluentd:23:in `load'
>         from /opt/td-agent/bin/fluentd:23:in `<top (required)>'
>         from /usr/sbin/td-agent:15:in `load'
>         from /usr/sbin/td-agent:15:in `<main>'
>
>
> */opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require': cannot load such file -- fluent/log-ext (LoadError)        from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'        from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluent-plugin-elasticsearch-5.3.0/lib/fluent/plugin/out_elasticsearch.rb:20:in
> `<top (required)>'*
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:103:in
> `block in search'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:100:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:100:in
> `search'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/registry.rb:44:in
> `lookup'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:169:in
> `new_impl'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:114:in
> `new_output'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:108:in
> `block in configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:99:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/multi_output.rb:99:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin/out_copy.rb:39:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/plugin.rb:187:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:132:in
> `add_match'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:74:in
> `block in configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:64:in
> `each'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/agent.rb:64:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/root_agent.rb:149:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/engine.rb:105:in
> `configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/engine.rb:80:in
> `run_configure'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/supervisor.rb:571:in
> `run_supervisor'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/lib/fluent/command/fluentd.rb:352:in
> `<top (required)>'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in
> `require'
>         from
> /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.16.1/bin/fluentd:15:in
> `<top (required)>'
>         from /opt/td-agent/bin/fluentd:23:in `load'
>         from /opt/td-agent/bin/fluentd:23:in `<top (required)>'
>         from /usr/sbin/td-agent:15:in `load'
>         from /usr/sbin/td-agent:15:in `<main>'
>
Regards.

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Le lun. 8 mai 2023 à 14:59, Michał Nasiadka <mnasiadka at gmail.com> a écrit :

> Hello,
>
> Then probably the only solution is to rebuild the container image with
> this patch: https://review.opendev.org/c/openstack/kolla/+/882289
> It will take a couple days before it’s merged upstream.
>
> Best regards,
> Michal
>
> On 8 May 2023, at 14:06, wodel youchi <wodel.youchi at gmail.com> wrote:
>
> Hi,
>
> I cannot access the fluentd container, it crashes almost instantly after
> each restart.
> Is there a workaround to make the container start and wait until the
> commands are executed?
>
> Regards.
>
> Le jeu. 4 mai 2023 à 14:12, Michał Nasiadka <mnasiadka at gmail.com> a
> écrit :
>
>> Hello,
>>
>> Kolla-Ansible is not supporting both opensearch and elasticsearch running
>> at the same time - so if you’re using cloudkitty - it’s better to stick for
>> Elasticsearch for now (CK does not support OpenSearch yet).
>>
>> I started working on the bug - will let you know in the bug report when a
>> fix will be merged and images published.
>> In the meantime you can try to uninstall the too-new elasticsearch gems
>> using td-agent-gem uninstall in your running container image.
>>
>> Best regards,
>> Michal
>>
>> On 4 May 2023, at 14:33, wodel youchi <wodel.youchi at gmail.com> wrote:
>>
>> Hi,
>>
>> I'll try to open a bug for this.
>>
>> I am using elasticsearch also with Cloudkitty :
>> cloudkitty_storage_backend: "elasticsearch" instead of influxdb to get some
>> HA.
>> Will I still get the fluentd problem even if I migrate to Opensearch
>> leaving  Cloudkitty with  elasticsearch???
>>
>> Regards.
>>
>>
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>> Virus-free.www.avast.com
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>>
>> Le jeu. 4 mai 2023 à 07:42, Michał Nasiadka <mnasiadka at gmail.com> a
>> écrit :
>>
>>> Hello,
>>>
>>> That probably is a Kolla bug - can you please raise a bug in
>>> launchpad.net?
>>> The other alternative is to migrate to OpenSearch (we’ve back ported
>>> this functionality recently) -
>>> https://docs.openstack.org/kolla-ansible/yoga/reference/logging-and-monitoring/central-logging-guide-opensearch.html#migration
>>>
>>> Best regards,
>>> Michal
>>>
>>> On 3 May 2023, at 17:13, wodel youchi <wodel.youchi at gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I have finished the update of my openstack platform with newer
>>> containers.
>>>
>>> While verifying I noticed that fluentd container keeps restarting.
>>>
>>> In the log file I am having this :
>>>
>>>> 2023-05-03 16:07:59 +0100 [error]: #0 config error
>>>> file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError
>>>> error="Using Elasticsearch client 8.7.1 is not compatible for your
>>>> Elasticsearch server. Please check your using elasticsearch gem version and
>>>> Elasticsearch server."
>>>> 2023-05-03 16:07:59 +0100 [error]: Worker 0 finished unexpectedly with
>>>> status 2
>>>> 2023-05-03 16:07:59 +0100 [info]: Received graceful stop
>>>>
>>>
>>> Those are the images I am using :
>>> (yogavenv) [deployer at rscdeployer ~]$ sudo docker images | grep -i elas
>>>
>>> 192.168.1.16:4000/openstack.kolla/centos-source-prometheus-elasticsearch-exporter
>>>   yoga030523       b48f63ed0072   12 hours ago   539MB
>>> 192.168.1.16:4000/openstack.kolla/centos-source-elasticsearch
>>>             yoga030523       3558611b0cf4   12 hours ago   1.2GB
>>> 192.168.1.16:4000/openstack.kolla/centos-source-elasticsearch-curator
>>>             yoga030523       83a6b48339ea   12 hours ago   637MB
>>>
>>> (yogavenv) [deployer at rscdeployer ~]$ sudo docker images | grep -i fluen
>>> 192.168.1.16:4000/openstack.kolla/centos-source-fluentd
>>>             yoga030523       bf6596e139e2   12 hours ago   847MB
>>>
>>> Any ideas?
>>>
>>> Regards.
>>>
>>>
>>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>>> Virus-free.www.avast.com
>>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230514/3379b479/attachment-0001.htm>