Hi,
tl;dr Besides sending CI logs, does someone use Logstash on Opendev to push information to the new Elasticsearch (Opensearch) service?
Long version:
I am currently looking into a new log workflow for OpenDev's OpenSearch that is using the logscraper tool [1], as an improvement over the current workflow [2]. I spotted a few bottlenecks that are complicating pushing logs to the new Elasticsearch service. The most important bottlenecks are:
- - gearman worker - this service needs at least 10 instances to keep up with the current influx of build logs. - As a reminder, this service consumes information prepared by gearman client, downloads the logs, adds information and
- pushes data to the logstash service, - - logstash service - this service is "freezing" from time to time and it requires many resources such as CPU and RAM.
- As a reminder, the service receives data and by using grok filter [3] prepares information that later will be pushed - to Elasticsearch.
-
The new workflow would eliminate these bottlenecks by pushing data directly into Elasticsearch. This would therefore drop the following services: - gearman client - gearman worker - logstash service
Before proceeding further, we need to know if these services, especially logstash, are (apart from sending logs from CI) not used anywhere, so that we don't accidentally pull the plug on any other workflows. If you are aware of these, please let me know.
Dan
[1] https://opendev.org/openstack/ci-log-processing [2] https://docs.opendev.org/opendev/system-config/latest/logstash.html [3] https://opendev.org/openstack/logstash-filters/src/branch/master/filters/ope...