[all][infra][qa] Retiring Logstash, Elasticsearch, subunit2sql, and Health

Jeremy Stanley fungi at yuggoth.org
Wed May 12 13:17:13 UTC 2021


On 2021-05-12 02:05:57 -0700 (-0700), Sorin Sbarnea wrote:
[...]
> TripleO health check project relies on being able to query ER from
> both opendev and rdo in order to easy identification of problems.

Since you say RDO has a similar setup, could they just expand to
start indexing our logs? As previously stated, doing that doesn't
require any special access to our infrastructure.

> Maybe instead of dropping we should rethink what it is supposed to
> index and not, set some hard limits per job and scale down the
> deployment. IMHO, one of the major issues with it is that it does
> try to index maybe too much w/o filtering noisy output before
> indexing.

Reducing how much we index doesn't solve the most pressing problem,
which is that we need to upgrade the underlying operating system,
therefore replace the current current configuration management which
won't work on newer platforms, and also almost certainly upgrade
versions of the major components in use for it. Nobody has time to
do that, at least nobody who has heard our previous cries for help.

> If we can delay making a decision a little bit so we can
> investigate all available options it would really be great.

This thread hasn't set any timeline for stopping the service, not
yet anyway.

> I worth noting that I personally do not have a special love for ES
> but I do value a lot what it does. I am also pragmatic and I would
> not be very upset to make use of a SaaS service as an alternative,
> especially as I recognize how costly is to run and maintain an
> instance.
[...]

It's been pointed out that OVH has a similar-sounding service, if
someone is interested in experimenting with it:

https://www.ovhcloud.com/en-ca/data-platforms/logs/

The case with this, and I think with any SaaS solution, is that
there would still need to be a separate ingestion mechanism to
identify when new logs are available, postprocess them to remove
debug lines, and then feed them to the indexing service at the
provider... something our current team doesn't have time to design
and manage.
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210512/32582c75/attachment.sig>


More information about the openstack-discuss mailing list