Hi,
thank you for your advice.
Created a PS for logsender [1]. Hope it helps.

Dan

[1] https://review.opendev.org/c/openstack/ci-log-processing/+/843929 

On Mon, May 30, 2022 at 2:39 PM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2022-05-30 14:02:15 +0200 (+0200), Daniel Pawlik wrote:
> In the last few days there is increase amount of CI jobs that are
> running on Zuul CI gates, where later logs are pushed into the
> Opensearch cluster [1]. The current cluster storage is almost
> full, which causes one of the actions to be performed: temporarily
> changing log retention (currently it is set to 14 days) or
> removing temporary non-essential logs [2]. I recommend removing
> not important logs, so if projects leaders could check whether
> entries related to the project are needed, it would be really
> helpful.
[...]

For the old system, we filtered out all debug level log lines, only
importing lines at info level or greater. If you're not doing that
yet on the new system, it would probably help free up a lot of
space.

We tried to maintain around 7-10 days of retention, so lowering the
retention from 14 days isn't really a regression over what we used
to provide.
--
Jeremy Stanley


--
Regards,
Daniel Pawlik