On 2022-05-30 14:02:15 +0200 (+0200), Daniel Pawlik wrote:
In the last few days there is increase amount of CI jobs that are running on Zuul CI gates, where later logs are pushed into the Opensearch cluster [1]. The current cluster storage is almost full, which causes one of the actions to be performed: temporarily changing log retention (currently it is set to 14 days) or removing temporary non-essential logs [2]. I recommend removing not important logs, so if projects leaders could check whether entries related to the project are needed, it would be really helpful.
[...]
For the old system, we filtered out all debug level log lines, only importing lines at info level or greater. If you're not doing that yet on the new system, it would probably help free up a lot of space.
We tried to maintain around 7-10 days of retention, so lowering the retention from 14 days isn't really a regression over what we used to provide.