OpenStack logs on Opensearch - storage problem

Daniel Pawlik dpawlik at redhat.com
Tue May 31 06:24:34 UTC 2022


Hi,
thank you for your advice.
Created a PS for logsender [1]. Hope it helps.

Dan

[1] https://review.opendev.org/c/openstack/ci-log-processing/+/843929
<https://review.opendev.org/c/openstack/ci-log-processing/+/843929>

On Mon, May 30, 2022 at 2:39 PM Jeremy Stanley <fungi at yuggoth.org> wrote:

> On 2022-05-30 14:02:15 +0200 (+0200), Daniel Pawlik wrote:
> > In the last few days there is increase amount of CI jobs that are
> > running on Zuul CI gates, where later logs are pushed into the
> > Opensearch cluster [1]. The current cluster storage is almost
> > full, which causes one of the actions to be performed: temporarily
> > changing log retention (currently it is set to 14 days) or
> > removing temporary non-essential logs [2]. I recommend removing
> > not important logs, so if projects leaders could check whether
> > entries related to the project are needed, it would be really
> > helpful.
> [...]
>
> For the old system, we filtered out all debug level log lines, only
> importing lines at info level or greater. If you're not doing that
> yet on the new system, it would probably help free up a lot of
> space.
>
> We tried to maintain around 7-10 days of retention, so lowering the
> retention from 14 days isn't really a regression over what we used
> to provide.
> --
> Jeremy Stanley
>


-- 
Regards,
Daniel Pawlik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20220531/fa51e4c0/attachment.htm>


More information about the openstack-discuss mailing list