[OpenStack-Infra] ask.openstack.org full disk

Ian Wienand iwienand at redhat.com
Mon Dec 5 05:08:37 UTC 2016


On 11/23/2016 01:51 AM, Jeremy Stanley wrote:
> Thanks! I removed a few old manual backups from some of our homedirs
> (mostly mine!) freeing up a few more GB on the rootfs. The biggest
> offender though seems to be /var/log/jetty which has about a week of
> retention. Whatever's rotating these daily at midnight UTC (doesn't
> seem to be logrotate doing it) isn't compressing them, so they're up
> to nearly 13GB now (which is a lot on a 40GB rootfs).

This happened again today.

I removed all the 2016_11_*.log files from there, which freed up about
16GiB.  Upon more investigation, it seems jetty is making these log
files directly, and "rotating" by just logging to a file with the
current day and removing files after a few days.

To change this to something smarter than can compress too, I think you
need to put log4j classes into the classpath and configure jetty to
use the slf4j facade to redirect logs and ... I dunno, it got too
hard.  fungi also went in a live-configured it to be less verbose.  I
think this also requires more configuration to be made persistent.

I've proposed [1] to just compress the logs and cleanup in cron which
seems like the KISS approach, at least for now.

-i

[1] https://review.openstack.org/#/c/406670/



More information about the OpenStack-Infra mailing list