Open Stack

On Tue, Apr 6, 2021, at 9:11 AM, Radosław Piliszek wrote:
> On Tue, Apr 6, 2021 at 6:02 PM Jeremy Stanley <fungi at yuggoth.org> wrote:
> > Looking at the error, I strongly suspect memory exhaustion. We could
> > try tuning xz to use less memory when compressing.

Worth noting that we continue to suspect memory pressure, and in particular diving into swap, for random failures that appear timing or performance related. I still think it would be a helpful exercise for OpenStack to look at its memory consumption (remember end users will experience this too) and see if there are any unexpected areas of memory use. I think the last time i skimmed logs the privsep daemon was a large consumer because we separate instance is run for each service and they all add up.

> 
> That was my hunch as well, hence why I test using gzip.
> 
> On Tue, Apr 6, 2021 at 5:51 PM Clark Boylan <cboylan at sapwetik.org> wrote:
> >
> > On Tue, Apr 6, 2021, at 8:14 AM, Radosław Piliszek wrote:
> > > I am testing whether replacing xz with gzip would solve the problem [1] [2].
> >
> > The reason we used xz is that the files are very large and gz compression is very poor compared to xz for these files and these files are not really human readable as is (you need to load them into journald first). Let's test it and see what the gz file sizes look like but if they are still quite large then this is unlikely to be an appropriate fix.
> 
> Let's see how bad the file sizes are.
> If they are acceptable, we can keep gzip and be happy.
> Otherwise we try to tune the params to make xz a better citizen as
> fungi suggested.
> 
> -yoctozepto
> 
>