Open Stack

On 24 Nov 2014, at 12:25, Matthew Mosesohn <mmosesohn at mirantis.com> wrote:
> I did this exercise over many iterations during Docker container
> packing and found that as long as the data is under 1gb, it's going to
> compress really well with xz. Over 1gb and lrzip looks more attractive
> (but only on high memory systems). In reality, we're looking at log
> footprints from OpenStack environments on the order of 500mb to 2gb.
> 
> xz is very slow on single-core systems with 1.5gb of memory, but it's
> quite a bit faster if you run it on a more powerful system. I've found
> level 4 compression to be the best compromise that works well enough
> that it's still far better than gzip. If increasing compression time
> by 3-5x is too much for you guys, why not just go to bzip? You'll
> still improve compression but be able to cut back on time.
> 
> Best Regards,
> Matthew Mosesohn

Alpha release of xz supports multithreading via -T (or —threads) parameter.
We could also use pbzip2 instead of regular bzip to cut some time on multi-core
systems.

Regards,
Bartłomiej Piotrowski