Open Stack

I did this exercise over many iterations during Docker container
packing and found that as long as the data is under 1gb, it's going to
compress really well with xz. Over 1gb and lrzip looks more attractive
(but only on high memory systems). In reality, we're looking at log
footprints from OpenStack environments on the order of 500mb to 2gb.

xz is very slow on single-core systems with 1.5gb of memory, but it's
quite a bit faster if you run it on a more powerful system. I've found
level 4 compression to be the best compromise that works well enough
that it's still far better than gzip. If increasing compression time
by 3-5x is too much for you guys, why not just go to bzip? You'll
still improve compression but be able to cut back on time.

Best Regards,
Matthew Mosesohn

On Mon, Nov 24, 2014 at 3:14 PM, Vladimir Kuklin <vkuklin at mirantis.com> wrote:
> IMO, we should get a bunch of snapshots and decide which compression to use
> according to the results of an experiment. XZ compression takes much longer,
> so you will need to use parallel xz compression utility.
>
> On Fri, Nov 21, 2014 at 9:09 PM, Tomasz Napierala <tnapierala at mirantis.com>
> wrote:
>>
>>
>> > On 21 Nov 2014, at 16:55, Dmitry Pyzhov <dpyzhov at mirantis.com> wrote:
>> >
>> > We have a request for change compression from gz to xz. This simple
>> > change halfs our snapshots. Does anyone has any objections? Otherwise we
>> > will include this change in 6.0 release.
>>
>> I agree with the change, but it shouldn’t be high
>>
>> Regards,
>> --
>> Tomasz 'Zen' Napierala
>> Sr. OpenStack Engineer
>> tnapierala at mirantis.com
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 45bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com
> www.mirantis.ru
> vkuklin at mirantis.com
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>