[openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

Mike Scherbakov mscherbakov at mirantis.com
Wed Nov 26 10:31:50 UTC 2014


Can we put it as a work item for diagnostic snapshot improvements, so we
won't forget about this in 6.1?

On Tuesday, November 25, 2014, Dmitry Pyzhov <dpyzhov at mirantis.com> wrote:

> Thank you all for your feedback. Request postponed to the next release. We
> will compare available solutions.
>
> On Mon, Nov 24, 2014 at 2:36 PM, Vladimir Kuklin <vkuklin at mirantis.com
> <javascript:_e(%7B%7D,'cvml','vkuklin at mirantis.com');>> wrote:
>
>> guys, there is already pxz utility in ubuntu repos. let's test it
>>
>> On Mon, Nov 24, 2014 at 2:32 PM, Bartłomiej Piotrowski <
>> bpiotrowski at mirantis.com
>> <javascript:_e(%7B%7D,'cvml','bpiotrowski at mirantis.com');>> wrote:
>>
>>> On 24 Nov 2014, at 12:25, Matthew Mosesohn <mmosesohn at mirantis.com
>>> <javascript:_e(%7B%7D,'cvml','mmosesohn at mirantis.com');>> wrote:
>>> > I did this exercise over many iterations during Docker container
>>> > packing and found that as long as the data is under 1gb, it's going to
>>> > compress really well with xz. Over 1gb and lrzip looks more attractive
>>> > (but only on high memory systems). In reality, we're looking at log
>>> > footprints from OpenStack environments on the order of 500mb to 2gb.
>>> >
>>> > xz is very slow on single-core systems with 1.5gb of memory, but it's
>>> > quite a bit faster if you run it on a more powerful system. I've found
>>> > level 4 compression to be the best compromise that works well enough
>>> > that it's still far better than gzip. If increasing compression time
>>> > by 3-5x is too much for you guys, why not just go to bzip? You'll
>>> > still improve compression but be able to cut back on time.
>>> >
>>> > Best Regards,
>>> > Matthew Mosesohn
>>>
>>> Alpha release of xz supports multithreading via -T (or —threads)
>>> parameter.
>>> We could also use pbzip2 instead of regular bzip to cut some time on
>>> multi-core
>>> systems.
>>>
>>> Regards,
>>> Bartłomiej Piotrowski
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> <javascript:_e(%7B%7D,'cvml','OpenStack-dev at lists.openstack.org');>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Yours Faithfully,
>> Vladimir Kuklin,
>> Fuel Library Tech Lead,
>> Mirantis, Inc.
>> +7 (495) 640-49-04
>> +7 (926) 702-39-68
>> Skype kuklinvv
>> 45bk3, Vorontsovskaya Str.
>> Moscow, Russia,
>> www.mirantis.com <http://www.mirantis.ru/>
>> www.mirantis.ru
>> vkuklin at mirantis.com
>> <javascript:_e(%7B%7D,'cvml','vkuklin at mirantis.com');>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> <javascript:_e(%7B%7D,'cvml','OpenStack-dev at lists.openstack.org');>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>

-- 
Mike Scherbakov
#mihgen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141126/62477ad4/attachment.html>


More information about the OpenStack-dev mailing list