Open Stack

Mattherw, Dmitry

I would suggest to use:
1) multi-threaded utilities
2) use xz for small snapshots (<1gb) and lrzip for bigger snapshots

On Mon, Nov 24, 2014 at 2:25 PM, Matthew Mosesohn <mmosesohn at mirantis.com>
wrote:

> I did this exercise over many iterations during Docker container
> packing and found that as long as the data is under 1gb, it's going to
> compress really well with xz. Over 1gb and lrzip looks more attractive
> (but only on high memory systems). In reality, we're looking at log
> footprints from OpenStack environments on the order of 500mb to 2gb.
>
> xz is very slow on single-core systems with 1.5gb of memory, but it's
> quite a bit faster if you run it on a more powerful system. I've found
> level 4 compression to be the best compromise that works well enough
> that it's still far better than gzip. If increasing compression time
> by 3-5x is too much for you guys, why not just go to bzip? You'll
> still improve compression but be able to cut back on time.
>
> Best Regards,
> Matthew Mosesohn
>
> On Mon, Nov 24, 2014 at 3:14 PM, Vladimir Kuklin <vkuklin at mirantis.com>
> wrote:
> > IMO, we should get a bunch of snapshots and decide which compression to
> use
> > according to the results of an experiment. XZ compression takes much
> longer,
> > so you will need to use parallel xz compression utility.
> >
> > On Fri, Nov 21, 2014 at 9:09 PM, Tomasz Napierala <
> tnapierala at mirantis.com>
> > wrote:
> >>
> >>
> >> > On 21 Nov 2014, at 16:55, Dmitry Pyzhov <dpyzhov at mirantis.com> wrote:
> >> >
> >> > We have a request for change compression from gz to xz. This simple
> >> > change halfs our snapshots. Does anyone has any objections? Otherwise
> we
> >> > will include this change in 6.0 release.
> >>
> >> I agree with the change, but it shouldn’t be high
> >>
> >> Regards,
> >> --
> >> Tomasz 'Zen' Napierala
> >> Sr. OpenStack Engineer
> >> tnapierala at mirantis.com
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> > --
> > Yours Faithfully,
> > Vladimir Kuklin,
> > Fuel Library Tech Lead,
> > Mirantis, Inc.
> > +7 (495) 640-49-04
> > +7 (926) 702-39-68
> > Skype kuklinvv
> > 45bk3, Vorontsovskaya Str.
> > Moscow, Russia,
> > www.mirantis.com
> > www.mirantis.ru
> > vkuklin at mirantis.com
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
45bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com <http://www.mirantis.ru/>
www.mirantis.ru
vkuklin at mirantis.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141124/0c87900f/attachment.html>