Open Stack

On Fri, 2020-11-27 at 18:35 +0100, Arne Wiebalck wrote:
> Hi,
> 
> I did a quick test (one data point):
> 
> - the image build time increased by 10 mins
>    (on a VM, this is more than double compared to gzip)
> - but: the resulting image size is ~30% smaller (421 vs 297 MB)
> - the cleaning time (unpacking on bare metal!) increased by ~30 seconds
> 
that is one of the main trade offs the lzma compression makes
it front loads the computeational load to the comppression step
requireing both more ram and time to do the initall compression while
also achiviving a better compresison ratio while allowing light weight
clients to decompress it without similar increase the time/ram requirement for the client.
you may also want to consider the .xz format

lzma and lzma2 have now been supperceed by .xz 
https://en.wikipedia.org/wiki/XZ_Utils

.xz format is becomming more an d more commen fro things liek compress kernel modules and packages.

> So, lzma looks like a good option to reduce the image size which
> we had to do in our deployment already to address boot issues
> (we removed some packages).
> 
> Keeping gzip as the default and offering lzma as an option to
> start with as suggested by Sergii seems like a good way forward.
> 
> I also think it would be good to have someone else test as well
> to have another data point :-)
> 
> Cheers,
>   Arne
> 
> 
> On 27.11.20 14:00, Sergii Golovatiuk wrote:
> > Hi,
> > 
> > LZMA causes very high CPU and memory usage for creating images, leaving 
> > less resources for other processes. If Ironic is running alongside with 
> > other services that may cause significant impact for them. I would leave 
> > gzip option as default, would introduce --lzma as well as --gzip and use 
> > lzma on 5-10% of our CI resources to test how it goes. Then after a 
> > significant amount of testing we could turn it on as default. Proper 
> > deprecation should be applied here as well IMHO.
> > 
> > чт, 26 нояб. 2020 г. в 17:57, Dmitry Tantsur <dtantsur at redhat.com 
> > <mailto:dtantsur at redhat.com>>:
> > 
> >     Hi folks,
> > 
> >     I've been playing with ways to reduce the size of our IPA images.
> >     While package removals can only save us tens of megabytes, switching
> >     from gzip to lzma reduces the size by around a third (from 373M to
> >     217M in my testing).
> > 
> >     What's the caveat? The unpacking time increases VERY substantially.
> >     On my nested virt lab the 217M image took around 5 minutes to
> >     unpack. I'm not sure how much it will impact real bare metal, please
> >     feel free to test
> >     https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/764371
> >     and tell me.
> > 
> >     So, what do you think? Switching to lzma by default will likely
> >     affect CI run time (assuming we still have DIB jobs somewhere...)
> >     and development environments, but it will also provide a visible
> >     reduction in the image size (which benefit all environments). Large
> >     TripleO images may particularly benefit from this (but also
> >     particularly affected by the unpacking time).
> > 
> >     Feedback is very welcome.
> > 
> >     Dmitry
> > 
> >     -- 
> >     Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
> >     Commercial register: Amtsgericht Muenchen, HRB 153243,
> >     Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs,
> >     Michael O'Neill
> > 
> > 
> > 
> > -- 
> > SergiiGolovatiuk
> > 
> > Senior Software Developer
> > 
> > Red Hat <https://www.redhat.com/>
> > 
> > <https://www.redhat.com/>
> > 
>