Open Stack

Kevin Carter
IRC: kecarter


On Fri, May 1, 2020 at 5:57 PM Alex Schultz <aschultz at redhat.com> wrote:

> On Fri, May 1, 2020 at 4:12 PM Sean Mooney <smooney at redhat.com> wrote:
> >
> > On Fri, 2020-05-01 at 15:18 -0500, Kevin Carter wrote:
> > > Hello Stackers,
> > >
> > > As you may have seen, the TripleO project has been testing the idea of
> > > building container images using a simplified toolchain [0]. The idea
> is to
> > > build smaller, more easily maintained images to simplify the lives of
> > > TripleO consumers. Since TripleO's move to containers, the project has
> been
> > > leveraging Kolla to provide Dockerfiles, and while this has worked,
> TripleO
> > > has created considerable tooling to bend Kolla images to its needs.
> Sadly
> > > this has resulted in an image size explosion and the proliferation of
> > > difficult to maintain tools, with low bus factors, which are essential
> to
> > > the success of the project. To address the risk centered around the
> TripleO
> > > containerization story, we've drafted a spec [0], which we believe
> outlines
> > > a more sustainable future. In this specification, we've designed a new,
> > > much more straightforward, approach to building container images for
> the
> > > TripleO project. The "simple container generation," specification does
> not
> > > intend to be a general-purpose tool used to create images for the
> greater
> > > OpenStack community, both Loci and Kolla do that already. This effort
> is to
> > > build containers only for TripleO using distro provided repositories,
> with
> > > distro maintained tooling. By focusing only on what we need, we're
> able to
> > > remove all general-purpose assumptions and create a vertically
> integrated
> > > stack resulting in a much smaller surface area.
> > >
> > > To highlight how all this works, we've put together several POC
> changes:
> > > * Role and playbook to implement the Containerfile specification [1].
> > > * Tripleoclient review to interface with the new role and playbook [2].
> > > * Directory structure for variable file layout [3].
> > > * To see how this works using the POC code, building images we've
> tested in
> > > real deployments, please watch the ASCII-cast [4].
> > > * Example configuration file examples are here [5][6][7].
> > >
> > > A few examples of size comparisons between our proposed tooling versus
> > > current Kolla based images [8]:
> > >   - base:
> > >       + Kolla: 588 MB
> > >       - new: 211 MB  # based on ubi8, smaller than centos8
> > kolla could also use the ubi8 image as a base you could select centos as
> the base image type and then pass the url to
> > the ubi image and that should would work.
>
>
> ubi8 is smaller, but it doesn't account for it all. We're likely not
> importing some other deps that make it into the normal base that
> perhaps we don't need. I'd still like to see rpm diffs to better
> understand if this savings is real.
>

+1 I think it would be a good exercise to produce an RPM diff for a set of
images. Maybe just the ones we've already ported?


>
> > >   - nova-base:
> > >       + Kolla: 1.09 GB
> > >       - new: 720 MB
> > unless you are using layers fo rthe new iamge keep in mind you have to
> subtract the size of the base imag form the nova
> > base image to caluate how big it actully is so its acully only usein
> about 500MB
> >
> > if you are using layser for the ubi nova-base then these iamge are
> actully the same size the diffenrec is entirly coming
> > for the reduction in the base image
> > >   - nova-libvirt:
> > >       + Kolla: 2.14 GB
> > >       - new: 1.9 GB
> > again here you have to do the same arithmatich  so 2.14-1.09 so this
> image is adding 1.05 GB of layers in the kolla case
> > and the ubi version is adding 1.2GB so the ubi image is acully using
> more space assumign tis using layer if tis not then
> > its 1.05GB vs 1.9GB and the kolla image still comes out better by an
> even larger margin
> > >   - keystone:
> > >       + Kolla: 973 MB
> > >       - new: 532 MB
> >
> > again here this si all in the deleta of the base image
> > >   - memcached:
> > >       + Kolla: 633 MB
> > >       - new: 379 MB
> >
> > as is this
> >
> > so over all i think the ubi based iamges are using more or the same
> space then the kolla ones.
> > they just have a smaller base image. so rather then doing this i think
> it makes more sense to just use the ubi iamge
> > with the kolla build system unless you expect be be able to signicicalty
> reduce the size of the images more.
> >
> > based on size alone i tone se anny really benifit so far
> > >
> > > While the links shown are many, the actual volume of the proposed
> change is
> > > small, although the impact is massive:
> >
> > > * With more straightforward to understand tools, we'll be able to get
> > > broader participation from the TripleO community to maintain our
> > > containerization efforts and extend our service capabilities.
> > given customer likely have built up ther own template override files
> unless you also have a way to support that this is
> > a breaking change to there workflow.
> > > * With smaller images, the TripleO community will serve OpenStack
> deployers
> > > and operators better; less bandwidth consumption and faster install
> times.
> > again when you take into the account that kolla uses layers the image
> dont appear to be smaller and the nova-libvert
> > image is actully bigger kolla gets large space saving buy using the copy
> on write layers that make up docker images
> > to share common code so you can just look a the size fo indivigual
> iamages you have look at the size of the total set
> > and compare those.
> >
> > >
> > > We're looking to chart a more reliable path forward, and making the
> TripleO
> > > user experience a great one. While the POC changes are
> feature-complete and
> > > functional, more work is needed to create the required variable files;
> > > however, should the proposed specification be ratified, we expect to
> make
> > > quick work of what's left. As such, if you would like to be involved or
> > > have any feedback on anything presented here, please don't hesitate to
> > > reach out.
> > >
> > > We aim to provide regular updates regarding our progress on the "Simple
> > > container generation" initiative, so stay tuned.
> > honestly im not sure this is a benifty or something that should be done
> but it seam like ye have already decieded on a
> > path. not the kolla image can also be made smaller then they currently
> are using multi state builds to remove and deps
> > installed for buidl requirement or reducing the optional depencies.
> >
> > many for the deps currently installed are to supprot the different
> vender backedns.
> > if we improve the bindep files in each project too group those depencies
> by the optional backedn kolla could easily be
> > updated to use bindep for package installation. and the image would get
> even smaller.
>
> The issues that we (tripleo) have primarily run into are the
> expectations around versions (rabbitmq being the latest) and being
> able to deploy via source.  Honestly if tripleo was going to support
> installations running source or alternative distros (which neither we
> currently plan to do), it would likely make sense to continue down the
> Kolla path. However we already end up doing so many overrides to the
> standard Kolla container configurations[0] that I'm not sure it really
> makes sense to continue. Additionally since we no longer support
> building via docker, we're basically using kolla as a glorified
> template engine to give us Dockerfiles.  The proposal is to stop using
> Kolla as a glorified templating engine and actually just manage one
> that fits our needs.  We're using a very specific path through the
> Kolla code and I'm uncertain if it's beneficial for either of us
> anymore.  This also frees up kolla + kolla-ansible improve their
> integration and likely be able to make some of the tougher choices
> that they've brought up in the other email about the future of kolla.
>
> Personally I see this move as freeing up Kolla to be able to innovate
> and TripleO being able to simplify.  As one of the few people who know
> how the container sausage is made in the TripleO world, I think it's
> likely for the best.
>
> [0]
> https://opendev.org/openstack/tripleo-common/src/branch/master/container-images/tripleo_kolla_template_overrides.j2
>
> > >
> > > Thanks,
> > >
> > > Kevin and Emilien
> > >
> > > [0] https://review.opendev.org/#/c/723665/
> > > [1] https://review.opendev.org/#/c/722557/
> > > [2] https://review.opendev.org/#/c/724147/
> > > [3] https://review.opendev.org/#/c/722486/
> > > [4] https://files.macchi.pro:8443/demo-container-images/
> > > [5] http://paste.openstack.org/show/792995/
> > > [6] http://paste.openstack.org/show/792994/
> > > [7] http://paste.openstack.org/show/792993/
> > > [8]
> > >
> >
> https://4ce3fa2efa42bb6b3a69-771991cd07d409aaec3e4ca5eafdd7e0.ssl.cf2.rackcdn.com/724436/2/check/tripleo-build-containers-centos-8/78eefc3/logs/containers-successfully-built.log
> > >
> > > Kevin Carter
> > > IRC: kecarter
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200501/6f43f097/attachment.html>