Open Stack

On Thursday, 8 October 2020 17:38:57 CEST Clark Boylan wrote:
> On Thu, Oct 8, 2020, at 1:17 AM, Luigi Toscano wrote:
> > On Thursday, 8 October 2020 06:18:35 CEST Ian Wienand wrote:
> > > On Wed, Oct 07, 2020 at 05:09:56PM +0200, Riccardo Pittau wrote:
> > > > This is possible using utilities (e.g. yumdownloader) included in
> > > > packages
> > > > still present in the ubuntu repositories, such as yum-utils and rpm.
> > > > Starting from Ubuntu focal, the yum-utils package has been removed
> > > > from
> > > > the
> > > > repositories because of lack of support of Python 2.x and there's no
> > > > plan
> > > > to provide such support, at least to my knowledge.
> > > 
> > > Yes, this is a problem for the "-minimal" elements that build an
> > > non-native chroot environment.  Similar issues have occured with Suse
> > > and the zypper package manager not being available on the build host.
> > > 
> > > The options I can see:
> > > 
> > > - use the native build-host; i.e. build on centos as you described
> > > 
> > > - the non-minimal, i.e. "centos" and "suse", for example, images might
> > > 
> > >   work under the current circumstances.  They use the upsream ISO to
> > >   create the initial chroot.  These are generally bigger, and we've
> > >   had stability issues in the past with the upstream images changing
> > >   suddenly in various ways that were a maintenance headache.
> > > 
> > > - use a container for dib.  DIB doesn't have a specific container, but
> > > 
> > >   is part of the nodepool-builder container [1].  This is ultimately
> > >   based on Debian buster [2] which has enough support to build
> > >   everything ... for now.  As noted this doesn't really solve the
> > >   problem indefinitely, but certainly buys some time if you run dib
> > >   out of that container (we could, of course, make a separate dib
> > >   container; but it would be basically the same just without nodepool
> > >   in it).  This is what OpenDev production is using now, and all the
> > >   CI is ultimately based on this container environment.
> > > 
> > > - As clarkb has mentioned, probably the most promising alternative is
> > > 
> > >   to use the upstream container images as the basis for the initial
> > >   chroot environments.  jeblair has done most of this work with [3].
> > >   I'm fiddling with it to merge to master and see what's up ... I feel
> > >   like maybe there were bootloader issues, although the basic
> > >   extraction was working.  This will allow the effort put into
> > >   existing elements to not be lost.
> > > 
> > > If I had to pick; I'd probably say that using the nodepool-builder
> > > container is the best path.  That has the most momentum behind it
> > > because it's used for the OpenDev image builds.  As we work on the
> > > container-image base elements, this work will be deployed into the
> > > container (meaning the container is less reliant on the underlying
> > > version of Debian) and you can switch to them as appropriate.
> > 
> > I have to mention at this point, at risk of reharshing old debates, that
> > an
> > alternative in various scenarios (maybe not all) is the usage of
> > libguestfs
> > and its tools which modifies an existing base image.
> > 
> > https://libguestfs.org/
> > 
> > We switched to it in Sahara for most of the guest images and that saved
> > some headaches when building from a different host.
> > https://docs.openstack.org/sahara/latest/user/building-guest-images.html
> > 
> > 
> > I'd like to mention that libguestfs has been carrying a virt-dib tool for
> > a
> > while, but it has been tested only back to a certain version of dib:
> > https://libguestfs.org/virt-dib.1.html
> 
> The major issues with libguestfs is that it seem primarily used to modify
> existing images. This has all of the problems that ianw points out above
> with size and stability. I'm sure you can bootstrap a base image to use
> with it too, but would you end up in the same toolchain problem as with dib
> if you did that?
> 
> Also, to be clear DIB supports the same use case of starting from an
> existing image (but using different tools) if you want to avoid the
> headaches with bootstrapping images yourself.
> 
> I think the more specific issue we're trying to figure out is "how do you
> bootstrap images from scratch for one distro on top of another if they use
> different bootstrapping tools". I don't think libguestfs helps with that
> much. And if you are starting from an existing image then DIB or libguestfs
> should work fine.

Uhm, maybe unattended virt-install could help there?

-- 
Luigi