On Thursday, 8 October 2020 06:18:35 CEST Ian Wienand wrote:
On Wed, Oct 07, 2020 at 05:09:56PM +0200, Riccardo Pittau wrote:
This is possible using utilities (e.g. yumdownloader) included in packages still present in the ubuntu repositories, such as yum-utils and rpm. Starting from Ubuntu focal, the yum-utils package has been removed from the repositories because of lack of support of Python 2.x and there's no plan to provide such support, at least to my knowledge.
Yes, this is a problem for the "-minimal" elements that build an non-native chroot environment. Similar issues have occured with Suse and the zypper package manager not being available on the build host.
The options I can see:
- use the native build-host; i.e. build on centos as you described
- the non-minimal, i.e. "centos" and "suse", for example, images might work under the current circumstances. They use the upsream ISO to create the initial chroot. These are generally bigger, and we've had stability issues in the past with the upstream images changing suddenly in various ways that were a maintenance headache.
- use a container for dib. DIB doesn't have a specific container, but is part of the nodepool-builder container [1]. This is ultimately based on Debian buster [2] which has enough support to build everything ... for now. As noted this doesn't really solve the problem indefinitely, but certainly buys some time if you run dib out of that container (we could, of course, make a separate dib container; but it would be basically the same just without nodepool in it). This is what OpenDev production is using now, and all the CI is ultimately based on this container environment.
- As clarkb has mentioned, probably the most promising alternative is to use the upstream container images as the basis for the initial chroot environments. jeblair has done most of this work with [3]. I'm fiddling with it to merge to master and see what's up ... I feel like maybe there were bootloader issues, although the basic extraction was working. This will allow the effort put into existing elements to not be lost.
If I had to pick; I'd probably say that using the nodepool-builder container is the best path. That has the most momentum behind it because it's used for the OpenDev image builds. As we work on the container-image base elements, this work will be deployed into the container (meaning the container is less reliant on the underlying version of Debian) and you can switch to them as appropriate.
I have to mention at this point, at risk of reharshing old debates, that an alternative in various scenarios (maybe not all) is the usage of libguestfs and its tools which modifies an existing base image. https://libguestfs.org/ We switched to it in Sahara for most of the guest images and that saved some headaches when building from a different host. https://docs.openstack.org/sahara/latest/user/building-guest-images.html I'd like to mention that libguestfs has been carrying a virt-dib tool for a while, but it has been tested only back to a certain version of dib: https://libguestfs.org/virt-dib.1.html -- Luigi