Open Stack

On Wed, Oct 7, 2020, at 1:18 PM, Julia Kreger wrote:
> On Wed, Oct 7, 2020 at 11:16 AM Mark Goddard <mark at stackhpc.com> wrote:
> >
> > On Wed, 7 Oct 2020 at 16:11, Riccardo Pittau <elfosardo at gmail.com> wrote:
> > >
> > > Hello fellow openstackers!
> > >
> > > At the moment it's not possible to migrate the ironic-python-agent-builder src jobs from bionic to focal nodeset because of diskimage-builder limitations.
> > > We're stuck with ubuntu bionic and we're pinning those jobs to the bionic nodeset for the time being:
> > > https://review.opendev.org/756291
> > >
> > > One of the community goals for victoria is to move the base nodeset of the CI jobs from ubuntu bionic to focal.
> > > In general, doing this for most of the ironic projects has not been trivial, but still doable, and it has been accomplished almost entirely.
> > > The biggest challenge comes from the src jobs in ironic-python-agent-builder where, for some of them, we build ironic-python-agent ramdisks using rpm-based distributions (mainly centos) with diskimage-builder on ubuntu bionic.
> > > This is possible using utilities (e.g. yumdownloader) included in packages still present in the ubuntu repositories, such as yum-utils and rpm.
> > > Starting from Ubuntu focal, the yum-utils package has been removed from the repositories because of lack of support of Python 2.x and there's no plan to provide such support, at least to my knowledge.
> > > The alternative provided by dnf is not usable as there's also no plan to compile and provide a package of dnf for deb-based distributions.
> > > For the reasons mentioned above, currently the ironic project team can't complete the migration of the CI jobs from bionic to focal and there's no ETA on when this can be accomplished.
> > >
> > > Considering all the things in the preamble, two possibilities are available:
> > > - change the mechanics in diskimage-builder; this process would completely change the way DIB builds rpm-based distros; this approach delegates the work almost entirely to the DIB team.
> > > - instead of migrating to focal, migrate to centos-8 nodeset; that would mean having devstack+ironic working on centos-8, which poses an interesting challenge and would consume no little resources from the ironic team.
> > >
> > > Opinions and advice are very welcome!
> >
> > My first reaction would be to consider the user impact beyond the box
> > checking of fulfilling a goal. Does this imply Ubuntu users can no
> > longer build IPA images from Focal? That would be a shame.
> 
> The goals are really meant to drive the community together as a group.
> And if box checking is not appropriate, then it is not appropriate. I
> think the key is in the meaning of the goal which is to drive everyone
> forward together. I do concur end user impact is the key item to focus
> on. I don't think this is helped due to pre-existing constraints. I
> seem to remember that you couldn't build ubuntu on centos previously,
> so this sort of issue does not surprise me. At least, not without
> having some extra packages present that one could install and it might
> work with some hope.
> 
> My guess is that we're effectively entering a situation where if I
> want to build a centos/rhel/fedora IPA image, I need to run the ipa
> builder command on one of those machine types, and if I want the same
> for debian/ubuntu, I need to run the build on one of those operating
> systems. Is that situation horrible for users, not really because they
> are and likely should keep the distribution the same for familiarity
> and compatibility. The thing we likely need to do is do an ubuntu IPA
> image test in CI but not save the artifact. Or debian!
> 

There has actually been some thought on this problem recently. Those tools are all used to bootstrap a chroot with the necessary components to then build the rest of the image. Fortunately there are other approaches that can be taken in DIB to get to that point. Some of the elements start with a preexisting cloud image for example (ubuntu does this where ubuntu-minimal starts with debootstrap).

More recently the thought has been that we should probably think about bootstrapping with docker images because basically all the distros publish such a thing. That element exists and is called "docker" [0] but may need more testing [1]. This is attractive because it means we should be able to run on just about any distro as long as the DIB host's kernel doesn't conflict with the userspace in the docker container used to bootstrap image building.

I think the work here has largely stalled out, but I'm sure help would be welcome if Ironic and others think this is a useful approach to take. I'm not in a great spot to pick this up myself, but can probably help out if someone else is driving it (reviews, testing assistance, etc)

[0] https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/docker
[1] https://review.opendev.org/#/c/700041/