[DIB][diskimage-builder] Rocky Linux image build method
Hello,
I am curious about the choice of providing only a rocky-container element in DIB, which works differently to the centos element, which uses cloud images.
It makes it hard to produce working images for VMs or bare metal, as various packages that would normally be installed are missing, such as cloud-utils-growpart or openssh-server. See the kickstarts for reference [1] [2].
It seems to also occasionally cause complex failures such as the one that rendered Rocky Linux 8 images unbootable last week [3]. I am guessing this wouldn't have happened had the build been from a cloud image.
Would the DIB community be open to also have a rocky element using GenericCloud images, like centos?
Thanks, Pierre Riteau (priteau)
[1] https://git.rockylinux.org/rocky/kickstarts/-/blob/r8/Rocky-8-Container-Base... [2] https://git.rockylinux.org/rocky/kickstarts/-/blob/r8/Rocky-8-GenericCloud.k... [3] https://review.opendev.org/c/openstack/diskimage-builder/+/851687
On 2022-08-01 10:21:18 +0200 (+0200), Pierre Riteau wrote:
I am curious about the choice of providing only a rocky-container element in DIB, which works differently to the centos element, which uses cloud images.
It makes it hard to produce working images for VMs or bare metal, as various packages that would normally be installed are missing, such as cloud-utils-growpart or openssh-server. See the kickstarts for reference [1] [2].
It seems to also occasionally cause complex failures such as the one that rendered Rocky Linux 8 images unbootable last week [3]. I am guessing this wouldn't have happened had the build been from a cloud image.
Would the DIB community be open to also have a rocky element using GenericCloud images, like centos?
[...]
At least for images we're booting in OpenDev, we've been gradually switching them over to the containerfile mechanism for the specific reason that it's increasingly hard to guarantee random distros' package bootstrapping tools can all be made to work outside a chroot on a single common/foreign distro, and we don't want "fat" images preinstalled with a lot of unnecessary packages (and certainly not with things like cloud-init).
For CentOS image building, we're still relying on the centos-minimal element, because we're able to install a working release of yum outside the image on the builder, but there have been times where some distros tools were simply not available for installation or required too new (or too old) system libraries to be able to run.
On Mon, 2022-08-01 at 10:21 +0200, Pierre Riteau wrote:
Hello,
I am curious about the choice of providing only a rocky-container element in DIB, which works differently to the centos element, which uses cloud images.
As Jeremy mentions, it was a conscious choice to use the containerfile method to build Rocky images for the reasons he discusses. It is more in line with the Fedora images, and from a building perspective, it's often better to layer things on top, rather than try and remove them afterwards.
It makes it hard to produce working images for VMs or bare metal, as various packages that would normally be installed are missing, such as cloud-utils-growpart or openssh-server. See the kickstarts for reference [1] [2].
It seems to also occasionally cause complex failures such as the one that rendered Rocky Linux 8 images unbootable last week [3]. I am guessing this wouldn't have happened had the build been from a cloud image.
Would the DIB community be open to also have a rocky element using GenericCloud images, like centos?
It's possible it may have not happened when building from a Cloud image, but you similarly begin to rely on the upstream to produce images the same way forever. Building from a known minimal source and layering on the elements required, including writing any boot/kernel files needed will a more manageable process for the DIB community, in my opinion (as the one who makes the Upstream images for Rocky). Building from the ground up guarantees a much more repeatable process for DIB image building.
In the short term, there's some uplift to make sure the images build correctly, but once they're working, working from Containerfile should lead to faster builds of more lean images with only what's needed on them-- and that's good for Security, too.
Please do reach out via email or IRC/Chat (either of OFTC or Libera in the #rockylinux- channels) any time, by the way :)
I'm one of the Infrastructure leads for Rocky Linux and spend time over here with the OSA folks! Nice to meet you, and thank you for using Rocky!
Thanks, Pierre Riteau (priteau)
[1] https://git.rockylinux.org/rocky/kickstarts/-/blob/r8/Rocky-8-Container-Base... [2] https://git.rockylinux.org/rocky/kickstarts/-/blob/r8/Rocky-8-GenericCloud.k... [3] https://review.opendev.org/c/openstack/diskimage-builder/+/851687
On Mon, Aug 01, 2022 at 10:21:18AM +0200, Pierre Riteau wrote:
It seems to also occasionally cause complex failures such as the one that rendered Rocky Linux 8 images unbootable last week [3]. I am guessing this wouldn't have happened had the build been from a cloud image. [3] https://review.opendev.org/c/openstack/diskimage-builder/+/851687
Others have responded and I don't really have anything to add; I would just say that the reason we moved from the cloud images many years ago was because various things just like [3] kept happening.
We used to "simply" boot generic upstream images on clouds and run an array of scripts to pre-configure them, take a snapshot image and then boot off that for the day. It was constantly breaking :)
The centos element has had it's fair share of "interesting" issues with disk-layouts, etc.
Would the DIB community be open to also have a rocky element using GenericCloud images, like centos?
In terms of having your dependencies available in final images, nothing really beats specifying them explicitly. Something like [1]. I think I'd encourage this rather than trying to add another platform to support in dib.
-i
[1] https://opendev.org/openstack/project-config/src/branch/master/nodepool/elem...
On 2022-08-02 12:17:58 +1000 (+1000), Ian Wienand wrote: [...]
In terms of having your dependencies available in final images, nothing really beats specifying them explicitly. Something like [1]. I think I'd encourage this rather than trying to add another platform to support in dib.
[...]
And, just to be clear, in OpenDev our goal is to have only the things necessary to start the VM and connect with Ansible. We do also cache some expensive-to-retrieve things like Git repositories and files nearly every job is otherwise going to try to fetch over the network into these images, in order to improve job startup times and stability, but job-specific or project-specific dependencies should be explicitly installed by the jobs themselves.
I've proposed a change to make a more feature complete rocky element here: https://review.opendev.org/c/openstack/diskimage-builder/+/855521. I was trying to make an image that is compatible with the officially distributed generic cloud images. It is essentially just layering on some additional packages on top of the rocky-container element. Would love to have some more feedback.
On Tue, 2 Aug 2022 at 13:08, Jeremy Stanley fungi@yuggoth.org wrote:
On 2022-08-02 12:17:58 +1000 (+1000), Ian Wienand wrote: [...]
In terms of having your dependencies available in final images, nothing really beats specifying them explicitly. Something like [1]. I think I'd encourage this rather than trying to add another platform to support in dib.
[...]
And, just to be clear, in OpenDev our goal is to have only the things necessary to start the VM and connect with Ansible. We do also cache some expensive-to-retrieve things like Git repositories and files nearly every job is otherwise going to try to fetch over the network into these images, in order to improve job startup times and stability, but job-specific or project-specific dependencies should be explicitly installed by the jobs themselves. -- Jeremy Stanley
One thing to note is that Rocky cloud images are unofficial or, as they phrase it, "not officially supported" [1] which boils down to the same thing IMHO...
I agree with Ian that it is probably enough that downstreams support their customised flavours as there is no set-in-stone guideline how a cloud image should look like.
[1] https://rockylinux.org/alternative-images
Cheers, Radek
On Wed, 7 Sept 2022 at 11:05, William Szumski will@stackhpc.com wrote:
I've proposed a change to make a more feature complete rocky element here: https://review.opendev.org/c/openstack/diskimage-builder/+/855521. I was trying to make an image that is compatible with the officially distributed generic cloud images. It is essentially just layering on some additional packages on top of the rocky-container element. Would love to have some more feedback.
On Tue, 2 Aug 2022 at 13:08, Jeremy Stanley fungi@yuggoth.org wrote:
On 2022-08-02 12:17:58 +1000 (+1000), Ian Wienand wrote: [...]
In terms of having your dependencies available in final images, nothing really beats specifying them explicitly. Something like [1]. I think I'd encourage this rather than trying to add another platform to support in dib.
[...]
And, just to be clear, in OpenDev our goal is to have only the things necessary to start the VM and connect with Ansible. We do also cache some expensive-to-retrieve things like Git repositories and files nearly every job is otherwise going to try to fetch over the network into these images, in order to improve job startup times and stability, but job-specific or project-specific dependencies should be explicitly installed by the jobs themselves. -- Jeremy Stanley
participants (6)
-
Ian Wienand
-
Jeremy Stanley
-
Neil Hanlon
-
Pierre Riteau
-
Radosław Piliszek
-
William Szumski