Thanks for all the info. Replying inline
On Wed, Jul 17, 2019 at 4:47 PM Jeremy Stanley fungi@yuggoth.org wrote:
On 2019-07-17 12:48:53 -0700 (-0700), Clark Boylan wrote: [...]
With your image build you can preinstall all of the software you need (and configure it too). You can also configure disk partitioning. The tool the OpenDev Infra team uses this for all of our test node images and some of our control plane images is called diskimage-builder [0]. This tool can partition images using both mbr and gpt tables.
[...]
Thank you for the info on diskimage-builder. From what I understood in https://docs.openstack.org/diskimage-builder/latest/user_guide/building_an_i..., its output is a partitoned disk image configured whatever way you want, which can then be fed to "openstack server create." After that it is a matter of installing the OS as you would in https://docs.openstack.org/image-guide/ubuntu-image.html. Correct me if I am wrong but that means customization done before OS is installed required separate images. Stuff like adding network cards can be customized because even adding the card itself can be done at any time.
I think I now understand that.
I gather there's another popular alternative, which is to boot a server instance with an ISO of an operating system installer image attached, use that to configure the supplied root disk and install software into it, then shut that down and create a snapshot of the resulting instance's rootfs to boot additional instances from. It's a more traditional analog to how you'd do server installation in classic non-cloudy environments, but some folks still seem to prefer it over creating images of fully-working servers and then uploading those. That said, I'd recommend sticking to using/modifying officially provided "cloud" images from the distros you want since they do some useful things like wiping system identifiers and host keys so that they're regenerated independently on each new instance booted from them (if you don't remember to do that on your custom images you can run afoul of support licensing contracts or create unnecessary security risks). -- Jeremy Stanley
I think the merit of having a secure barebones image and then using the usual suspects -- ansible, chef -- to install and configure packages cuts down on how many images you have to keep current on the hard drive. They can force server keys if needed. Having customized images with some software installed and configured will lead to faster deployment, however.