On 20/06/2025 14:55, prmjh4@gmail.com wrote:
Hello OpenStack Community Folks, NOTE: Below by the term templateless I simply mean to Provision the Server on the OpenStack Cloud without using the existing images that are already present on the OpenStack cloud.
that not a supported workflow today. as a cloud platform our workflows are optimized for running hundreds of semi identical vms that are customized post intall rather then proving a way to create 1 off pet instances. i say semi identical because per vm customization is supported using user-data https://docs.openstack.org/nova/latest/user/metadata.html#user-data user data allows you to provide cloud-config or ignition configs to a first boot agent to customize the guest.
I again thank you for the above valuable inputs that Community Folks have shared and on the similar lines now I am looking for final guidance and best practices on how to achieve templateless server provisioning (the one without using the image present in the OpenStack already) in OpenStack, where the goal is to provision a VM by booting from a custom ISO rather than cloning an existing image from Glance.
Our current understanding and approach (inspired by similar workflows in other virtualization platforms like VMware)
to be very clear on that point OpenStack is not an enterprise virtualization platform. it is a cloud platform like aws, azure or gcp. you should not expect the workflows that are supported on an enterprise virutal8sation platform to be supported in a cloud platform in general as they are built to optimist for cloud workflows. openstack does not have a way of attaching viuratal media ot a guest for install on boot funcitonality or the ablity to boot form a http url without a glance image.
involves the following steps:
1. VM Creation and Custom ISO Preparation:
1.1 Create a new VM instance in OpenStack. 1.2 Dynamically modify/create a custom ISO (either a Windows PE-based ISO or a Linux-based ISO, such as one built on SySRescueCD). 1.3 This ISO is customized to embed crucial network configuration details (e.g., static IP, DHCP settings, multiple NIC configurations, routes). These details are processed by scripts within the ISO upon boot. 1.4 The ISO also contains scripts designed to communicate guest information back to the hypervisor, similar to VMware Tools executables. For WinPE, this involves manual command invocation to set VM info values.
2. ISO Deployment and VM Boot:
2.1 The modified custom ISO needs to be uploaded to an accessible location within OpenStack (e.g., Glance for ephemeral use, Swift, or an external web server). 2.2. This custom ISO then needs to be attached to the newly created VM instance, and the VM configured to boot from this attached ISO.
right this does not fit with the cloud model. in the cloud model you have a generic image that is shared by many (100s or 1000s) of VMs and customization is done on first boot with cloud-init/glean or post boot with ansible or other framewoks. https://github.com/canonical/cloud-init, https://opendev.org/opendev/glean creating per vms image or install media si an anti pattern. you can use tools like deboostrap or sysprep to prepare they images ot do customisation on first boot but that shoudl be geneeric enouch that it is reusable across many vms. https://docs.openstack.org/image-guide/ is our offial docuemation on how to build resuabel images for opentack. https://docs.openstack.org/image-guide/openstack-images.html descibe the standed expecations for cloud images. here is an example with ubuntu https://docs.openstack.org/image-guide/create-images-manually-example-ubuntu... but here are some automated tools as well https://docs.openstack.org/image-guide/create-images-automatically.html for example https://github.com/cloudbase/windows-imaging-tools was mained by cloudbase solution for windows images in the past. but more generic tools like https://developer.hashicorp.com/packer or even openstack disk image builder are probably a better choice.
3. Specific Questions and Areas for Community Input:
My primary challenges and questions revolve around how OpenStack can facilitate this process, particularly regarding:
3.1 Guest-to-Hypervisor Communication: What are the recommended OpenStack-native tools or mechanisms to enable the guest OS (booting from our custom ISO) to notify the hypervisor of its details? I am unsure if qemu-guest-agent is suitable for this specific boot-from-ISO scenario, or if other methods (e.g., metadata service interactions) are more appropriate.
that would generally be considered a security risk. there shoudl be no gust to host communication channel but the way this si normally done is via a qemu guest agent. the openstack community tool for building guest images is https://github.com/openstack/diskimage-builder and rackspace have an example agent that is based on how they did this https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder... https://github.com/Rackspace-DOT/nova-agent
3.2 Initial Guest Configuration: How can user data or initial configuration information (like the network settings we're embedding in the ISO) be effectively provided to a VM during this templateless provisioning flow in OpenStack? This includes handling complex networking scenarios with both internal and external DHCP servers. Is cloud-init (or its Windows equivalent) the recommended approach, and if so, how can it be leveraged when booting from a custom ISO?
the cloud image shoudl have a first boot init package like cloud-init or glen preinstalled opentack (nova) has a metadata component that will provide networking information automatically to the guest and you can use our config drive support to have nova automatically put that into an iso that is attached to the guest. https://docs.openstack.org/nova/latest/user/metadata.html if you as an operator want to providee addtional info beyond the standard set you have 2 options using the vendor data feature https://docs.openstack.org/nova/latest/user/metadata.html#vendordata ther eare two types of vender data https://docs.openstack.org/nova/latest/admin/vendordata.html#staticjson and https://docs.openstack.org/nova/latest/admin/vendordata.html#dynamicjson static vendor data is for somethin that is common to all vms the dynamic vendor data woudl allow you to run a simple webserver/rest api to provide per vm data.
3.3 Programmatic Network Utilization: My current OpenStack environment has limited default networking. I am keen to understand how to programmatically utilize and attach custom-created networks (beyond the default setup) via OpenStack API calls during the instance provisioning process.
this is something that the end user who is creating the vm should do. they can pass multiple neutron port or networks to nova when creating the vm and they will be attached to the vm when its created.
I am looking for insights on programmatically implementing this "boot from custom ISO" provisioning method in OpenStack, rather than relying on pre-existing Glance images.
you are going to have to build your own tooling for this. many people use terreform or ansible to do this type of customiastion by starting form a pre-existing generic glance image. if you dont want to do that and you not going to use boot form volume then you cant do this today. the closest thing you can do is either via an agent in the iso or some other method backed into a glance image. but its not a workflow that is natively supported in openstack. its defintly possibel to do as talos have done this https://www.talos.dev/v1.7/talos-guides/install/cloud-platforms/openstack/ but they did it in a cloud native way where the have a standard image that you boot into and then there install agent loads into ram and overwrites the root disk. core-os uses ignition script to do somethign very similar. https://docs.fedoraproject.org/en-US/fedora-coreos/provisioning-openstack/
Any valuable inputs, sample code implementation or examples, or pointers to relevant documentation would be highly appreciated.
Thank you all for your time and assistance!
Best regards, Prem