[dev] OpenStack Templateless Provisioning
Do we have any mechanism in OpenStack environment where we do not need an appropriate template already exists in the OpenStack environment in order to create server instance on the OpenStack Cloud programmatically? where we can do the templateless provisioning of the server instance on OpenStack?
On 2025-06-12 12:03:14 -0000 (-0000), prmjh4@gmail.com wrote:
Do we have any mechanism in OpenStack environment where we do not need an appropriate template already exists in the OpenStack environment in order to create server instance on the OpenStack Cloud programmatically? where we can do the templateless provisioning of the server instance on OpenStack?
There are a variety of options... The OpenStackClient CLI can be used in scripts, or the OpenStackSDK could be imported in a Python program (there are lots of unofficial third-party SDKs out there for other languages too), or you could use an HTTP library in your software to directly call OpenStack's various REST APIs. See https://developer.openstack.org/ for more details on these options. -- Jeremy Stanley
Thanks Jeremy, I tried looking at the OpenStackSDK which we can import in a Python program and then can try creating server on OpenStack but the thing is that It always asked me to pass a template image in order to create the server without the template image it does not allow me to create a server. Even if there is an way to create a server without using templates then I couldn't find that any. Is there any specific sample python code snippet that you ever used using OpenStackSDK in Python to create server on OpenStack without using template images then too share with me it would be a great help. Thanks! -Prem
It feels like the question is sort of along the lines of: What are the clear and concise steps to deploy an instance *without* having any prior templates or base artifacts in the cloud. Is that sort of along the same lines? AFAIK, there is not really an inherent default capacity to, using a REST API to create a disk image from almost nothing, and then request that image be deployed. I guess in the context of this sequence of thoughts, a template is also any artifact which exists in the cloud already, and operating system images are always template based on some level, which makes this challenging. I guess it would be an interesting thought experiment, and ultimately might be worthwhile feature wise, to enable getting something from basically nothing but credentials. On Thu, Jun 12, 2025 at 6:57 AM <prmjh4@gmail.com> wrote:
Thanks Jeremy, I tried looking at the OpenStackSDK which we can import in a Python program and then can try creating server on OpenStack but the thing is that It always asked me to pass a template image in order to create the server without the template image it does not allow me to create a server. Even if there is an way to create a server without using templates then I couldn't find that any. Is there any specific sample python code snippet that you ever used using OpenStackSDK in Python to create server on OpenStack without using template images then too share with me it would be a great help. Thanks! -Prem
On 2025-06-12 13:57:17 -0000 (-0000), prmjh4@gmail.com wrote: [...]
It always asked me to pass a template image in order to create the server without the template image it does not allow me to create a server. [...]
Okay, I didn't understand what you meant by "template" in your first message, since in OpenStack we refer to "images" for what you seem to be talking about. No, as far as I know there is no way to create a server instance without the cloud provider or the user supplying a disk image to boot from. In the case of most OpenStack clouds, the operator of the cloud environment includes a catalogue of generic images for different operating systems and you would select one of those to boot as the initial state of the server. If there are none or the available options aren't suitable, the user can typically upload an image (these are most often in qcow2 or raw disk format, like the ones offered as "cloud image" downloads by may GNU/Linux distributions). It's also possible to use a tool like diskimage-builder to create or customize an image of your own and then upload that. I guess what you're hoping for is a way to boot a server instance with an empty disk and then run some sort of interactive operating system installer through the server's remote console? I'm not aware of any options for that... maybe there's a trick someone else knows? I've added the [ops] and [nova] subject tags in hopes of drawing broader attention to your question, since [dev] is mostly used for discussions about developing OpenStack itself, not developing other software that interacts with OpenStack interfaces or resources. -- Jeremy Stanley
I've done OS install by booting to install iso but add a disk to the VM, install to disk, then create image from volume or snapshot or something like that I could use as an image. Internal Use - Confidential -----Original Message----- From: Jeremy Stanley <fungi@yuggoth.org> Sent: Thursday, June 12, 2025 9:45 AM To: openstack-discuss@lists.openstack.org Subject: Re: [dev][ops][nova] OpenStack Templateless Provisioning On 2025-06-12 13:57:17 -0000 (-0000), prmjh4@gmail.com wrote: [...]
It always asked me to pass a template image in order to create the server without the template image it does not allow me to create a server. [...]
Okay, I didn't understand what you meant by "template" in your first message, since in OpenStack we refer to "images" for what you seem to be talking about. No, as far as I know there is no way to create a server instance without the cloud provider or the user supplying a disk image to boot from. In the case of most OpenStack clouds, the operator of the cloud environment includes a catalogue of generic images for different operating systems and you would select one of those to boot as the initial state of the server. If there are none or the available options aren't suitable, the user can typically upload an image (these are most often in qcow2 or raw disk format, like the ones offered as "cloud image" downloads by may GNU/Linux distributions). It's also possible to use a tool like diskimage-builder to create or customize an image of your own and then upload that. I guess what you're hoping for is a way to boot a server instance with an empty disk and then run some sort of interactive operating system installer through the server's remote console? I'm not aware of any options for that... maybe there's a trick someone else knows? I've added the [ops] and [nova] subject tags in hopes of drawing broader attention to your question, since [dev] is mostly used for discussions about developing OpenStack itself, not developing other software that interacts with OpenStack interfaces or resources. -- Jeremy Stanley
On 12/06/2025 22:16, Barrera, Eric wrote:
I've done OS install by booting to install iso but add a disk to the VM, install to disk, then create image from volume or snapshot or something like that I could use as an image.
ya so here are the options nova allows you to boot form an existing image in glance, that image can be a root disk image, an iso, or separate images containing a kernel + ram disk + filesystem https://docs.openstack.org/nova/latest/user/launch-instances.html https://docs.openstack.org/nova/latest/user/launch-instance-using-ISO-image.... that is the primary way of create a server. the next option is to boot form a cinder volume. you can precreate the volume with the root files system or you can ask nova to do that from a glance image. https://docs.openstack.org/nova/latest/user/launch-instance-from-volume.html the third option is a variation of the first two. you can create a "empty" image that just has a a valid gpt partion table. in the case of a volume you can create a blank volume directly. this will result in a server that cannot boot to an os but will start and error in the bios. at this point you have 3 sub options 1 you can use rescure with an iso and manually do an install https://docs.openstack.org/nova/latest/user/rescue.html 2 if you add metadta to your neutorn network you can use the ipxe booth functionaltiy that they supprot for ironic by using an ipxe rescure imagne https://docs.openstack.org/nova/latest/user/boot-instance-using-PXE.html 3 if the blank image/volume has the correct image properites you can enable the vms boot menue and access the ipxe capablites build into the uefi enviornment to install. to do this you need to set hw_boot_menu=true in the image or hw:boot_menu=true in the flavor. your 4th option is to booth form a blank cinder voluem and attch a second cinder volume with your install media. https://docs.openstack.org/nova/latest/user/block-device-mapping.html#block-... this is mujch more complex but you can ask nova to create a service rwith a black local or cinder root disk. you can also create a secon cinder volume and attach it as a cdrom form na iso in glance nad use that to install form by usign the boot index atribute on the block device mappigns we do not have docs for this as doing it that way is very complex and even core reviers will get this wrong first tiem. its just too erro prone to do and we dont test it. nova does not directly support virutal media like you might have in some vituraliation enviomnents nor does it offically supprot pxe booting but it is possible to do.
Internal Use - Confidential -----Original Message----- From: Jeremy Stanley <fungi@yuggoth.org> Sent: Thursday, June 12, 2025 9:45 AM To: openstack-discuss@lists.openstack.org Subject: Re: [dev][ops][nova] OpenStack Templateless Provisioning
On 2025-06-12 13:57:17 -0000 (-0000), prmjh4@gmail.com wrote: [...]
It always asked me to pass a template image in order to create the server without the template image it does not allow me to create a server. [...]
Okay, I didn't understand what you meant by "template" in your first message, since in OpenStack we refer to "images" for what you seem to be talking about.
No, as far as I know there is no way to create a server instance without the cloud provider or the user supplying a disk image to boot from. In the case of most OpenStack clouds, the operator of the cloud environment includes a catalogue of generic images for different operating systems and you would select one of those to boot as the initial state of the server. If there are none or the available options aren't suitable, the user can typically upload an image (these are most often in qcow2 or raw disk format, like the ones offered as "cloud image" downloads by may GNU/Linux distributions). It's also possible to use a tool like diskimage-builder to create or customize an image of your own and then upload that.
I guess what you're hoping for is a way to boot a server instance with an empty disk and then run some sort of interactive operating system installer through the server's remote console? I'm not aware of any options for that... maybe there's a trick someone else knows?
I've added the [ops] and [nova] subject tags in hopes of drawing broader attention to your question, since [dev] is mostly used for discussions about developing OpenStack itself, not developing other software that interacts with OpenStack interfaces or resources. -- Jeremy Stanley
Thank you all for sharing your valuable observations and helping me out. I am going through all the possible options that are there and will return back with some more queries on it, and believe that I am get the same level of help and I got so far. Thanks again!
Hello OpenStack Community Folks, NOTE: Below by the term templateless I simply mean to Provision the Server on the OpenStack Cloud without using the existing images that are already present on the OpenStack cloud. I again thank you for the above valuable inputs that Community Folks have shared and on the similar lines now I am looking for final guidance and best practices on how to achieve templateless server provisioning (the one without using the image present in the OpenStack already) in OpenStack, where the goal is to provision a VM by booting from a custom ISO rather than cloning an existing image from Glance. Our current understanding and approach (inspired by similar workflows in other virtualization platforms like VMware) involves the following steps: 1. VM Creation and Custom ISO Preparation: 1.1 Create a new VM instance in OpenStack. 1.2 Dynamically modify/create a custom ISO (either a Windows PE-based ISO or a Linux-based ISO, such as one built on SySRescueCD). 1.3 This ISO is customized to embed crucial network configuration details (e.g., static IP, DHCP settings, multiple NIC configurations, routes). These details are processed by scripts within the ISO upon boot. 1.4 The ISO also contains scripts designed to communicate guest information back to the hypervisor, similar to VMware Tools executables. For WinPE, this involves manual command invocation to set VM info values. 2. ISO Deployment and VM Boot: 2.1 The modified custom ISO needs to be uploaded to an accessible location within OpenStack (e.g., Glance for ephemeral use, Swift, or an external web server). 2.2. This custom ISO then needs to be attached to the newly created VM instance, and the VM configured to boot from this attached ISO. 3. Specific Questions and Areas for Community Input: My primary challenges and questions revolve around how OpenStack can facilitate this process, particularly regarding: 3.1 Guest-to-Hypervisor Communication: What are the recommended OpenStack-native tools or mechanisms to enable the guest OS (booting from our custom ISO) to notify the hypervisor of its details? I am unsure if qemu-guest-agent is suitable for this specific boot-from-ISO scenario, or if other methods (e.g., metadata service interactions) are more appropriate. 3.2 Initial Guest Configuration: How can user data or initial configuration information (like the network settings we're embedding in the ISO) be effectively provided to a VM during this templateless provisioning flow in OpenStack? This includes handling complex networking scenarios with both internal and external DHCP servers. Is cloud-init (or its Windows equivalent) the recommended approach, and if so, how can it be leveraged when booting from a custom ISO? 3.3 Programmatic Network Utilization: My current OpenStack environment has limited default networking. I am keen to understand how to programmatically utilize and attach custom-created networks (beyond the default setup) via OpenStack API calls during the instance provisioning process. I am looking for insights on programmatically implementing this "boot from custom ISO" provisioning method in OpenStack, rather than relying on pre-existing Glance images. Any valuable inputs, sample code implementation or examples, or pointers to relevant documentation would be highly appreciated. Thank you all for your time and assistance! Best regards, Prem
On 20/06/2025 14:55, prmjh4@gmail.com wrote:
Hello OpenStack Community Folks, NOTE: Below by the term templateless I simply mean to Provision the Server on the OpenStack Cloud without using the existing images that are already present on the OpenStack cloud.
that not a supported workflow today. as a cloud platform our workflows are optimized for running hundreds of semi identical vms that are customized post intall rather then proving a way to create 1 off pet instances. i say semi identical because per vm customization is supported using user-data https://docs.openstack.org/nova/latest/user/metadata.html#user-data user data allows you to provide cloud-config or ignition configs to a first boot agent to customize the guest.
I again thank you for the above valuable inputs that Community Folks have shared and on the similar lines now I am looking for final guidance and best practices on how to achieve templateless server provisioning (the one without using the image present in the OpenStack already) in OpenStack, where the goal is to provision a VM by booting from a custom ISO rather than cloning an existing image from Glance.
Our current understanding and approach (inspired by similar workflows in other virtualization platforms like VMware)
to be very clear on that point OpenStack is not an enterprise virtualization platform. it is a cloud platform like aws, azure or gcp. you should not expect the workflows that are supported on an enterprise virutal8sation platform to be supported in a cloud platform in general as they are built to optimist for cloud workflows. openstack does not have a way of attaching viuratal media ot a guest for install on boot funcitonality or the ablity to boot form a http url without a glance image.
involves the following steps:
1. VM Creation and Custom ISO Preparation:
1.1 Create a new VM instance in OpenStack. 1.2 Dynamically modify/create a custom ISO (either a Windows PE-based ISO or a Linux-based ISO, such as one built on SySRescueCD). 1.3 This ISO is customized to embed crucial network configuration details (e.g., static IP, DHCP settings, multiple NIC configurations, routes). These details are processed by scripts within the ISO upon boot. 1.4 The ISO also contains scripts designed to communicate guest information back to the hypervisor, similar to VMware Tools executables. For WinPE, this involves manual command invocation to set VM info values.
2. ISO Deployment and VM Boot:
2.1 The modified custom ISO needs to be uploaded to an accessible location within OpenStack (e.g., Glance for ephemeral use, Swift, or an external web server). 2.2. This custom ISO then needs to be attached to the newly created VM instance, and the VM configured to boot from this attached ISO.
right this does not fit with the cloud model. in the cloud model you have a generic image that is shared by many (100s or 1000s) of VMs and customization is done on first boot with cloud-init/glean or post boot with ansible or other framewoks. https://github.com/canonical/cloud-init, https://opendev.org/opendev/glean creating per vms image or install media si an anti pattern. you can use tools like deboostrap or sysprep to prepare they images ot do customisation on first boot but that shoudl be geneeric enouch that it is reusable across many vms. https://docs.openstack.org/image-guide/ is our offial docuemation on how to build resuabel images for opentack. https://docs.openstack.org/image-guide/openstack-images.html descibe the standed expecations for cloud images. here is an example with ubuntu https://docs.openstack.org/image-guide/create-images-manually-example-ubuntu... but here are some automated tools as well https://docs.openstack.org/image-guide/create-images-automatically.html for example https://github.com/cloudbase/windows-imaging-tools was mained by cloudbase solution for windows images in the past. but more generic tools like https://developer.hashicorp.com/packer or even openstack disk image builder are probably a better choice.
3. Specific Questions and Areas for Community Input:
My primary challenges and questions revolve around how OpenStack can facilitate this process, particularly regarding:
3.1 Guest-to-Hypervisor Communication: What are the recommended OpenStack-native tools or mechanisms to enable the guest OS (booting from our custom ISO) to notify the hypervisor of its details? I am unsure if qemu-guest-agent is suitable for this specific boot-from-ISO scenario, or if other methods (e.g., metadata service interactions) are more appropriate.
that would generally be considered a security risk. there shoudl be no gust to host communication channel but the way this si normally done is via a qemu guest agent. the openstack community tool for building guest images is https://github.com/openstack/diskimage-builder and rackspace have an example agent that is based on how they did this https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder... https://github.com/Rackspace-DOT/nova-agent
3.2 Initial Guest Configuration: How can user data or initial configuration information (like the network settings we're embedding in the ISO) be effectively provided to a VM during this templateless provisioning flow in OpenStack? This includes handling complex networking scenarios with both internal and external DHCP servers. Is cloud-init (or its Windows equivalent) the recommended approach, and if so, how can it be leveraged when booting from a custom ISO?
the cloud image shoudl have a first boot init package like cloud-init or glen preinstalled opentack (nova) has a metadata component that will provide networking information automatically to the guest and you can use our config drive support to have nova automatically put that into an iso that is attached to the guest. https://docs.openstack.org/nova/latest/user/metadata.html if you as an operator want to providee addtional info beyond the standard set you have 2 options using the vendor data feature https://docs.openstack.org/nova/latest/user/metadata.html#vendordata ther eare two types of vender data https://docs.openstack.org/nova/latest/admin/vendordata.html#staticjson and https://docs.openstack.org/nova/latest/admin/vendordata.html#dynamicjson static vendor data is for somethin that is common to all vms the dynamic vendor data woudl allow you to run a simple webserver/rest api to provide per vm data.
3.3 Programmatic Network Utilization: My current OpenStack environment has limited default networking. I am keen to understand how to programmatically utilize and attach custom-created networks (beyond the default setup) via OpenStack API calls during the instance provisioning process.
this is something that the end user who is creating the vm should do. they can pass multiple neutron port or networks to nova when creating the vm and they will be attached to the vm when its created.
I am looking for insights on programmatically implementing this "boot from custom ISO" provisioning method in OpenStack, rather than relying on pre-existing Glance images.
you are going to have to build your own tooling for this. many people use terreform or ansible to do this type of customiastion by starting form a pre-existing generic glance image. if you dont want to do that and you not going to use boot form volume then you cant do this today. the closest thing you can do is either via an agent in the iso or some other method backed into a glance image. but its not a workflow that is natively supported in openstack. its defintly possibel to do as talos have done this https://www.talos.dev/v1.7/talos-guides/install/cloud-platforms/openstack/ but they did it in a cloud native way where the have a standard image that you boot into and then there install agent loads into ram and overwrites the root disk. core-os uses ignition script to do somethign very similar. https://docs.fedoraproject.org/en-US/fedora-coreos/provisioning-openstack/
Any valuable inputs, sample code implementation or examples, or pointers to relevant documentation would be highly appreciated.
Thank you all for your time and assistance!
Best regards, Prem
Hello Community Folks, Thank you all so much for the incredibly clear and detailed guidance regarding "templateless" server provisioning in OpenStack. Your insights have been invaluable in clarifying OpenStack's core design philosophy. I now clearly understand that OpenStack, as a cloud platform, optimizes for rapid and scalable deployment from pre-existing Glance images, rather than supporting on-the-fly interactive OS installations from custom ISOs (as might be possible in traditional virtualization platforms like OLVM/vCenter). I also grasp that attempting to force an ISO-boot-and-install workflow in OpenStack is considered an anti-pattern due to its complexity and lack of native support. My focus will now entirely shift to leveraging this intended OpenStack workflow: Ensuring we have a robust, generalized Windows Server image in Glance that is pre-installed with Cloudbase-Init and VirtIO drivers. Utilizing the user_data mechanism extensively via the OpenStack SDK (or CLI) to perform all necessary post-boot customizations, including network configuration, firewall rules, and application installations. I appreciate you steering me towards the correct and efficient cloud-native approach. Best regards, Prem
participants (5)
-
Barrera, Eric
-
Jeremy Stanley
-
Julia Kreger
-
prmjh4＠gmail.com
-
Sean Mooney