Hi, we do have some extra-dhcp-options set for the neutron port, it all works great when we issue PXE boot. It's just that we would like to avoid manual intervention like exiting from the EFI shell and select PXE boot manually. What I realised just now is, the EFI shell isn't in the PXE image we attach, but apparently it comes from libvirt (?). So when I add the property hw_firmware_type=uefi to an image (any image), the resulting instance boots into the EFI shell. I didn't expect that, tbh. So is there any way to avoid this EFI shell? Zitat von Sean Mooney <smooney@redhat.com>:
On 21/10/2025 18:22, Alvin Starr wrote:
Potentially stupid idea.
How about using some variant of the netboot disk on a very small boot disk partition(sda) and then install the OS on a larger partition alocated at VM creation time.
It does cause you to need 2 disks for every system and I am sure there are a few quirks I have overlooked.
that would work for a normal system that will boot form the root disk as you could update grub to then prefer the newly installed os.
for Eugen's case of always booting form pxe on every boot there is a better way
What about using a very small On 2025-10-21 09:45, Eugen Block wrote:
Thanks, Sean, I was kind of hoping you would chime in. ;-)
I experimented for a couple of hours now, trying to make it work as we require it (always boot VMs from network via PXE, the management server decides if it can boot from local disk or if an installation has to be initiated). And UEFI is required as well (we set hw_firmware_type=uefi for the pxe image).
One issue seems to be that if we use an uefi image, the VM boots into the EFI shell. So it's interactive which we don't want. I haven't found a way to avoid the shell and directly boot from network.
ya so i believe this works today if you boot form an ipxe iso like https://boot.ipxe.org/ipxe.iso
basically you woudl either use a flavor with additional ephemeral disk to store your ephemeral data or use a cinder volume for data storage. you can boot form image or volume using the ipxe images as the boot media.
then you can use the exact same setting on the neutron port that ironic uses to specify your pxe server details
https://docs.openstack.org/api-ref/network/v2/index.html#extra-dhcp-option-e...
basically if you have an assailable tftp server on the neutron tenant network
you can specify PXE boot options to DHCP clients such as tftp-server, bootfile-name ectra requried to pxe boot.
the boot order is boot form local root disk which is actually ipxe which then chainloads the actual boot media form the tftp server sepcifed in the dhcp options which can basically be anything you want.
ipxe used to have a bug that ment it could not boot form a different subnet then the dhcp respocne, that is why i said the tftp server should be on the tenet network
ill admit the only time i have pxe booted a nova vm was when it was attached to a flat external network where my home router was handling dhcp and coudl set all these extra dhcp options but i belive if you use the extensions for ironic the neutron dhcp server shoudl be able to do this correctly although if your using ovn there might be some limitations.
but in principal if you vm is using an ipx image you shoudl be able to effectively pretend its ironic and use the support for pxe booting that is in neutron for ironic.
I can't tell if the mentioned patch would avoid that, is a non-interactive pxe boot feasible? Or do we need (maybe build ourselves) a custom pxe image or something like that?
the referenced patch
https://opendev.org/openstack/openstack-virtual-baremetal/src/branch/stable/...
is just setting the boot order to boot form the network first before the root disk.
libvirt has native support to specify the boot order the same way as you can in the bios of a normal physical server
you can either do it per device
https://libvirt.org/formatdomain.html#specifying-boot-order
or you can do what the patch does and set it in the boot loader element based on the type of device
https://libvirt.org/formatdomain.html#bios-bootloader
```
boot
The dev attribute takes one of the values "fd", "hd", "cdrom" or "network" and is used to specify the next boot device to consider. The boot element can be repeated multiple times to setup a priority list of boot devices to try in turn. Multiple devices of the same type are sorted according to their targets while preserving the order of buses. After defining the domain, its XML configuration returned by libvirt (through virDomainGetXMLDesc) lists devices in the sorted order. Once sorted, the first device is marked as bootable. Thus, e.g., a domain configured to boot from "hd" with vdb, hda, vda, and hdc disks assigned to it will boot from vda (the sorted list is vda, vdb, hda, hdc). Similar domain with hdc, vda, vdb, and hda disks will boot from hda (sorted disks are: hda, hdc, vda, vdb). It can be tricky to configure in the desired way, which is why per-device boot elements (see Hard drives, floppy disks, CDROMs <https://libvirt.org/formatdomain.html#hard-drives-floppy-disks-cdroms>, Network interfaces <https://libvirt.org/formatdomain.html#network-interfaces>, and Host device assignment <https://libvirt.org/formatdomain.html#host-device-assignment> sections below) were introduced and they are the preferred way providing full control over booting order. The boot element and per-device boot elements are mutually exclusive. Since 0.1.3, per-device boot since 0.8.8
```
the patch does it via instance metadata which is a bit of a hack but its also one of the only way to set something per
if we were to support this properly we would likely want to do it as an extension on the port (i.e. annotate the port with a flag to say boot form me) so that the boot request is associated with the netwrok interface that also has the dhcp options set or we could expose a new boot parameter i guess directly or on the flavor.
the only way to set the boot order today in the nova api is in the block device mappings but that obviously does not allow you to set a port as the first device.
that why i suggested booting form an ipxe image above as https://ipxe.org/ images default to booting form the network.
i believe the https://netboot.xyz/downloads can also be used for interactive installs but it sound like you dont want that. you want to externally manage what the server is booting over the network and have the vms just pxe boot that correct?
nova does not have a one click way to do this today but i think its possible to do today.
The main objective of this question is automation, it's how we manage our physical server infrastructure as well. And it would be incredibly helpful to be able to rebuild parts of it in our virtual infrastructure.
Thanks! Eugen
Zitat von Sean Mooney <smooney@redhat.com>:
On 17/10/2025 15:30, Eugen Block wrote:
Why didn't we think of this?! That's a great idea! I'm gonna try that as soon as possible.
Thanks a lot!
Zitat von Harald Jensas <hjensas@redhat.com>:
Hi,
openstack-virtual baremetal carried patches[1] to enable PXE booting instances, I think the nova project declined including them back when, I do not know why ... not as far as i am awre
we have enve dicsused this a few time over the last few cycle and said fi soem actully wanto to propose a spec to upstream addtional support for pxe boot we woudl review it.
i am not aware of anyone trying to upstream those patches before.
currently there are 3 ways to pxe boot in nova
1 you can enable the qemu boot menu. if you have uefi boot enabel the ovmf uefi image supprot pxe booting the other way is to use an iso or cinder volume as you suggest below
the third ay is vis rescue
we have docs for that here https://docs.openstack.org/nova/latest/user/boot-instance-using-PXE.html
we have breiflly dcised the idea of also having a way to annotat a neuton port as a boot device to stream line the
pxe boot process.
the reason this has not been done is just that no one express interest in actually doing it.
in any case its possibel to pxe boot vms today in nova
1 interactivly via the pxe supprot in the ovmf uefi firmware
2 via using a ipxe image as an addtional disk or the root disk
3 via rescue
patches shoudl not be required but it woudl be nice if we coudl suport this more trivially if folks were actully
interested in that capablity.
Another potential solution is to use block_device_mappings and attach an iPXE image as a secondary disk and set that as boot_index: 0. I use the following when creating an instance, the "server_img.id" can be a "blank" image initially. Then the second image is an iPXE image. The result is that the instance boots into iPXE which does network boot ...
block_device_mapping = [ {'uuid': server_img.id, 'boot_index': 1, 'source_type': 'image', 'destination_type': 'volume', 'device_type': 'disk', 'volume_size': VOLUME_SIZE, 'delete_on_termination': True, }, {'uuid': ipxe_image.id, 'boot_index': 0, 'source_type': 'image', 'destination_type': 'volume', 'device_type': 'cdrom', 'disk_bus': 'sata', 'volume_size': VOLUME_SIZE, 'delete_on_termination': True, } ]
[1] https://opendev.org/openstack/openstack-virtual-baremetal/src/branch/stable/... // Harald