[openstack-dev] Advanced PXE provisioning notes

Vladimir Eremin veremin at mirantis.com
Fri Nov 13 14:13:33 UTC 2015


Hi folks,

As you know, to speed up provisioning stage we could use HTTP for downloading kernel and initramfs. There are 3 options to do this: lpxelinux, iPXE (which is successor/fork of gPXE) and GRUB 2, which we were assessed during my work in Yandex and I’d like to share some experience.

In this note I leave UEFI/iPXE embedding for IPv6 out of scope. Yandex has chosen with iPXE embedding mostly because it’s well-known already and there was less problem to embed iPXE to custom-build hardware than mess with UEFI.

lpxelinux is a HTTP/FTP enabled variant of usual pxelinux (since syslinux 5.10), so it’s required no chainloading (so no mess with DHCP-server) and no boot config re-design. To provide HTTP URI instead of TFTP, two variants may be used:

* replace entries in boot config like LINUX from relative path like boot/mykernel to absolute URL like http://boot-server/boot/mykernel
* provide pxelinux.pathprefix DHCP option [1] contains URL prefix like http://boot-server/

This is most convenient variant to speed up pxelinux setup. Unfortunately, lpxelinux hasn't built for Ubuntu Trusty, so it should be rebuilt from Debian Sid.


iPXE is advanced boot loader with many features like IPv6, HTTP and scripting language. Actually, it allows to pass hardware-related information to provisioning server.

Boot script should be compiled into iPXE, or you will need to set up your DHCP-server [2] to serve different options for different loaders. This option will require to re-write provisioning logic.
It also support UEFI, so it could be used for IPv6 provisioning.

I recommend this variant for advanced IPv6 + HTTP provisioning.


GRUB2 is most advanced option. Unfortunately, it’s still has a bug with IPv6 [3], so there is no point to overcomplicate the setup.


Note that UNDI API is provided correctly by most of modern NIC’s. However, some cards like Mellanox ConnectX and weird Intels is not working correctly. To fix it, iPXE could be built with vendor-specific driver [4]. There is no workaround for lpxelinux.

[1] http://www.syslinux.org/wiki/index.php/PXELINUX#DHCP_options
[2] http://ipxe.org/howto/chainloading
[3] https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1229458
[4] http://ipxe.org/appnote/hardware_drivers

-- 
With best regards,
Vladimir Eremin,
Fuel Deployment Engineer,
Mirantis, Inc.






More information about the OpenStack-dev mailing list