Open Stack

On Fri, 2021-12-03 at 15:54 -0700, Alex Schultz wrote:
> On Fri, Dec 3, 2021 at 3:45 PM Thomas Goirand <zigo at debian.org> wrote:
> > 
> > On 12/3/21 11:34 PM, Alex Schultz wrote:
> > > On Fri, Dec 3, 2021 at 3:22 PM Thomas Goirand <zigo at debian.org> wrote:
> > > > 
> > > > On 12/3/21 10:41 PM, Julia Kreger wrote:
> > > > > Extreme magic - OpenStack Edition. At least that feels like the answer.
> > > > > 
> > > > > In essence, we create VMs. We attach them to the network, then
> > > > > configure virtualbmc or sushy-tools to play the role of a BMC and one
> > > > > of the things that library understand is the giant switch of "boot to
> > > > > network". From there we ipxe boot. The VMs play the role of baremetal
> > > > > machines through the emulation of the BMC and they are just hooked up
> > > > > to the network.
> > > > > 
> > > > > This does require the job to use root privileges to setup
> > > > > networking/vms. And the virtualbmc or sushy-tools service needs enough
> > > > > rights to toggle the VM's config. Would be happy to try and discuss
> > > > > specifics of the devstack plugin, but I'm unsure if that would really
> > > > > help. Regardless, you know where to find us. :)
> > > > > 
> > > > > As an aside, when I first read the post I was thinking of booting from
> > > > > ISO images. I've known people to embed ipxe into disk images, and you
> > > > > can embed static config/instructions for booting into an ipxe binary.
> > > > > It does require custom binary builds, but it might make what you're
> > > > > trying to achieve easier.
> > > > 
> > > > Hi Julia!
> > > > 
> > > > Thanks for your answer.
> > > > 
> > > > Well, I've read multiple times your reply, but I have to admit I still
> > > > don't get all the details. For example, how would you "embed ipxe into
> > > > disk images" for example? Should I for example run the iPXE binary from
> > > > something like syslinux (or grub?)? But isn't iPXE usually running
> > > > *after* the BIOS of a PC gets an IP from DHCP?
> > > > 
> > > 
> > > I'm sure there are other examples around but the one we used to (maybe
> > > still do) use for ovb can be built via:
> > > 
> > > https://opendev.org/openstack/openstack-virtual-baremetal/src/branch/master/ipxe
> > > 
> > > You basically build an image that just attempts ipxe on repeat.
> > 
> > Hi Alex!
> > 
> > In fact, I just found out my solution: http://boot.ipxe.org/ipxe.iso
> > 
> 
> Yea I think that's what gets built with make and then it qemu-img
> converts it for you.
> 
> > Just download this, then:
> > 
> > openstack image create \
> >         --disk-format iso \
> >         --container-format bare \
> >         --file ipxe.iso ipxe.iso
> > 
> > Et voila! The VM boots from PXE (I checked on the VNC console, and it
> > also gets DHCP). I didn't go further, but it should be easy to setup the
> > rest of the components (ie: a private network without DHCP, and a VM
> > acting as DHCPd + tftp-hpa).
> > 
> 
> Yea as previously mentioned you may just need to disable port security
> and it should be fine.  This is basically what we (tripleo) do in our
> 3rd party testing where the dhcpd is from the undercloud.  I've also
> used the same image locally in libvirt + sushy-emulator for testing as
> well.
disabling port security might help but in general pxe booting nova instance is not supported.
as in this is intentally not a supproted usecase currently, not that it coudl not be enable but we dont intend it to work outside of
a ironic context. disbaling port security really shoudl only be required if you want to run your own dhcp server, if you willing ot manually
pxe boot interactivly you have ohter options.

we also dont expclitly block pxe booting if the qemu seabios/uefi firmware support it so you can enable the boot menu vai hw:boot_menu=True extra spec or hw_boot_menu=True image property
https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt.json#L24-L29
and select pxe boot if the firmware on the host loaded by qemu support it, but nova makes no guarentee that it will work with the libvirt driver.

ironic is sort of a special case with regards to allowing/using pxe booting when its operating as a nova virt driver.
in ironics case it more that nova does not care how the ironic servcei interally boots the instance and ironci just happens to use ipmi and pxe/ipxe as one of the options.
you do not nesssiasarly need to use an ipxe boot image to pxeboot either as the seabios image, depening on your host useually has pxe supprot built in.
your milage will vary however as this is outside the bounds of what is intended to work so if you encounterd issues adressing those would be new feature blueprint
not bugs so they would not be backportable.

if your intent is to just use this for testing then following in the path of tripleo and ironic by using virtualbmc to provide ipmi access to the vm might be ok but that assuems you are deploying an
openstack cloud which you can then ssh into the hosts to install an run the virtual bmc.

if you really want to do this in an unaltered cloud we would nee to configre the vm to boot form the network
https://libvirt.org/formatdomain.html#bios-bootloader
https://libvirt.org/formatdomain.html#specifying-boot-order

im not sure what the best way to extend nova to support that would be, likely a neutron extention to mark ports a bootable which nova would read and then use to update the xml.
host config is not really an option as it would be problematic for move operations, but flavor extra specs or image properites would be viable alternitive to the neutron approch.
although neutorn ports would give a better ux.
if you dont make progress with other approchs then you should consider fileing a spec for network boot support in nova/neutron.


> 
> > I'll let you guys know how much progress I'm making with all of this
> > (probably I'll start working on this next week...).
> > 
> > Cheers,
> > 
> > Thomas Goirand (zigo)
> > 
> 
>