Hi Sean,

Sorry I filed a bug https://bugs.launchpad.net/bugs/1958636 and wrote some of my own patches which are linked in the bug, should have updated this thread!

https://review.opendev.org/c/openstack/nova/+/825496 https://review.opendev.org/c/openstack/nova/+/825729

-Imran

On 2022-01-28 19:11, Sean Mooney wrote:

On Wed, 2022-01-19 at 17:31 +0000, Sean Mooney wrote:

...

On Wed, 2022-01-19 at 14:21 +0000, Imran Hussain wrote:

...

Hi,

Deployed Wallaby on Ubuntu 20.04 nodes. Having issues with libvirt XML being incorrect, I need the smm bit (<smm state='on'/>) and it isn't being added to the XML. Anyone seen this before? Or any ideas? More info below...

Error message: : libvirt.libvirtError: unsupported configuration: Secure boot requires SMM feature enabled

Versions: libvirt version: 6.0.0, package: 0ubuntu8.15 QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.18) Nova 23.1.1 (deployed via kolla, so kolla/ubuntu-source-nova-compute:wallaby is the image) ovmf 0~20191122.bd85bf54-2ubuntu3.3

Context: https://specs.openstack.org/openstack/nova-specs/specs/wallaby/implemented/a...

Image metadata:

hw_firmware_type: uefi hw_machine_type: q35 os_secure_boot: required

ok those d seam to be allinged with the documentaiton https://docs.openstack.org/nova/latest/admin/secure-boot.html how in addtion to those option the uefi firmware image used but qemu which is provide by the ovmf package also need to provide a secure boot capable image

waht failing here is the system manamgemt mode feature.

when os_secure_boot is set we defien the "secure" attibute on the loader element.

https://github.com/openstack/nova/blob/7aa3a0f558ddbcac3cb97a7eef58cd878acc3...

based on the https://libvirt.org/formatdomain.html#hypervisor-features

smm should be enabled by default

ok so i quickly hacked togheter a patch to test this.

https://review.opendev.org/c/openstack/nova/+/826931 i was able to repoduce the secure boot failure on ubuntu 20.04

this looks like its a ubuntu libvirt bug or upstram libvirts docs are wrong but with that patch i can boot with secure boot enabeld on ubuntu 20.04

full final xml

<domain type='kvm' id='1'> <name>instance-0000000c</name> <uuid>fc2284e5-a46d-4ee9-aedc-f6d0058eb797</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1"> <nova:package version="24.1.0"/> <nova:name>test</nova:name> <nova:creationTime>2022-01-28 19:01:22</nova:creationTime> <nova:flavor name="test.secure-boot"> <nova:memory>2048</nova:memory> <nova:disk>20</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>2</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="006e2a3094b54b188d898486a0737f68">admin</nova:user> <nova:project uuid="25d631b1f0a844e099ca03824d963b5b">demo</nova:project> </nova:owner> <nova:root type="image" uuid="5a696736-805e-42af-bb8a-9344cee1b5c3"/> <nova:ports> <nova:port uuid="1d70c00c-7c8b-4461-9199-3f0657613248"> <nova:ip type="fixed" address="192.168.233.195" ipVersion="4"/> </nova:port> </nova:ports> </nova:instance> </metadata> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>2</vcpu> <cputune> <shares>2048</shares> </cputune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>OpenStack Foundation</entry> <entry name='product'>OpenStack Nova</entry> <entry name='version'>24.1.0</entry> <entry name='serial'>fc2284e5-a46d-4ee9-aedc-f6d0058eb797</entry> <entry name='uuid'>fc2284e5-a46d-4ee9-aedc-f6d0058eb797</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-q35-4.2'>hvm</type> <loader readonly='yes' secure='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.ms.fd</loader> <nvram template='/usr/share/OVMF/OVMF_VARS.ms.fd'>/var/lib/libvirt/qemu/nvram/instance-0000000c_VARS.fd</nvram> <boot dev='cdrom'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> <smm state='on'/> <vmcoreinfo state='on'/> </features> <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Nehalem</model> <topology sockets='2' cores='1' threads='1'/> <feature policy='require' name='vme'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='cdrom'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/opt/stack/data/nova/instances/fc2284e5-a46d-4ee9-aedc-f6d0058eb797/disk' index='1'/> <backingStore type='file' index='2'> <format type='raw'/> <source file='/opt/stack/data/nova/instances/_base/9d2733539a4f854af18ce27c7c32d49978ace484'/> <backingStore/> </backingStore> <target dev='sda' bus='sata'/> <readonly/> <alias name='sata0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='usb' index='0' model='none'> <alias name='usb'/> </controller> <controller type='sata' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'> <alias name='pcie.0'/> </controller> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <alias name='pci.1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <alias name='pci.2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <alias name='pci.3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <alias name='pci.4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <interface type='ethernet'> <mac address='fa:16:3e:d8:bf:46'/> <target dev='tap1d70c00c-7c'/> <model type='virtio'/> <mtu size='1442'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/5'/> <log file='/opt/stack/data/nova/instances/fc2284e5-a46d-4ee9-aedc-f6d0058eb797/console.log' append='off'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/5'> <source path='/dev/pts/5'/> <log file='/opt/stack/data/nova/instances/fc2284e5-a46d-4ee9-aedc-f6d0058eb797/console.log' append='off'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <input type='mouse' bus='ps2'> <alias name='input0'/> </input> <input type='keyboard' bus='ps2'> <alias name='input1'/> </input> <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='virtio' heads='1' primary='yes'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <memballoon model='virtio'> <stats period='10'/> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </memballoon> <rng model='virtio'> <backend model='random'>/dev/urandom</backend> <alias name='rng0'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </rng> </devices> <seclabel type='dynamic' model='apparmor' relabel='yes'> <label>libvirt-fc2284e5-a46d-4ee9-aedc-f6d0058eb797</label>

<imagelabel>libvirt-fc2284e5-a46d-4ee9-aedc-f6d0058eb797</imagelabel>

</seclabel> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+64055:+108</label> <imagelabel>+64055:+108</imagelabel> </seclabel> </domain> > > smm > > Depending on the state attribute (values on, off, default on) > enable or disable System Management Mode. Since 2.1.0 > > Optional sub-element tseg can be used to specify the amount of > memory dedicated to SMM's extended TSEG. That offers a fourth option > size apart > from the existing ones (1 MiB, 2 MiB and 8 MiB) that the guest OS (or > rather loader) can choose from. The size can be specified as a value > of that > element, optional attribute unit can be used to specify the unit of > the aforementioned value (defaults to 'MiB'). If set to 0 the extended > size is not > advertised and only the default ones (see above) are available. > > If the VM is booting you should leave this option alone, unless > you are very certain you know what you are doing. > > This value is configurable due to the fact that the calculation > cannot be done right with the guarantee that it will work correctly. > In QEMU, the > user-configurable extended TSEG feature was unavailable up to and > including pc-q35-2.9. Starting with pc-q35-2.10 the feature is > available, with > default size 16 MiB. That should suffice for up to roughly 272 vCPUs, > 5 GiB guest RAM in total, no hotplug memory range, and 32 GiB of > 64-bit PCI MMIO > aperture. Or for 48 vCPUs, with 1TB of guest RAM, no hotplug DIMM > range, and 32GB of 64-bit PCI MMIO aperture. The values may also vary > based on the > loader the VM is using. > > Additional size might be needed for significantly higher vCPU > counts or increased address space (that can be memory, maxMemory, > 64-bit PCI MMIO > aperture size; roughly 8 MiB of TSEG per 1 TiB of address space) which > can also be rounded up. > > Due to the nature of this setting being similar to "how much RAM > should the guest have" users are advised to either consult the > documentation of > the guest OS or loader (if there is any), or test this by > trial-and-error changing the value until the VM boots successfully. > Yet another guiding > value for users might be the fact that 48 MiB should be enough for > pretty large guests (240 vCPUs and 4TB guest RAM), but it is on > purpose not set as > default as 48 MiB of unavailable RAM might be too much for small > guests (e.g. with 512 MiB of RAM). > > See Memory Allocation for more details about the unit attribute. > Since 4.5.0 (QEMU only) > > > so my guess is you are missing the secure boot capable ovmf image on > the host or there is a bug in your libvirt and smm is not being > enabled by > default. > > > os_hidden: false > > > > hw_disk_bus: scsi > > hw_qemu_guest_agent: yes > > hw_scsi_model: virtio-scsi > > hw_video_model: virtio > > os_require_quiesce: yes > > os_secure_boot: required > > os_hidden: false > > > > XML snippets taken from nova-compute.log: > > <sysinfo type="smbios"> > > <system> > > <entry name="manufacturer">OpenStack Foundation</entry> > > <entry name="product">OpenStack Nova</entry> > > <entry name="version">23.1.1</entry> > > <entry name="serial">2798e3fe-ffae-4c26-955b-ef150b849561</entry> > > <entry name="uuid">2798e3fe-ffae-4c26-955b-ef150b849561</entry> > > <entry name="family">Virtual Machine</entry> > > </system> > > </sysinfo> > > <os> > > <type machine="q35">hvm</type> > > <loader type="pflash" readonly="yes" > > secure="yes">/usr/share/OVMF/OVMF_CODE.ms.fd</loader> > > <nvram template="/usr/share/OVMF/OVMF_VARS.ms.fd"/> > > <boot dev="cdrom"/> > > <smbios mode="sysinfo"/> > > </os> > > <features> > > <acpi/> > > <apic/> > > </features> > > > > Other info: > > # cat /usr/share/qemu/firmware/40-edk2-x86_64-secure-enrolled.json > > { > > "description": "UEFI firmware for x86_64, with Secure Boot and SMM, > > SB enabled, MS certs enrolled", > > "interface-types": [ > > "uefi" > > ], > > "mapping": { > > "device": "flash", > > "executable": { > > "filename": "/usr/share/OVMF/OVMF_CODE.ms.fd", > > "format": "raw" > > }, > > "nvram-template": { > > "filename": "/usr/share/OVMF/OVMF_VARS.ms.fd", > > "format": "raw" > > } > > }, > > "targets": [ > > { > > "architecture": "x86_64", > > "machines": [ > > "pc-q35-*" > > ] > > } > > ], > > "features": [ > > "acpi-s3", > > "amd-sev", > > "enrolled-keys", > > "requires-smm", > > "secure-boot", > > "verbose-dynamic" > > ], > > "tags": [ > > > > ] > > } > > > > > > >