Re: Secure Boot VM issues (libvirt / SMM) | Secure boot requires SMM feature enabled

29 Jan 2022

      On Wed, 2022-01-19 at 17:31 +0000, Sean Mooney wrote:
...
On Wed, 2022-01-19 at 14:21 +0000, Imran Hussain wrote:
...
Hi,
Deployed Wallaby on Ubuntu 20.04 nodes. Having issues with libvirt XML 
being incorrect, I need the smm bit (<smm state='on'/>) and it isn't 
being added to the XML. Anyone seen this before? Or any ideas? More info 
below...
Error message:
: libvirt.libvirtError: unsupported configuration: Secure boot requires 
SMM feature enabled
Versions:
libvirt version: 6.0.0, package: 0ubuntu8.15
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.18)
Nova 23.1.1 (deployed via kolla, so 
kolla/ubuntu-source-nova-compute:wallaby is the image)
ovmf 0~20191122.bd85bf54-2ubuntu3.3
Context:
https://specs.openstack.org/openstack/nova-specs/specs/wallaby/implemented/a...
Image metadata:
hw_firmware_type: uefi
hw_machine_type: q35
os_secure_boot: required
ok those d seam to be allinged with the documentaiton 
https://docs.openstack.org/nova/latest/admin/secure-boot.html
how in addtion to those option the uefi firmware image used but qemu which is provide by the ovmf package also need
to provide a secure boot capable image
waht failing here is the system manamgemt mode feature.
when os_secure_boot is set
we defien  the "secure" attibute on the loader element.
https://github.com/openstack/nova/blob/7aa3a0f558ddbcac3cb97a7eef58cd878acc3...
based on the  
https://libvirt.org/formatdomain.html#hypervisor-features
smm should be enabled by default
ok so i quickly hacked togheter a patch to test this.
https://review.opendev.org/c/openstack/nova/+/826931
i was able to repoduce the secure boot failure on ubuntu 20.04
this looks like its a ubuntu libvirt bug or upstram libvirts docs are wrong
but with that patch i can boot with secure boot enabeld on ubuntu 20.04
full final xml
<domain type='kvm' id='1'>
  <name>instance-0000000c</name>
  <uuid>fc2284e5-a46d-4ee9-aedc-f6d0058eb797</uuid>
  <metadata>
    <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1%22%3E
      <nova:package version="24.1.0"/>
      nova:nametest</nova:name>
      nova:creationTime2022-01-28 19:01:22</nova:creationTime>
      <nova:flavor name="test.secure-boot">
        nova:memory2048</nova:memory>
        nova:disk20</nova:disk>
        nova:swap0</nova:swap>
        nova:ephemeral0</nova:ephemeral>
        nova:vcpus2</nova:vcpus>
      </nova:flavor>
      nova:owner
        <nova:user uuid="006e2a3094b54b188d898486a0737f68">admin</nova:user>
        <nova:project uuid="25d631b1f0a844e099ca03824d963b5b">demo</nova:project>
      </nova:owner>
      <nova:root type="image" uuid="5a696736-805e-42af-bb8a-9344cee1b5c3"/>
      nova:ports
        <nova:port uuid="1d70c00c-7c8b-4461-9199-3f0657613248">
          <nova:ip type="fixed" address="192.168.233.195" ipVersion="4"/>
        </nova:port>
      </nova:ports>
    </nova:instance>
  </metadata>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <shares>2048</shares>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <sysinfo type='smbios'>
    <system>
      <entry name='manufacturer'>OpenStack Foundation</entry>
      <entry name='product'>OpenStack Nova</entry>
      <entry name='version'>24.1.0</entry>
      <entry name='serial'>fc2284e5-a46d-4ee9-aedc-f6d0058eb797</entry>
      <entry name='uuid'>fc2284e5-a46d-4ee9-aedc-f6d0058eb797</entry>
      <entry name='family'>Virtual Machine</entry>
    </system>
  </sysinfo>
  <os>
    <type arch='x86_64' machine='pc-q35-4.2'>hvm</type>
    <loader readonly='yes' secure='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.ms.fd</loader>
    <nvram template='/usr/share/OVMF/OVMF_VARS.ms.fd'>/var/lib/libvirt/qemu/nvram/instance-0000000c_VARS.fd</nvram>
    <boot dev='cdrom'/>
    <smbios mode='sysinfo'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <smm state='on'/>
    <vmcoreinfo state='on'/>
  </features>
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Nehalem</model>
    <topology sockets='2' cores='1' threads='1'/>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='hypervisor'/>
  </cpu>
  <clock offset='utc'>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/opt/stack/data/nova/instances/fc2284e5-a46d-4ee9-aedc-f6d0058eb797/disk' index='1'/>
      <backingStore type='file' index='2'>
        <format type='raw'/>
        <source file='/opt/stack/data/nova/instances/_base/9d2733539a4f854af18ce27c7c32d49978ace484'/>
        <backingStore/>
      </backingStore>
      <target dev='sda' bus='sata'/>
      <readonly/>
      <alias name='sata0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='none'>
      <alias name='usb'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <interface type='ethernet'>
      <mac address='fa:16:3e:d8:bf:46'/>
      <target dev='tap1d70c00c-7c'/>
      <model type='virtio'/>
      <mtu size='1442'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/5'/>
      <log file='/opt/stack/data/nova/instances/fc2284e5-a46d-4ee9-aedc-f6d0058eb797/console.log' append='off'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/5'>
      <source path='/dev/pts/5'/>
      <log file='/opt/stack/data/nova/instances/fc2284e5-a46d-4ee9-aedc-f6d0058eb797/console.log' append='off'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='virtio' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <stats period='10'/>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/urandom</backend>
      <alias name='rng0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </rng>
  </devices>
  <seclabel type='dynamic' model='apparmor' relabel='yes'>
    <label>libvirt-fc2284e5-a46d-4ee9-aedc-f6d0058eb797</label>
    <imagelabel>libvirt-fc2284e5-a46d-4ee9-aedc-f6d0058eb797</imagelabel>
  </seclabel>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+64055:+108</label>
    <imagelabel>+64055:+108</imagelabel>
  </seclabel>
</domain>
...
smm
Depending on the state attribute (values on, off, default on) enable or disable System Management Mode. Since 2.1.0

Optional sub-element tseg can be used to specify the amount of memory dedicated to SMM's extended TSEG. That offers a fourth option size apart

from the existing ones (1 MiB, 2 MiB and 8 MiB) that the guest OS (or rather loader) can choose from. The size can be specified as a value of that
element, optional attribute unit can be used to specify the unit of the aforementioned value (defaults to 'MiB'). If set to 0 the extended size is not
advertised and only the default ones (see above) are available.
If the VM is booting you should leave this option alone, unless you are very certain you know what you are doing.

This value is configurable due to the fact that the calculation cannot be done right with the guarantee that it will work correctly. In QEMU, the

user-configurable extended TSEG feature was unavailable up to and including pc-q35-2.9. Starting with pc-q35-2.10 the feature is available, with
default size 16 MiB. That should suffice for up to roughly 272 vCPUs, 5 GiB guest RAM in total, no hotplug memory range, and 32 GiB of 64-bit PCI MMIO
aperture. Or for 48 vCPUs, with 1TB of guest RAM, no hotplug DIMM range, and 32GB of 64-bit PCI MMIO aperture. The values may also vary based on the
loader the VM is using.
Additional size might be needed for significantly higher vCPU counts or increased address space (that can be memory, maxMemory, 64-bit PCI MMIO

aperture size; roughly 8 MiB of TSEG per 1 TiB of address space) which can also be rounded up.
Due to the nature of this setting being similar to "how much RAM should the guest have" users are advised to either consult the documentation of

the guest OS or loader (if there is any), or test this by trial-and-error changing the value until the VM boots successfully. Yet another guiding
value for users might be the fact that 48 MiB should be enough for pretty large guests (240 vCPUs and 4TB guest RAM), but it is on purpose not set as
default as 48 MiB of unavailable RAM might be too much for small guests (e.g. with 512 MiB of RAM).
See Memory Allocation for more details about the unit attribute. Since 4.5.0 (QEMU only)

so my guess is you are missing the secure boot capable ovmf image on the host or there is a bug in your libvirt and smm is not being enabled by
default.
...
os_hidden: false
hw_disk_bus: scsi
hw_qemu_guest_agent: yes
hw_scsi_model: virtio-scsi
hw_video_model: virtio
os_require_quiesce: yes
os_secure_boot: required
os_hidden: false
XML snippets taken from nova-compute.log:
   <sysinfo type="smbios">
     <system>
       <entry name="manufacturer">OpenStack Foundation</entry>
       <entry name="product">OpenStack Nova</entry>
       <entry name="version">23.1.1</entry>
       <entry name="serial">2798e3fe-ffae-4c26-955b-ef150b849561</entry>
       <entry name="uuid">2798e3fe-ffae-4c26-955b-ef150b849561</entry>
       <entry name="family">Virtual Machine</entry>
     </system>
   </sysinfo>
   <os>
     <type machine="q35">hvm</type>
     <loader type="pflash" readonly="yes" 
secure="yes">/usr/share/OVMF/OVMF_CODE.ms.fd</loader>
     <nvram template="/usr/share/OVMF/OVMF_VARS.ms.fd"/>
     <boot dev="cdrom"/>
     <smbios mode="sysinfo"/>
   </os>
   <features>
     <acpi/>
     <apic/>
   </features>
Other info:
# cat /usr/share/qemu/firmware/40-edk2-x86_64-secure-enrolled.json
{
     "description": "UEFI firmware for x86_64, with Secure Boot and SMM, 
SB enabled, MS certs enrolled",
     "interface-types": [
         "uefi"
     ],
     "mapping": {
         "device": "flash",
         "executable": {
             "filename": "/usr/share/OVMF/OVMF_CODE.ms.fd",
             "format": "raw"
         },
         "nvram-template": {
             "filename": "/usr/share/OVMF/OVMF_VARS.ms.fd",
             "format": "raw"
         }
     },
     "targets": [
         {
             "architecture": "x86_64",
             "machines": [
                 "pc-q35-*"
             ]
         }
     ],
     "features": [
         "acpi-s3",
         "amd-sev",
         "enrolled-keys",
         "requires-smm",
         "secure-boot",
         "verbose-dynamic"
     ],
     "tags": [
 ]

}