Re: SecureBoot and PCI passthrough with kernel lockdown in place (on Xen)

[Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>]

On 14/02/2022 15:02, Dario Faggioli wrote:
> Hello,
>
> We have run into an issue when trying to use PCI passthrough for a Xen
> VM running on an host where dom0 kernel is 5.14.21 (but we think it
> could be any kernel > 5.4) and SecureBoot is enabled.

Back up a bit...

Xen doesn't support SecureBoot and there's a massive pile of work to
make it function, let alone work in a way that MSFT aren't liable to
revoke your cert on 0 notice.

>
> The error we get, when (for instance) trying to attach a device to an
> (HVM) VM, on such system is:
>
> # xl pci-attach 2-fv-sles15sp4beta2 0000:58:03.0 
> libxl: error: libxl_qmp.c:1838:qmp_ev_parse_error_messages: Domain 12:Failed 
> to initialize 12/15, type = 0x1, rc: -1
> libxl: error: libxl_pci.c:1777:device_pci_add_done: Domain 
> 12:libxl__device_pci_add failed for PCI device 0:58:3.0 (rc -28)
> libxl: error: libxl_device.c:1420:device_addrm_aocomplete: unable to add 
> device
>
> QEMU, is telling us the following:
>
> [00:04.0] xen_pt_msix_init: Error: Can't open /dev/mem: Operation not 
> permitted
> [00:04.0] xen_pt_msix_size_init: Error: Internal error: Invalid 
> xen_pt_msix_init.
>
> And the kernel reports this:
>
> Jan 27 16:20:53 narvi-sr860v2-bps-sles15sp4b2 kernel: Lockdown: 
> qemu-system-i38: /dev/mem,kmem,port is restricted; see man kernel_lockdown.7
>
> So, it's related to lockdown. Which AFAIUI it's consistent with the
> fact that the problem only shows up when SecureBoot is enabled, as
> that's implies lockdown. It's also consistent with the fact that we
> don't seem to have any problems doing the same with a 5.3.x dom0
> kernel... As there's no lockdown there!
>
> Some digging revealed that QEMU tries to open /dev/mem in
> xen_pt_msix_init():
>
>     fd = open("/dev/mem", O_RDWR);
>     ...
>     msix->phys_iomem_base =
>             mmap(NULL,
>                  total_entries * PCI_MSIX_ENTRY_SIZE + 
> msix->table_offset_adjust,
>                  PROT_READ,
>                  MAP_SHARED | MAP_LOCKED,
>                  fd,
>                  msix->table_base + table_off - msix->table_offset_adjust);
>     close(fd);

Yes.  Use of /dev/mem is not permitted in lockdown mode.  This wants
reworking into something which is lockdown compatible.

The real elephant in the room is that privcmd is not remotely safe to
use in a SecureBoot environment, because it lets any root userspace
trivially escalate privilege into the dom0 kernel, bypassing the
specific protection that SecureBoot is trying to achieve.

~Andrew