TrenchBoot Anti Evil Maid for Qubes OS

[Michał Żygowski <michal.zygowski@xxxxxxxxx>]

Dear TrenchBoot Community,

As you may know, 3mdeb has been doing a Proof of Concept (PoC) 
integration of TrenchBoot into Qubes OS to provide Anti Evil Maid (AEM) 
service in place of Trusted Boot (tboot), the former solution available 
for Intel TXT launch. Our (3mdeb's) initial plan and motivation has been 
explained on Dasharo documentation page[1].
We (3mdeb) would like to continue the work to extend the TrenchBoot 
support in Qubes OS further. It will include AMD and Intel platforms 
with TPM 2.0 and both legacy and UEFI boot mode. A lot of time has 
passed since the initial plan has been devised and the upstream work of 
implementing Secure Launch support for Linux kernel[2] has progressed as 
well. Because of that the initial plan for TrenchBoot AEM support for 
Qubes OS has to be changed. Below I have briefly explained what is 
needed to achieve the goal using the most recent protocol version 
designed for Linux kernel.
Project Plan
------------

The PoC phase for Intel platforms in legacy boot mode is approaching its 
finish line and results shall be published soon. Most of the work 
already done will still be applicable in the new TrenchBoot boot 
protocol for both UEFI and legacy BIOS when it comes to Xen hypervisor 
support for DRTM post-launch. The changes introduced in the upstream 
patches for TrenchBoot are mainly affecting the DRTM pre-launch phase 
which will have to be rebased to most recent version.
The plan is divided into multiple phases, each achieving a different
functionality of the Qubes OS Anti Evil Maid with TrenchBoot:

1. TPM 2.0 support in Qubes OS AEM  (Intel hardware):

   - Implement support for TPM 2.0 module in Xen
   - Implement support for TPM 2.0 event log in Xen

     The two above are needed to handle the measurements and event log
     created and returned to kernel (Xen) by specific DRTM technology
     implemented in the silicon. Xen also needs to measured the Dom0
     kernel and initrd prior to Dom0 construction to ensure the
     principle load-measure-execute.

   - Implement parallel CPU cores bring-up for DRTM launch

     Currently the CPU cores are being woken up in parallel, but later
     they are hacked to be waiting in a queue. If any interrupt would
     come at that time, it could be a serious danger. It has to be
     fixed as soon as possible, as required by Intel TXT specification.

   - Integrate TPM 2.0 software stack into Qubes OS Dom0
   - Extend the AEM scripts to detect TPM version on the platform
   - Extend the AEM scripts to use appropriate software stack for TPM
     2.0

     Currently, only TPM 1.2 is supported in Qubes OS AEM service code.
     The 3 items above will ensure the necessary software for TPM 2.0
     is available and AEM scripts executed early from the initrd can
     detect which TPM family is present on the platform and use
     appropriate software stack and functions. TPM 1.2 and TPM 2.0
     software stacks are not compatible so the scripts themselves must
     use proper API for given TPM and its respective software stack.

   - Update Qubes OS AEM documentation

     This phase is merely an expansion of the initial PoC to support
     TPM 2.0 in Qubes OS AEM with TrenchBoot. Next phases will focus on
     enabling UEFI boot mode support and aligning with the most recent
     TrenchBoot Secure Launch protocol.

   - Test the solution on Intel hardware with TPM 1.2 and 2.0 using
     legacy boot mode

2. Update to the newest TrenchBoot boot protocol

   - Code rebase onto the most recent work implementing Secure Launch
     protocol being upstreamed to Linux and GRUB

     Modifications introduced in GRUB and Xen for the AEM project will
     have to be aligned with the TrenchBoot code being upstreamed to
     GRUB and Linux kernel. The main idea is to have a generic Secure
     Launch module being exposed by firmware/bootloader to target
     operating system kernel which can be called by the kernel to
     perform Secure Launch using DRTM. The advantage of such approach
     is lesser maintenance burden in multiple projects providing
     kernels for operating systems (such as Xen and Linux kernel),
     standardized boot protocol and centralized code in one place.
     Unfortunately, there is no documentation explaining the details of
     the most recent protocol neither in TrenchBoot documentation nor
     was it announced on TrenchBoot mailing list. Given that, our
     proposal may not be exactly correct because of assumptions made by
     3mdeb about the details of the most recent boot protocol.

   - Test the solution on Intel hardware with TPM 1.2 and TPM 2.0 using
     legacy boot mode

3. AMD support for Qubes OS AEM with TrenchBoot

   - Clean up the Secure Kernel Loader (formerly LandingZone) package
     support for Qubes OS

     An initial integration of the Secure Kernel Loader (SKL, formerly
     LandingZone) for Qubes OS[3] has been made by 3mdeb as a Proof of
     Concept for running DRTM in Qubes OS on AMD platforms[4]. This
     code, however, is very outdated, to the extent where the project
     name happened to change in the mean time. A lot of changes and
     improvements have been made to SKL and Qubes OS should use the
     most recent version. The work here will include a clean up of the
     existing Qubes OS package integration and update to the most
     recent version of SKL.

   - TrenchBoot Secure Kernel Loader (SKL) improvements for AMD server
     CPUs with multiple nodes

     As SKL was mostly validated to work on consumer devices, it does
     not take into account AMD CPUs with multiple nodes. Each node can
     be treated as a separate processor with a full set of CPU
     registers. This implies additional action to be made by SKL, that
     is handling the DMA protection for SKL on each node. Currently the
     DMA protection is lifted when exiting SKL only on a single node
     system. The work includes extending the SKL code to handle multi
     node systems like servers.

   - Clean up the AMD DRTM (SKINIT) support in TrenchBoot GRUB2

     The TrenchBoot code in GRUB supporting AMD is very old as well and
     is definitely not aligned with current work. Moreover it needs a
     cleanup of obsolete code.

   - Test the solution on AMD and Intel hardware with TPM 2.0 and TPM
     1.2 using legacy boot mode

4. Xen UEFI boot mode with DRTM sign TrenchBoot:

   - TrenchBoot support for UEFI boot mode for AMD in GRUB

     During the PoC work on booting Qubes OS with DRTM using
     TrenchBoot, UEFI boot mode did not work. UEFI also has not been
     tested extensively so a misimplementation in some places is
     possible. It has to be analyzed and fixed.

   - TrenchBoot support for UEFI boot mode in Xen for Intel
   - TrenchBoot support for UEFI boot mode in Xen for AMD

     The scope of the work for the two above items will include
     implementing support for the most recent TrenchBoot boot protocol
     in the Xen hypervisor. Xen must be able to find the Secure Launch
     module, feed it with necessary data and call it to perform DRTM
     launch.

   - Test the solution on AMD and Intel hardware with TPM 2.0 and TPM
     1.2 using legacy and UEFI boot mode

Future improvements
-------------------

Initially there was S3 support in the scope of the work, however after
consulting other TrenchBoot Developers (thank you Daniel P. Smith and
Andrew Cooper) and a longer consideration we (3mdeb) came to the
following conclusions:

1. Performing DRTM launch on S3 resume seems to be out of scope for
   current AEM implementation (AEM does not play any role in Qubes OS
   resume process). AEM takes advantage of DRTM only during normal boot.
2. DRTM launch of Xen hypervisor on S3 resume is a complex topic which
   needs thorough debate on how to approach that and how it should be
   implemented. it may be even eligible for a separate project.
3. DRTM on S3 resume will not be able to do the same measurements as on
   normal boot. The Xen and Dom0 images in memory will be different than
   on normal boot. What one would get from DRTM launch during S3 resume
   is a footprint of current state of the hypervisor. It has to be taken
   in appropriate place and time to make the measurements as consistent
   and meaningful as possible. Moreover, measuring the footprint of
   runtime components is more suitable for attestation purposes than
   Anti Evil Maid use case.

Despite of the above conclusions I highly encourage to debate on the 
following:
- if and how Qubes OS AEM could use the S3 resume DRTM measurements of
  Xen (and Dom0)
- what parts of Xen hypervisor (and possibly Dom0) should be measured
  during S3 resume with DRTM
- when the measurement of Xen hypervisor should be made during S3 resume
  with DRTM
- if there are any outstanding issues or blockers to implement DRTM
  launch of Xen hypervisor during S3 resume

One idea to perform a DRTM during S3 resume would be as follows:

Cold boot: GRUB -> DRTM -> Xen (init up to Dom0 creation) -> DRTM -> Xen 
-> Dom0
S3 resume: Xen -> DRTM -> Xen (!) -> Dom0 resume

(!) - here we have to teach Xen how to return to previous state without
restarting Dom0

Such approach would result in identical measurements after coldboot and 
S3 resume of Xen hypervisor.
Summary
-------

Please keep in mind that 3mdeb has only a vague understanding of the
newest TrenchBoot boot protocol and because of that the details
explained in this message may not be accurate. It would be appreciated
to have a draft of the specification or documentation explaining the
details of the current approach of performing Secure Launch using
TrenchBoot.

We kindly ask for feedback and comments from the community with
constructive critique of our plan.

Best regards,
3mdeb team

[1] https://docs.dasharo.com/projects/trenchboot-aem/
[2] https://lkml.org/lkml/2022/2/18/699
[3] https://github.com/3mdeb/qubes-landing-zone/tree/lz_support
[4] https://youtu.be/rM0vRi6qABE?t=1461

Attachment: OpenPGP_0x6B5BA214D21FCEB2.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature
Description: OpenPGP digital signature