[Xen-devel] [PATCH OSSTEST v3] Arrange to upgrade microcode on x86 test hosts.

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

Both Xen and Linux support extracting a microcode update from an
initramfs early during boot. This requires prepending a suitable
uncompressed cpio archive containing the necessary files to the
initrd.

Xen also supports loading the microcode cpio from any multiboot
module, but for in order to allow the possibility of loading on native
boots (e.g. for build jobs) we prefer the prepend method.

This patch provides mg-cpu-microcode-update which creates a suitable
microcode cpio in the images directory and arranges for it to be added
to the host during installation (so it is done before the kernel is
installed and initramfs generated etc, saving faff). It also adds
"ucode=scan" to the Xen command line when necessary.

The version of initramfs-tools in Wheezy does not yet support
prepending things to the initrm, so we use a custom compression
command which sneakily does it for us. This could be done better from
Jessie onwards.

Note that Linux only supports this from v3.8 onwards, so this doesn't
work for Wheezy (which uses v3.2). From Jessie onwards we should
benefit from microcode updates even for native (build) jobs.

Tested on both Intel and AMD where it appeared to have the desired
effect under Xen. Under native I only tried Wheezy which doesn't
support early microcode.

Note that I've not bothered to implement late microcode updates, which
would work on v3.2 too, since that would need a different set of files
etc and it doesn't seem especially pressing.

Signed-off-by: Ian Campbell <ian.campbell@xxxxxxxxxx>
Cc: Jan Beulich <JBeulich@xxxxxxxx>
---
v3:
- iucode-tool is iucode_tool in Wheezy, Jessie has both names
- This exact version mg-cpu-microcode-update has been run in colo,
  updated production-config*
- Avoid proxy with wget, since it was pointing at the apt-cache which
  gave "403 Sorry, not allowed to fetch that type of file:
  fiano0_microcode-cpio"

v2:
- Downcase local variables, avoid $CPIO/$cpio completely.
- Add /usr/sbin to $PATH.
- Make mg-cpu-microcode-update idemotent and deterministic.
- Operate in locally constructed tmp dir.
- Clone linux-firmware from git instead of curl on gitweb.
- More idiomatic Perl
- Move log message to where everything is defined.
---
 Osstest/Debian.pm           | 44 ++++++++++++++++++++++++
 mg-cpu-microcode-update     | 82 +++++++++++++++++++++++++++++++++++++++++++++
 production-config           |  4 +++
 production-config-cambridge |  4 +++
 ts-xen-install              |  7 ++++
 5 files changed, 141 insertions(+)
 create mode 100755 mg-cpu-microcode-update

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index f09cad6..c6ca159 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -626,6 +626,48 @@ chroot /target chown -R \$u.\$u \$h/.ssh
 END
 }
 
+sub preseed_microcode($$)
+{
+    my ($ho,$sfx) = @_;
+    my $prop = "MicrocodeUpdate".ucfirst($r{arch});
+    return unless $c{$prop};
+    logm("ucode=$prop $c{$prop}");
+    my $ucode = get_filecontents("$c{Images}/$c{$prop}");
+    my $cpio_url = create_webfile($ho, "microcode-cpio",
+       sub {
+           my $f = "$c{Images}/$c{$prop}";
+           copy($f, $_[0]) or die "Copy $f failed: $!";
+    });
+
+    # The ability to prepend from an initramfs-hook was not added
+    # until Jessie, therefore for Wheezy we use a custom compression
+    # method which sneaks the necessary cpio onto the front.
+    my $gzip_url = create_webfile($ho, "microcode-gzip",<<END);
+#!/bin/bash
+if [ -f /boot/microcode.cpio ]; then
+    cat /boot/microcode.cpio
+fi
+exec gzip
+END
+
+    preseed_hook_installscript($ho, $sfx,
+       '/usr/lib/base-installer.d/','osstest-microcode', <<END);
+#!/bin/sh
+set -ex
+
+mkdir -p /target/boot
+wget -Y off -O /target/boot/microcode.cpio $cpio_url
+
+mkdir -p /target/usr/sbin
+wget -Y off -O /target/usr/sbin/osstest-initramfs-gzip $gzip_url
+chmod +x /target/usr/sbin/osstest-initramfs-gzip
+
+mkdir -p /target/etc/initramfs-tools/conf.d/
+echo COMPRESS=/usr/sbin/osstest-initramfs-gzip >> \\
+       /target/etc/initramfs-tools/conf.d/osstest-initramfs-gzip.conf
+END
+}
+
 sub preseed_base ($$$$;@) {
     my ($ho,$suite,$sfx,$extra_packages,%xopts) = @_;
 
@@ -941,6 +983,8 @@ d-i partman-auto/expert_recipe string                       
                \\
 
 END
 
+    preseed_microcode($ho,$sfx);
+
     $preseed_file .= preseed_hook_cmds();
 
     if ($ho->{Flags}{'no-di-kernel'}) {
diff --git a/mg-cpu-microcode-update b/mg-cpu-microcode-update
new file mode 100755
index 0000000..759f676
--- /dev/null
+++ b/mg-cpu-microcode-update
@@ -0,0 +1,82 @@
+#!/bin/bash
+
+set -e
+
+. cri-getconfig
+
+# iucode_tool is in /usr/sbin, see #788459.
+export PATH="/usr/local/sbin:$PATH:/sbin:/usr/sbin"
+
+images=`getconfig Images`
+gitproxy=`getconfig GitCacheProxy`
+date=`date +%Y-%m-%d`
+
+cpiodir=cpio.x86
+ucodepath=kernel/x86/microcode
+
+intelbin=GenuineIntel.bin
+amdbin=AuthenticAMD.bin
+
+ucodecpio=$images/microcode.x86.$date.cpio
+
+linuxfw=git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
+
+rm -rf $ucodecpio.tmp
+mkdir $ucodecpio.tmp
+cd $ucodecpio.tmp
+
+mkdir -p $cpiodir/$ucodepath/
+
+# Intel
+#
+# From http://feeds.downloadcenter.intel.com/rss/?p=483&lang=eng look for
+# Linux* Processor Microcode Data File which will take you to e.g.
+# 
https://downloadcenter.intel.com/downloads/eula/24661/Linux-Processor-Microcode-Data-File?httpDown=http%3A%2F%2Fdownloadmirror.intel.com%2F24661%2Feng%2Fmicrocode-20150121.tgz'
 which redirects to the following:
+INTEL_TGZ='http://downloadmirror.intel.com/24661/eng/microcode-20150121.tgz'
+
+# The microcode-YYYYMMDD.tgz contains a microcode.dat which we must
+# then convert to the appropriate binary format using iucode_tool
+# (available in Debian).
+
+mkdir intel-ucode
+
+echo >&2 "Fetching Intel ucode"
+curl -s $INTEL_TGZ > intel-ucode/microcode.tgz
+
+tar -C intel-ucode -xaf intel-ucode/microcode.tgz microcode.dat
+
+echo >&2 "Converting Intel ucode"
+iucode_tool -t d -q \
+    --write-to=$cpiodir/$ucodepath/$intelbin \
+    intel-ucode/microcode.dat
+
+# AMD
+#
+# http://xenbits.xen.org/docs/unstable/misc/amd-ucode-container.txt
+#
+# From linux-firmware.git:
+AMD_BINS='
+  amd-ucode/microcode_amd.bin
+  amd-ucode/microcode_amd_fam15h.bin
+  amd-ucode/microcode_amd_fam16h.bin
+'
+
+echo >&2 "Cloning $gitproxy$linuxfw"
+git clone --quiet --depth=1 $gitproxy$linuxfw linux-firmware
+
+# Concatenate into $ucodepath/$amdbin within the
+# cpio.
+echo >&2 "Conveting AMD ucode"
+( cd linux-firmware && cat $AMD_BINS ) > $cpiodir/$ucodepath/$amdbin
+
+# Ensure the cpio is reproducible, we don't care about timestamps
+# inside the archive.
+find $cpiodir -exec touch -d @0 {} \;
+
+echo >&2 "Building cpio archive"
+( cd $cpiodir && find . | cpio -o -H newc --quiet > $ucodecpio )
+
+echo "New x86 microcode: $ucodecpio"
+
+cd $images # Get out of $ucodecpio.tmp
+rm -rf $ucodecpio.tmp
diff --git a/production-config b/production-config
index 3a0b768..47c0c4c 100644
--- a/production-config
+++ b/production-config
@@ -87,6 +87,10 @@ TftpPxeTemplatesReal pxelinux.cfg/%ipaddrhex%
 TftpPxeGroup osstest
 TftpDiVersion 2015-01-10
 
+# These should normally be the same.
+MicrocodeUpdateAmd64 microcode.x86.2015-06-12.cpio
+MicrocodeUpdateI386 microcode.x86.2015-06-12.cpio
+
 XenUsePath /usr/groups/xencore/systems/bin/xenuse
 XenUseUser osstest
 
diff --git a/production-config-cambridge b/production-config-cambridge
index f32cd84..8f6fbe4 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -73,6 +73,10 @@ TftpPxeTemplates %ipaddrhex%/pxelinux.cfg
 TftpPxeGroup osstest
 TftpDiVersion 2015-01-10
 
+# These should normally be the same.
+MicrocodeUpdateAmd64 microcode.x86.2015-06-12.cpio
+MicrocodeUpdateI386 microcode.x86.2015-06-12.cpio
+
 XenUsePath /usr/groups/xencore/systems/bin/xenuse
 XenUseUser osstest
 
diff --git a/ts-xen-install b/ts-xen-install
index b2f667f..d55401c 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -145,6 +145,13 @@ sub setupboot () {
     if (toolstack($ho)->{Dom0MemFixed}) {
         $xenhopt .= " dom0_mem=512M,max:512M";
     }
+
+    # If /boot/microcode.cpio is present then ts-host-install has
+    # arranged that it will be prepended to the initrd already, all we
+    # need to do is look for it.
+    $xenhopt .= " ucode=scan"
+       if target_file_exists($ho, "/boot/microcode.cpio");
+
     my $append= $r{xen_boot_append};
     $xenhopt .= " $append" if defined $append;
     $append = get_host_property($ho, 'xen-commandline-append', undef);
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel