[Xen-devel] [PATCH RFC] xen: arm: use uncached foreign mappings when building guests

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

When building an ARM guest we need to take care of cache maintenance
because the guest starts with MMU and cache disabled, which means we
need to make sure that the initial images (kernel, initrd, dtb) which we
write to guest memory are not in the cache.

We thought we had solved this with "tools: libxc: flush data cache after
loading images into guest memory" (a0035ecc0d82) however it turns out
that there are a couple of issues with this approach:

Firstly we need to do a cache flush from userspace, on arm64 this is
possible by directly using the instructions from userspace, but on arm32
this is not possible and so we need to use a system call. Unfortunately
the system call provided by Linux for this purpose does not flush far
enough down the cache hierarchy. Extending the system call would not be
an insurmountable barrier, were it not for the second issue:

Secondly, and more importantly, Catalin Marinas points out (via Marc
Zyngier) that there is a race between the cache flush and the point
where we tear down the mappings, where the processor might speculatively
pull some data into the cache (cache flushes are by Virtual Address, so
this race is unavoidable).

If this happens then guest kernels which modify some code/data before
enabling MMUs + caches may see stale data in the cache.

The solution to this is to use a non-cacheable mapping to populate the
guest RAM, which prevents the processor from pulling anything into the
cache lines. Since we need to write all the way through to RAM anyway
there is not any downside to this.

In order to do this we need a new privcmd ioctl which creates uncached
mappings and a libxc patch to use it. Both of these will follow.

I'm sending this out early to validate that this approach does actually
work and makes sense.

I've tried this on arm64 and it seems ok, Julien, please can you give
this a go in your midway test case which exposes this issue.

The libxc patch here is especially RFC and not at all ready to go, the
kernel patch is pretty simple and probably OK, but still flagged as RFC
since it is part of the whole pair.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel