[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [PATCH v7 00/34] arm64: Dom0 ITS emulation
Hi, an only slightly modified repost of the last version. I added the Reviewed-by: and Acked-by: tags from Stefano and Julien and rebased on top of the latest staging tree: commit 89216c7999eb5b8558bfac7d61ae0d5ab844ce3f Author: Dario Faggioli <dario.faggioli@xxxxxxxxxx> Date: Fri Apr 7 18:57:14 2017 +0200 xen: credit1: treat pCPUs more evenly during balancing. Other than that and one typo and comment fix the first 10 patches have not been changed. I dropped the addition of the GIC_IRQ_GUEST_LPI_PENDING bit in patch 12/36 and followed Stefano's suggestion, which led to the removal of former patches 17/36 and 23/36 and some simplification in later patches. I haven't been able to address most review comments from the last part of v5 yet, but will definitely still fix them. Detailed changelog below. Cheers, Andre ---------------------------------- This series adds support for emulation of an ARM GICv3 ITS interrupt controller. For hardware which relies on the ITS to provide interrupts for its peripherals this code is needed to get a machine booted into Dom0 at all. ITS emulation for DomUs is only really useful with PCI passthrough, which is not yet available for ARM. It is expected that this feature will be co-developed with the ITS DomU code. However this code drop here considered DomU emulation already, to keep later architectural changes to a minimum. This is technical preview version to allow early testing of the feature. Things not (properly) addressed in this release: - The MOVALL command is not emulated. In our case there is really nothing to do here. We might need to revisit this in the future for DomU support. - The INVALL command might need some rework to be more efficient. Currently we iterate over all mapped LPIs, which might take a bit longer. - Indirect tables are not supported. This affects both the host and the virtual side. - The command queue locking is currently suboptimal and should be made more fine-grained in the future, if possible. - We need to properly investigate the possible interaction when devices get removed. This requires to properly clean up and remove any associated resources like pending or in-flight LPIs, for instance. Some generic design principles: * The current GIC code statically allocates structures for each supported IRQ (both for the host and the guest), which due to the potentially millions of LPI interrupts is not feasible to copy for the ITS. So we refrain from introducing the ITS as a first class Xen interrupt controller, also we don't hold struct irq_desc's or struct pending_irq's for each possible LPI. Fortunately LPIs are only interesting to guests, so we get away with storing only the virtual IRQ number and the guest VCPU for each allocated host LPI, which can be stashed into one uint64_t. This data is stored in a two-level table, which is both memory efficient and quick to access. We hook into the existing IRQ handling and VGIC code to avoid accessing the normal structures, providing alternative methods for getting the needed information (priority, is enabled?) for LPIs. Whenever a guest maps a device, we allocate the maximum required number of struct pending_irq's, so that any triggering LPI can find its data structure. Upon the guest actually mapping the LPI, this pointer to the corresponding pending_irq gets entered into a radix tree, so that it can be quickly looked up. * On the guest side we (later will) have to deal with malicious guests trying to hog Xen with mapping requests for a lot of LPIs, for instance. As the ITS actually uses system memory for storing status information, we use this memory (which the guest has to provide) to naturally limit a guest. Whenever we need information from any of the ITS tables, we temporarily map them (which is cheap on arm64) and copy the required data. * An obvious approach to handling some guest ITS commands would be to propagate them to the host, for instance to map devices and LPIs and to enable or disable LPIs. However this (later with DomU support) will create an attack vector, as a malicious guest could try to fill the host command queue with propagated commands. So we try to avoid this situation: Dom0 sending a device mapping (MAPD) command is the only time we allow queuing commands to the host ITS command queue, as this seems to be the only reliable way of getting the required information at the moment. However at the same time we map all events to LPIs already, also enable them. This avoids sending commands later at runtime, as we can deal with mappings and LPI enabling/disabling internally. To accomodate the tech preview nature of this feature at the moment, there is a Kconfig option to enable it. Also it is supported on arm64 only, which will most likely not change in the future. This leads to some hideous constructs like an #ifdef'ed header file with empty function stubs to accomodate arm32 and non-ITS builds, which share some generic code paths with the ITS emulation. The number of supported LPIs can be limited on the command line, in case the number reported by the hardware is too high. As Xen cannot foresee how many interrupts the guests will need, we cater for as many as possible. The command line parameter is called max-lpi-bits and expresses the number of bits required to hold an interrupt ID. It defaults to 20, if that is lower than the number supported by the hardware. This code boots Dom0 on an ARM Fast Model with ITS support. I tried to address the issues seen by people running the previous versions on real hardware, though couldn't verify this here for myself. So any testing, bug reports (and possibly even fixes) are very welcome. The code can also be found on the its/v7 branch here: git://linux-arm.org/xen-ap.git http://www.linux-arm.org/git?p=xen-ap.git;a=shortlog;h=refs/heads/its/v7 Cheers, Andre Changelog v6 ... v7: - got rid of GIC_IRQ_GUEST_LPI_PENDING - typo and comment fix in 09/36 - rebased against latest staging - added Reviewed-by: and Acked-by: tags Changelog v5 ... v6: - reordered patches to allow splitting the series - introduced functions later to avoid warnings on intermediate builds - refactored common code changes into separate patches - removed bogus host_its_list from non-ITS build - dropped GENMASK_ULL and BIT_ULL (both patches and their usage later) - rework locking in MMIO register reads and writes - protect new code from being executed without an ITS being configured - fix vgic_access_guest_memory (now a separate patch) - some more comments and TODOs Changelog v4 ... v5: - adding many comments - spinlock asserts - rename r_host_lpis to max_host_lpi_ids - remove max_its_device_bits command line - add warning on high number of LPIs - avoid potential leak on host MAPD - properly handle nr_events rounding - remove unmap_all_devices(), replace with ASSERT - add barriers for (lockless) host LPI lookups - add proper locking in ITS and redist MMIO register handling - rollback failing device mapping - fix various printks - add vgic_access_guest_memory() and use it - (getting rid of page mapping functions and helpers) - drop table mapping / unmapping on redist/ITS enable/disable - minor reworks in functions as per review comments - fix ITS enablement check - move lpi_to_pending() and lpi_get_priority() to vgic_ops - move do_LPI() to gic_hw_ops - whitespace and hard tabs fixes - introduce ITS domain init function (and use it for the rbtree) - enable IRQs around do_LPI - implement TODOs for later optimizations - add "v" prefix to variables holding virtual properties - provide locked and normal versions of read/write_itte - only CLEAR LPI if not already guest visible (plus comment) - update LPI property on MAPTI - store vcpu_id in pending_irq for LPIs (helps INVALL) - improve INVALL implementation to only cover LPIs on this VCPU - improve virtual BASE register initialization - limit number of virtual LPIs to 24 bits (Linux bug at 32??) - only inject LPIs if redistributor is actually enabled Changelog v3 .. v4: - make HAS_ITS depend on EXPERT - introduce new patch 02 to initialize host ITS early - fix cmd_lock init position - introduce warning on high number of LPI allocations - various int -> unsigned fixes - adding and improving comments - rate limit ITS command queue full msg - drop unneeded checks - validate against allowed number of device IDs - avoid memory leaks when removing devices - improve algorithm for finding free host LPI - convert unmap_all_devices from goto to while loop - add message on remapping ITS device - name virtual device / event IDs properly - use atomic read when reading ITT entry Changelog v2 .. v3: - preallocate struct pending_irq's - map ITS and redistributor tables only on demand - store property, enable and pending bit in struct pending_irq - improve error checking and handling - add comments Changelog v1 .. v2: - clean up header file inclusion - rework host ITS table allocation: observe attributes, many fixes - remove patch 1 to export __flush_dcache_area, use existing function instead - use number of LPIs internally instead of number of bits - keep host_its_list as private as possible - keep struct its_devices private - rework gicv3_its_map_guest_devices - fix rbtree issues - more error handling and propagation - cope with GICv4 implementations (but no virtual LPI features!) - abstract host and guest ITSes by using doorbell addresses - join per-redistributor variables into one per-CPU structure - fix data types (unsigned int) - many minor bug fixes (Rough) changelog RFC-v2 .. v1: - split host ITS driver into gic-v3-lpi.c and gic-v3-its.c part - rename virtual ITS driver file to vgic-v3-its.c - use macros and named constants for all magic numbers - use atomic accessors for accessing the host LPI data - remove leftovers from connecting virtual and host ITSes - bail out if host ITS is disabled in the DT - rework map/unmap_guest_pages(): - split off p2m part as get/put_guest_pages (to be done on allocation) - get rid of vmap, using map_domain_page() instead - delay allocation of virtual tables until actual LPI/ITS enablement - properly size both virtual and physical tables upon allocation - fix put_domain() locking issues in physdev_op and LPI handling code - add and extend comments in various areas - fix lotsa coding style and white space issues, including comment style - add locking to data structures not yet covered - fix various locking issues - use an rbtree to deal with ITS devices (instead of a list) - properly handle memory attributes for ITS tables - handle cacheable/non-cacheable ITS table mappings - sanitize guest provided ITS/LPI table attributes - fix breakage on non-GICv2 compatible host GICv3 controllers - add command line parameters on top of Kconfig options - properly wait for an ITS to become quiescient before enabling it - handle host ITS command queue errors - actually wait for host ITS command completion (READR==WRITER) - fix ARM32 compilation - various patch splits and reorderings Andre Przywara (33): ARM: GICv3 ITS: parse and store ITS subnodes from hardware DT ARM: GICv3 ITS: initialize host ITS ARM: GICv3: allocate LPI pending and property table ARM: GICv3 ITS: allocate device and collection table ARM: GICv3 ITS: map ITS command buffer ARM: GICv3 ITS: introduce ITS command handling ARM: GICv3 ITS: introduce host LPI array ARM: vGICv3: introduce ITS emulation stub ARM: GICv3 ITS: introduce device mapping ARM: GIC: Add checks for NULL pointer pending_irq's ARM: GICv3: introduce separate pending_irq structs for LPIs ARM: GICv3: forward pending LPIs to guests ARM: GICv3: enable ITS and LPIs on the host ARM: vGICv3: handle virtual LPI pending and property tables ARM: vGICv3: re-use vgic_reg64_check_access ARM: GIC: export vgic_init_pending_irq() ARM: VGIC: add vcpu_id to struct pending_irq ARM: vGICv3: add virtual ITS list head and comment about iteration ARM: GICv3: prepare for virtual ITS subnodes ARM: vGIC: advertise LPI support ARM: vITS: add command handling stub and MMIO emulation ARM: vITS: introduce translation table walks ARM: vITS: handle CLEAR command ARM: vITS: handle INT command ARM: vITS: handle MAPC command ARM: vITS: handle MAPD command ARM: vITS: handle MAPTI command ARM: vITS: handle MOVI command ARM: vITS: handle DISCARD command ARM: vITS: handle INV command ARM: vITS: handle INVALL command ARM: vITS: create and initialize virtual ITSes for Dom0 ARM: vITS: create ITS subnodes for Dom0 DT Vijaya Kumar K (1): ARM: introduce vgic_access_guest_memory() docs/misc/xen-command-line.markdown | 9 + xen/arch/arm/Kconfig | 5 + xen/arch/arm/Makefile | 3 + xen/arch/arm/gic-v2.c | 7 + xen/arch/arm/gic-v3-its.c | 1020 +++++++++++++++++++++++++++ xen/arch/arm/gic-v3-lpi.c | 584 +++++++++++++++ xen/arch/arm/gic-v3.c | 81 ++- xen/arch/arm/gic.c | 31 +- xen/arch/arm/vgic-v2.c | 15 + xen/arch/arm/vgic-v3-its.c | 1325 +++++++++++++++++++++++++++++++++++ xen/arch/arm/vgic-v3.c | 274 +++++++- xen/arch/arm/vgic.c | 62 +- xen/include/asm-arm/domain.h | 14 +- xen/include/asm-arm/gic.h | 2 + xen/include/asm-arm/gic_v3_defs.h | 54 ++ xen/include/asm-arm/gic_v3_its.h | 263 +++++++ xen/include/asm-arm/irq.h | 16 + xen/include/asm-arm/vgic-emul.h | 9 + xen/include/asm-arm/vgic.h | 11 + 19 files changed, 3748 insertions(+), 37 deletions(-) create mode 100644 xen/arch/arm/gic-v3-its.c create mode 100644 xen/arch/arm/gic-v3-lpi.c create mode 100644 xen/arch/arm/vgic-v3-its.c create mode 100644 xen/include/asm-arm/gic_v3_its.h -- 2.8.2 _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx https://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |