|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [v3][PATCH 00/16] Fix RMRR
v3:
* Rearrange all patches orderly as Wei suggested
* Rebase on the latest tree
* Address some Wei's comments on tools side
* Two changes for runtime cycle
patch #2,xen/x86/p2m: introduce set_identity_p2m_entry, on hypervisor side
a>. Introduce paging_mode_translate()
Otherwise, we'll see this error when boot Xen/Dom0
(XEN) Assertion 'paging_mode_translate(p2m->domain)' failed at p2m-pt.c:702
(XEN) ----[ Xen-4.6-unstable x86_64 debug=y Tainted: C ]----
....
(XEN) Xen call trace:
(XEN) [<ffff82d0801f53db>] p2m_pt_get_entry+0x29/0x558
(XEN) [<ffff82d0801f0b5c>] set_identity_p2m_entry+0xfc/0x1f0
(XEN) [<ffff82d08014ebc8>] rmrr_identity_mapping+0x154/0x1ce
(XEN) [<ffff82d0802abb46>] intel_iommu_hwdom_init+0x76/0x158
(XEN) [<ffff82d0802ab169>] iommu_hwdom_init+0x179/0x188
(XEN) [<ffff82d0802cc608>] construct_dom0+0x2fed/0x35d8
(XEN) [<ffff82d0802bdaa0>] __start_xen+0x22d8/0x2381
(XEN) [<ffff82d080100067>] __high_start+0x53/0x55
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'paging_mode_translate(p2m->domain)' failed at p2m-pt.c:702
Note I don't copy all info since I think the above is enough.
b>. Actually we still need to use "mfn_x(mfn) == INVALID_MFN" to confirm
we're getting an invalid mfn.
* Add patch #16 to handle those devices which share same RMRR.
v2:
* Instead of that fixed predefined rdm memory boundary, we'd like to
introduce a parameter, "rdm_mem_boundary", to set this threshold value.
* Remove that existing USB hack.
* Make sure the MMIO regions all fit in the available resource window
* Rename our policy, "force/try" -> "strict/relaxed"
* Indeed, Wei and Jan gave me more and more comments to refine codes
* Code style
* Better and reasonable code implementation
* Correct or improve code comments.
* A little bit to work well with ARM.
Open:
* We should fail assigning device which has a shared RMRR with
another device. We can only do group assignment when RMRR is shared
among devices.
We need more time to figure a good policy/way out because something
is not clear to me.
As you know all devices are owned by Dom0 firstly before we create any
DomU, right? Do we allow Dom0 still own a group device while assign another
device in the same group?
Really appreciate any comments to policy.
v1:
RMRR is an acronym for Reserved Memory Region Reporting, expected to
be used for legacy usages (such as USB, UMA Graphics, etc.) requiring
reserved memory. Special treatment is required in system software to
setup those reserved regions in IOMMU translation structures, otherwise
passing through a device with RMRR reported may not work correctly.
This patch set tries to enhance existing Xen RMRR implementation to fix
various reported and theoretical problems. Most noteworthy changes are
to setup identity mapping in p2m layer and handle possible conflicts between
reported regions and gfn space. Initial proposal can be found at:
http://lists.xenproject.org/archives/html/xen-devel/2015-01/msg00524.html
and after a long discussion a summarized agreement is here:
http://lists.xen.org/archives/html/xen-devel/2015-01/msg01580.html
Below is a key summary of this patch set according to agreed proposal:
1. Use RDM (Reserved Device Memory) name in user space as a general
description instead of using ACPI RMRR name directly.
2. Introduce configuration parameters to allow user control both per-device
and global RDM resources along with desired policies upon a detected conflict.
3. Introduce a new hypercall to query global and per-device RDM resources.
4. Extend libxl to be a central place to manage RDM resources and handle
potential conflicts between reserved regions and gfn space. One simplification
goal is made to keep existing lowmem / mmio / highmem layout which is
passed around various function blocks. So a reasonable assumption
is made, that conflicts falling into below areas are not re-arranged otherwise
it will result in a more scattered layout:
a) in highmem region (>4G)
b) in lowmem region, and below a predefined boundary (default 2G)
a) is a new assumption not discussed before. From VT-d spec this is
possible but no such observation in real-world. So we can make this
reasonable assumption until there's real usage on it.
5. Extend XENMEM_set_memory_map usable for HVM guest, and then have
libxl to use that hypercall to carry RDM information to hvmloader. There
is one difference from original discussion. Previously we discussed to
introduce a new E820 type specifically for RDM entries. After more thought
we think it's OK to just tag them as E820_reserved. Actually hvmloader
doesn't need to know whether the reserved entries come from RDM or
from other purposes.
6. Then in hvmloader the change is generic for XENMEM_memory_map
change. Given a predefined memory layout, hvmloader should avoid
allocating all reserved entries for other usages (opregion, mmio, etc.)
7. Extend existing device passthrough hypercall to carry conflict handling
policy.
8. Setup identity map in p2m layer for RMRRs reported for the given
device. And conflicts are handled according to specified policy in hypercall.
Current patch set contains core enhancements calling for comments.
There are still several tasks not implemented now. We'll include them
in final version after RFC is agreed:
- remove existing USB hack
- detect and fail assigning device which has a shared RMRR with another device
- add a config parameter to configure that memory boundary flexibly
- In the case of hotplug we also need to figure out a way to fix that policy
conflict between the per-pci policy and the global policy but firstly we think
we'd better collect some good or correct ideas to step next in RFC.
So here I made this as RFC to collect your any comments.
----------------------------------------------------------------
Jan Beulich (1):
xen: introduce XENMEM_reserved_device_memory_map
Tiejun Chen (15):
xen/x86/p2m: introduce set_identity_p2m_entry
xen/vtd: create RMRR mapping
xen/passthrough: extend hypercall to support rdm reservation policy
xen: enable XENMEM_memory_map in hvm
hvmloader: get guest memory map into memory_map[]
hvmloader/pci: skip reserved ranges
hvmloader/e820: construct guest e820 table
tools/libxc: Expose new hypercall xc_reserved_device_memory_map
tools: extend xc_assign_device() to support rdm reservation policy
tools: introduce some new parameters to set rdm policy
tools/libxl: passes rdm reservation policy
tools/libxl: detect and avoid conflicts with RDM
tools/libxl: extend XENMEM_set_memory_map
xen/vtd: enable USB device assignment
xen/vtd: prevent from assign the device with shared rmrr
docs/man/xl.cfg.pod.5 | 71 ++++++
docs/man/xl.pod.1 | 7 +-
docs/misc/vtd.txt | 24 ++
tools/firmware/hvmloader/e820.c | 62 +++--
tools/firmware/hvmloader/e820.h | 7 +
tools/firmware/hvmloader/hvmloader.c | 37 +++
tools/firmware/hvmloader/pci.c | 36 ++-
tools/firmware/hvmloader/util.c | 26 ++
tools/firmware/hvmloader/util.h | 11 +
tools/libxc/include/xenctrl.h | 11 +-
tools/libxc/xc_domain.c | 42 +++-
tools/libxc/xc_hvm_build_x86.c | 5 +-
tools/libxl/libxl.h | 6 +
tools/libxl/libxl_create.c | 19 +-
tools/libxl/libxl_dm.c | 255 ++++++++++++++++++++
tools/libxl/libxl_dom.c | 16 +-
tools/libxl/libxl_internal.h | 37 ++-
tools/libxl/libxl_pci.c | 14 +-
tools/libxl/libxl_types.idl | 26 ++
tools/libxl/libxl_x86.c | 78 ++++++
tools/libxl/libxlu_pci.c | 92 +++++++
tools/libxl/libxlutil.h | 4 +
tools/libxl/xl_cmdimpl.c | 36 ++-
tools/libxl/xl_cmdtable.c | 2 +-
tools/ocaml/libs/xc/xenctrl_stubs.c | 18 +-
tools/python/xen/lowlevel/xc/xc.c | 29 ++-
xen/arch/x86/hvm/hvm.c | 2 -
xen/arch/x86/mm.c | 6 -
xen/arch/x86/mm/p2m.c | 47 ++++
xen/common/compat/memory.c | 66 +++++
xen/common/memory.c | 64 +++++
xen/drivers/passthrough/amd/pci_amd_iommu.c | 3 +-
xen/drivers/passthrough/arm/smmu.c | 2 +-
xen/drivers/passthrough/device_tree.c | 11 +-
xen/drivers/passthrough/iommu.c | 10 +
xen/drivers/passthrough/pci.c | 10 +-
xen/drivers/passthrough/vtd/dmar.c | 32 +++
xen/drivers/passthrough/vtd/dmar.h | 1 -
xen/drivers/passthrough/vtd/extern.h | 1 +
xen/drivers/passthrough/vtd/iommu.c | 63 +++--
xen/drivers/passthrough/vtd/utils.c | 7 -
xen/include/asm-x86/p2m.h | 4 +
xen/include/public/domctl.h | 5 +
xen/include/public/memory.h | 32 ++-
xen/include/xen/iommu.h | 12 +-
xen/include/xen/pci.h | 2 +
xen/include/xlat.lst | 3 +-
47 files changed, 1264 insertions(+), 90 deletions(-)
Thanks
Tiejun
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |