[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH v7] xSplice v1 design and implementation.


v6: http://lists.xen.org/archives/html/xen-devel/2016-04/msg00871.html
 - Acted on all comments from Andrew, Julien, and Jan.
 - Got help from Andrew on one of them over the weekend.
 - Finally got my EFI box working (tests passed).
v5: http://lists.xen.org/archives/html/xen-devel/2016-03/msg03286.html
 - Acted at all comments from Jan.
v4: http://lists.xen.org/archives/html/xen-devel/2016-03/msg01776.html
 - Lots of review. Lots of rework. Some patches checked in.
v3: http://www.gossamer-threads.com/lists/xen/devel/418262
 - Act on all reviews.
 - Redo the flow of patches
v2: http://lists.xen.org/archives/html/xen-devel/2016-01/msg01597.html
 - Updated code/docs/design with review comments.
 - Make xen also have an PT_NOTE
 - Added more of Ross's patches
 - Combined build-id patchset with this.
(since the RFC and the Seattle Xen presentation)
 - Finished off some of the work around the build-id.
 - Settled on the preemption mechanism.
 - Cleaned the patches a lot up, broke them up to easy
   review for maintainers.
v1: http://lists.xenproject.org/archives/html/xen-devel/2015-09/msg02116.html
  - Put all the design comments in the code
[Posting by Ross]
 - Took all reviews into account.
 - Redid the patches

*Tools Maintainers*

All patches acked. The patch #3: "libxc: Implementation of XEN_XSPLICE_op in 
has errno=-EINVAL sprinkled on some return paths.

*Hypervisor Maintainers*

Jan, the hypervisor patches #2, #5-#17, #21-#23 need your Ack.

All hypervisor patches have been Reviewed by Andrew.

There is a new patch thanks to Andrew's help over the weekend!

 [PATCH v7 07/24] x86/mm: Introduce modify_xen_mappings()

All the XSM parts have been Acked.

The patch related to VERSION_build_id has been Acked:
 HYPERCALL_version_op: Add VERSION_build_id to retrieve build-id.

Patchset has been tested on ARM (tweaking the Kconfig to build xSplice and then
booting and using it under ARM CubieTruck - granted there is no patching
but the hypercalls work), ARM64 (only built it), and x86 (legacy and EFI)

*Are there any TODOs left from v5 or v6 reviews?*

One I hope can be deferred - that is xensyms_read which we use in
 "[PATCH v7 12/24] xsplice,symbols: Implement symbol name resolution on
 address." is not the fastest. It will need some tweaking (or a new
function will have to be written) and I hope that this can be done in v4.8.

The other is to test this on a 8 socket machine with tons of CPUs.
Somebody else is using this beast right now. The impact is whether the default
timeout of 30ms to quisce the CPUs should be increased on those beasts.

*What is xSplice?*

A mechanism to binarily patch the running hypervisor with new
opcodes that have come about due to primarily security updates.

*What will this patchset do once I've it*

Patch the hypervisor.

*Why are you emailing me?*

Please please review as many patches as possible.

*OK, what do you have?*

They are located at a git tree:
  git://xenbits.xen.org/people/konradwilk/xen.git xsplice.v7

(Copying from Ross's email):

Much of the work is implementing a basic version of the Linux kernel module
loader. The code:
* Loading of xSplice ELF payloads.
* Copying allocated sections into a new executable region of memory.
* Resolving symbols.
* Applying relocations.
* Patching of altinstructions.
* Special handling of bug frames and exception tables.
* Unloading of xSplice ELF payloads.
* Compiling a sample xSplice ELF payload
* Resolving symbols
* Using build-id dependencies
* Support for shadow variable framework
* Support for executing ELF payload functions on load/unload.

The other main bit of this work is applying and reverting the patches safely.
As implemented, the code is patched with each CPU waiting in the
return-to-guest path (i.e. with no stack) or on the cpu-idle path
which appears to be the safest way of patching. While it is safe we should
still (in the next wave of patches) to verify to not patch cetain critical
sections (say the code doing the patching)

All of the following should work:
* Applying patches safely.
* Reverting patches safely.
* Replacing patches safely (e.g. reverting any applied patches and applying
   a new patch).
* Bug frames as part of modules. This means adding or
  changing WARN, ASSERT, BUG, and run_in_exception_handler works correctly.
  Line number only changes _are ignored_.
* Exception tables as part of modules. E.g. wrmsr_safe and copy_to_user work
  correctly when used in a patch module.
* Stacking of patches on top of each other
* Resolving symbols (even of patches)


The above is enough to fully implement an update system where multiple source
patches are combined (using combinediff) and built into a single binary
which then atomically replaces any existing loaded patches
(this is why Ross added a REPLACE operation). This is the approach used
by kPatch and kGraft.

Multiple completely independent patches can also be loaded but unexpected
interactions may occur.

As it stands, the patches are statically linked which means that independent
patches cannot be linked against one another (e.g. if one introduces a
new symbol). Using the combinediff approach above fixes this.

Backtraces containing functions from a patch module do not show the symbol name.

There is no checking that a patch which is loaded is built for the
correct hypervisor (need to use build-id).

Binary patching works at the function level.


You can use the example code included in this patchset:

# xl info | grep extra
xen_extra              : -unstable
# xen-xsplice load /usr/lib/debug/xen_hello_world.xsplice
Uploading /usr/lib/debug/xen_hello_world.xsplice (2071 bytes)
Performing check: completed
Performing apply:. completed
# xl info | grep extra
xen_extra              : Hello World
# xen-xsplice revert xen_hello_world
Performing revert:. completed
# xen-xsplice unload xen_hello_world
Performing unload: completed
# xl info | grep extra
xen_extra              : -unstable

Or you can use git://xenbits.xen.org/people/konradwilk/xsplice-build-tools.git
which generates the ELF payloads.

This link has a nice description of how to use the tool:

Andrew Cooper (1):
      x86/mm: Introduce modify_xen_mappings()

Konrad Rzeszutek Wilk (14):
      xsplice: Design document
      xen/xsplice: Hypervisor implementation of XEN_XSPLICE_op
      libxc: Implementation of XEN_XSPLICE_op in libxc
      xen-xsplice: Tool to manipulate xsplice payloads
      arm/x86: Use struct virtual_region to do bug, symbol, and (x86) exception 
tables lookup.
      arm/x86/vmap: Add vmalloc_xen, vfree_xen and vm_init_type
      x86/xen_hello_world.xsplice: Test payload for patching 
      build_id: Provide ld-embedded build-ids
      HYPERCALL_version_op: Add VERSION_build_id to retrieve build-id.
      libxl: info: Display build_id of the hypervisor using XEN_VERSION_build_id
      xsplice: Print build_id in keyhandler and on bootup.
      xsplice: Stacking build-id dependency checking.
      xsplice/xen_replace_world: Test-case for XSPLICE_ACTION_REPLACE
      MAINTAINERS/xsplice: Add myself and Ross as the maintainers.

Ross Lagerwall (9):
      xsplice: Add helper elf routines
      xsplice: Implement payload loading
      xsplice: Implement support for applying/reverting/replacing patches.
      xsplice,symbols: Implement symbol name resolution on address.
      x86, xsplice: Print payload's symbol name and payload name in backtraces
      xsplice: Add support for bug frames.
      xsplice: Add support for exception tables.
      xsplice: Add support for alternatives
      xsplice: Prevent duplicate payloads from being loaded.

 .gitignore                                   |    5 +
 Config.mk                                    |   12 +
 MAINTAINERS                                  |   10 +
 docs/misc/xsplice.markdown                   | 1110 +++++++++++++++++++
 tools/flask/policy/policy/modules/xen/xen.te |    2 +
 tools/libxc/include/xenctrl.h                |   62 ++
 tools/libxc/xc_misc.c                        |  355 ++++++
 tools/libxl/libxl.c                          |   18 +-
 tools/libxl/libxl.h                          |    6 +
 tools/libxl/libxl_types.idl                  |    1 +
 tools/libxl/xl_cmdimpl.c                     |    1 +
 tools/misc/Makefile                          |    4 +
 tools/misc/xen-xsplice.c                     |  463 ++++++++
 xen/Makefile                                 |    8 +-
 xen/arch/arm/Makefile                        |    5 +-
 xen/arch/arm/kernel.c                        |    2 +-
 xen/arch/arm/mm.c                            |    2 +-
 xen/arch/arm/setup.c                         |    4 +
 xen/arch/arm/traps.c                         |   39 +-
 xen/arch/arm/xen.lds.S                       |   15 +-
 xen/arch/arm/xsplice.c                       |   87 ++
 xen/arch/x86/Makefile                        |   55 +-
 xen/arch/x86/alternative.c                   |   42 +-
 xen/arch/x86/boot/mkelf32.c                  |  129 ++-
 xen/arch/x86/domain.c                        |    2 +
 xen/arch/x86/extable.c                       |   41 +-
 xen/arch/x86/mm.c                            |   71 +-
 xen/arch/x86/setup.c                         |   28 +-
 xen/arch/x86/test/Makefile                   |   85 ++
 xen/arch/x86/test/xen_bye_world.c            |   34 +
 xen/arch/x86/test/xen_bye_world_func.c       |   24 +
 xen/arch/x86/test/xen_hello_world.c          |   33 +
 xen/arch/x86/test/xen_hello_world_func.c     |   38 +
 xen/arch/x86/test/xen_replace_world.c        |   34 +
 xen/arch/x86/test/xen_replace_world_func.c   |   24 +
 xen/arch/x86/traps.c                         |   45 +-
 xen/arch/x86/xen.lds.S                       |   28 +
 xen/arch/x86/xsplice.c                       |  264 +++++
 xen/common/Kconfig                           |   16 +
 xen/common/Makefile                          |    3 +
 xen/common/kernel.c                          |    4 +
 xen/common/symbols.c                         |   45 +-
 xen/common/sysctl.c                          |    6 +
 xen/common/version.c                         |   70 ++
 xen/common/virtual_region.c                  |  148 +++
 xen/common/vmap.c                            |  202 ++--
 xen/common/vsprintf.c                        |   19 +-
 xen/common/xsplice.c                         | 1528 ++++++++++++++++++++++++++
 xen/common/xsplice_elf.c                     |  495 +++++++++
 xen/drivers/acpi/osl.c                       |    2 +-
 xen/include/asm-x86/alternative.h            |    4 +
 xen/include/asm-x86/current.h                |   10 +-
 xen/include/asm-x86/uaccess.h                |    2 +
 xen/include/public/sysctl.h                  |  184 ++++
 xen/include/public/version.h                 |    3 +
 xen/include/xen/elfstructs.h                 |    7 +
 xen/include/xen/mm.h                         |    2 +
 xen/include/xen/symbols.h                    |   11 +
 xen/include/xen/version.h                    |    1 +
 xen/include/xen/virtual_region.h             |   47 +
 xen/include/xen/vmap.h                       |   21 +-
 xen/include/xen/xsplice.h                    |  157 +++
 xen/include/xen/xsplice_elf.h                |   58 +
 xen/xsm/flask/hooks.c                        |    7 +
 xen/xsm/flask/policy/access_vectors          |    4 +
 65 files changed, 6039 insertions(+), 205 deletions(-)

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.