[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v2 5/6] ioreq-server: add support for multiple servers



> -----Original Message-----
> From: Andrew Cooper [mailto:andrew.cooper3@xxxxxxxxxx]
> Sent: 04 March 2014 12:06
> To: Paul Durrant
> Cc: xen-devel@xxxxxxxxxxxxx
> Subject: Re: [Xen-devel] [PATCH v2 5/6] ioreq-server: add support for
> multiple servers
> 
> On 04/03/14 11:40, Paul Durrant wrote:
> > The legacy 'catch-all' server is always created with id 0. Secondary
> > servers will have an id ranging from 1 to a limit set by the toolstack
> > via the 'max_emulators' build info field. This defaults to 1 so ordinarily
> > no extra special pages are reserved for secondary emulators. It may be
> > increased using the secondary_device_emulators parameter in xl.cfg(5).
> > There's no clear limit to apply to the number of emulators so I've not
> > applied one.
> >
> > Because of the re-arrangement of the special pages in a previous patch we
> > only need the addition of parameter HVM_PARAM_NR_IOREQ_SERVERS
> to determine
> > the layout of the shared pages for multiple emulators. Guests migrated in
> > from hosts without this patch will be lacking the save record which stores
> > the new parameter and so the guest is assumed to only have had a single
> > emulator.
> >
> > Added some more emacs boilerplate to xenctrl.h and xenguest.h
> >
> > Signed-off-by: Paul Durrant <paul.durrant@xxxxxxxxxx>
> 
> How does the build param interact with the hvmparam?  It appears not to.
> 

The build param is the value used to set the HVM param on domain create. (See 
setup_guest() just below where all the special PFNs are set).

> On migrate, the receiving side will have to know, out-of-band, what to
> set max_emulators to when building the domain.  The setparam code needs
> to validate the hvmparam against the build param and return
> -EINVAL/-E2BIG in the case that the hvmparam is too large.
> xc_domain_restore() needs to detect this and abort the migration if the
> guest can't be restored with the expected number of emulators.
> 

I don't think we need any of this. The HVM param is in the save record and so 
will be preserved. Migration in from a prior version of Xen will initialize the 
value to 1, which is correct for that case. The only need for max_emulators is 
to reserve the correct number of special pages when a new domain is built, so 
this is irrelevant for a domain restore; the only important thing is that the 
maximum number of emulators should never exceed the value of max_emulators when 
the domain was originally built, and the preservation of the HVM param ensures 
this.

  Paul

> ~Andrew
> 
> > ---
> >  docs/man/xl.cfg.pod.5            |    7 +
> >  tools/libxc/xc_domain.c          |  175 +++++++
> >  tools/libxc/xc_domain_restore.c  |   20 +
> >  tools/libxc/xc_domain_save.c     |   12 +
> >  tools/libxc/xc_hvm_build_x86.c   |   24 +-
> >  tools/libxc/xenctrl.h            |   51 ++
> >  tools/libxc/xenguest.h           |   12 +
> >  tools/libxc/xg_save_restore.h    |    1 +
> >  tools/libxl/libxl.h              |    8 +
> >  tools/libxl/libxl_create.c       |    3 +
> >  tools/libxl/libxl_dom.c          |    1 +
> >  tools/libxl/libxl_types.idl      |    1 +
> >  tools/libxl/xl_cmdimpl.c         |    3 +
> >  xen/arch/x86/hvm/hvm.c           |  951
> +++++++++++++++++++++++++++++++++++---
> >  xen/arch/x86/hvm/io.c            |    2 +-
> >  xen/include/asm-x86/hvm/domain.h |   23 +-
> >  xen/include/asm-x86/hvm/hvm.h    |    1 +
> >  xen/include/public/hvm/hvm_op.h  |   70 +++
> >  xen/include/public/hvm/ioreq.h   |    1 +
> >  xen/include/public/hvm/params.h  |    4 +-
> >  20 files changed, 1300 insertions(+), 70 deletions(-)
> >
> > diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> > index e15a49f..0226c55 100644
> > --- a/docs/man/xl.cfg.pod.5
> > +++ b/docs/man/xl.cfg.pod.5
> > @@ -1281,6 +1281,13 @@ specified, enabling the use of XenServer PV
> drivers in the guest.
> >  This parameter only takes effect when device_model_version=qemu-xen.
> >  See F<docs/misc/pci-device-reservations.txt> for more information.
> >
> > +=item B<secondary_device_emulators=NUMBER>
> > +
> > +If a number of secondary device emulators (i.e. in addition to
> > +qemu-xen or qemu-xen-traditional) are to be invoked to support the
> > +guest then this parameter can be set with the count of how many are
> > +to be used. The default value is zero.
> > +
> >  =back
> >
> >  =head2 Device-Model Options
> > diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
> > index 369c3f3..dfa905b 100644
> > --- a/tools/libxc/xc_domain.c
> > +++ b/tools/libxc/xc_domain.c
> > @@ -1284,6 +1284,181 @@ int xc_get_hvm_param(xc_interface *handle,
> domid_t dom, int param, unsigned long
> >      return rc;
> >  }
> >
> > +int xc_hvm_create_ioreq_server(xc_interface *xch,
> > +                               domid_t domid,
> > +                               ioservid_t *id)
> > +{
> > +    DECLARE_HYPERCALL;
> > +    DECLARE_HYPERCALL_BUFFER(xen_hvm_create_ioreq_server_t, arg);
> > +    int rc;
> > +
> > +    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
> > +    if ( arg == NULL )
> > +        return -1;
> > +
> > +    hypercall.op     = __HYPERVISOR_hvm_op;
> > +    hypercall.arg[0] = HVMOP_create_ioreq_server;
> > +    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
> > +    arg->domid = domid;
> > +    rc = do_xen_hypercall(xch, &hypercall);
> > +    *id = arg->id;
> > +    xc_hypercall_buffer_free(xch, arg);
> > +    return rc;
> > +}
> > +
> > +int xc_hvm_get_ioreq_server_info(xc_interface *xch,
> > +                                 domid_t domid,
> > +                                 ioservid_t id,
> > +                                 xen_pfn_t *pfn,
> > +                                 xen_pfn_t *buf_pfn,
> > +                                 evtchn_port_t *buf_port)
> > +{
> > +    DECLARE_HYPERCALL;
> > +    DECLARE_HYPERCALL_BUFFER(xen_hvm_get_ioreq_server_info_t,
> arg);
> > +    int rc;
> > +
> > +    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
> > +    if ( arg == NULL )
> > +        return -1;
> > +
> > +    hypercall.op     = __HYPERVISOR_hvm_op;
> > +    hypercall.arg[0] = HVMOP_get_ioreq_server_info;
> > +    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
> > +    arg->domid = domid;
> > +    arg->id = id;
> > +    rc = do_xen_hypercall(xch, &hypercall);
> > +    if ( rc != 0 )
> > +        goto done;
> > +
> > +    if ( pfn )
> > +        *pfn = arg->pfn;
> > +
> > +    if ( buf_pfn )
> > +        *buf_pfn = arg->buf_pfn;
> > +
> > +    if ( buf_port )
> > +        *buf_port = arg->buf_port;
> > +
> > +done:
> > +    xc_hypercall_buffer_free(xch, arg);
> > +    return rc;
> > +}
> > +
> > +int xc_hvm_map_io_range_to_ioreq_server(xc_interface *xch, domid_t
> domid,
> > +                                        ioservid_t id, int is_mmio,
> > +                                        uint64_t start, uint64_t end)
> > +{
> > +    DECLARE_HYPERCALL;
> > +
> DECLARE_HYPERCALL_BUFFER(xen_hvm_map_io_range_to_ioreq_server_t,
> arg);
> > +    int rc;
> > +
> > +    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
> > +    if ( arg == NULL )
> > +        return -1;
> > +
> > +    hypercall.op     = __HYPERVISOR_hvm_op;
> > +    hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
> > +    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
> > +    arg->domid = domid;
> > +    arg->id = id;
> > +    arg->is_mmio = is_mmio;
> > +    arg->start = start;
> > +    arg->end = end;
> > +    rc = do_xen_hypercall(xch, &hypercall);
> > +    xc_hypercall_buffer_free(xch, arg);
> > +    return rc;
> > +}
> > +
> > +int xc_hvm_unmap_io_range_from_ioreq_server(xc_interface *xch,
> domid_t domid,
> > +                                            ioservid_t id, int is_mmio,
> > +                                            uint64_t start)
> > +{
> > +    DECLARE_HYPERCALL;
> > +
> DECLARE_HYPERCALL_BUFFER(xen_hvm_unmap_io_range_from_ioreq_ser
> ver_t, arg);
> > +    int rc;
> > +
> > +    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
> > +    if ( arg == NULL )
> > +        return -1;
> > +
> > +    hypercall.op     = __HYPERVISOR_hvm_op;
> > +    hypercall.arg[0] = HVMOP_unmap_io_range_from_ioreq_server;
> > +    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
> > +    arg->domid = domid;
> > +    arg->id = id;
> > +    arg->is_mmio = is_mmio;
> > +    arg->start = start;
> > +    rc = do_xen_hypercall(xch, &hypercall);
> > +    xc_hypercall_buffer_free(xch, arg);
> > +    return rc;
> > +}
> > +
> > +int xc_hvm_map_pcidev_to_ioreq_server(xc_interface *xch, domid_t
> domid,
> > +                                      ioservid_t id, uint16_t bdf)
> > +{
> > +    DECLARE_HYPERCALL;
> > +
> DECLARE_HYPERCALL_BUFFER(xen_hvm_map_pcidev_to_ioreq_server_t,
> arg);
> > +    int rc;
> > +
> > +    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
> > +    if ( arg == NULL )
> > +        return -1;
> > +
> > +    hypercall.op     = __HYPERVISOR_hvm_op;
> > +    hypercall.arg[0] = HVMOP_map_pcidev_to_ioreq_server;
> > +    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
> > +    arg->domid = domid;
> > +    arg->id = id;
> > +    arg->bdf = bdf;
> > +    rc = do_xen_hypercall(xch, &hypercall);
> > +    xc_hypercall_buffer_free(xch, arg);
> > +    return rc;
> > +}
> > +
> > +int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
> domid_t domid,
> > +                                          ioservid_t id, uint16_t bdf)
> > +{
> > +    DECLARE_HYPERCALL;
> > +
> DECLARE_HYPERCALL_BUFFER(xen_hvm_unmap_pcidev_from_ioreq_serve
> r_t, arg);
> > +    int rc;
> > +
> > +    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
> > +    if ( arg == NULL )
> > +        return -1;
> > +
> > +    hypercall.op     = __HYPERVISOR_hvm_op;
> > +    hypercall.arg[0] = HVMOP_unmap_pcidev_from_ioreq_server;
> > +    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
> > +    arg->domid = domid;
> > +    arg->id = id;
> > +    arg->bdf = bdf;
> > +    rc = do_xen_hypercall(xch, &hypercall);
> > +    xc_hypercall_buffer_free(xch, arg);
> > +    return rc;
> > +}
> > +
> > +int xc_hvm_destroy_ioreq_server(xc_interface *xch,
> > +                                domid_t domid,
> > +                                ioservid_t id)
> > +{
> > +    DECLARE_HYPERCALL;
> > +    DECLARE_HYPERCALL_BUFFER(xen_hvm_destroy_ioreq_server_t, arg);
> > +    int rc;
> > +
> > +    arg = xc_hypercall_buffer_alloc(xch, arg, sizeof(*arg));
> > +    if ( arg == NULL )
> > +        return -1;
> > +
> > +    hypercall.op     = __HYPERVISOR_hvm_op;
> > +    hypercall.arg[0] = HVMOP_destroy_ioreq_server;
> > +    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
> > +    arg->domid = domid;
> > +    arg->id = id;
> > +    rc = do_xen_hypercall(xch, &hypercall);
> > +    xc_hypercall_buffer_free(xch, arg);
> > +    return rc;
> > +}
> > +
> >  int xc_domain_setdebugging(xc_interface *xch,
> >                             uint32_t domid,
> >                             unsigned int enable)
> > diff --git a/tools/libxc/xc_domain_restore.c
> b/tools/libxc/xc_domain_restore.c
> > index 1f6ce50..3116653 100644
> > --- a/tools/libxc/xc_domain_restore.c
> > +++ b/tools/libxc/xc_domain_restore.c
> > @@ -746,6 +746,7 @@ typedef struct {
> >      uint64_t acpi_ioport_location;
> >      uint64_t viridian;
> >      uint64_t vm_generationid_addr;
> > +    uint64_t nr_ioreq_servers;
> >
> >      struct toolstack_data_t tdata;
> >  } pagebuf_t;
> > @@ -996,6 +997,16 @@ static int pagebuf_get_one(xc_interface *xch,
> struct restore_ctx *ctx,
> >          DPRINTF("read generation id buffer address");
> >          return pagebuf_get_one(xch, ctx, buf, fd, dom);
> >
> > +    case XC_SAVE_ID_HVM_NR_IOREQ_SERVERS:
> > +        /* Skip padding 4 bytes then read the acpi ioport location. */
> > +        if ( RDEXACT(fd, &buf->nr_ioreq_servers, sizeof(uint32_t)) ||
> > +             RDEXACT(fd, &buf->nr_ioreq_servers, sizeof(uint64_t)) )
> > +        {
> > +            PERROR("error reading the number of IOREQ servers");
> > +            return -1;
> > +        }
> > +        return pagebuf_get_one(xch, ctx, buf, fd, dom);
> > +
> >      default:
> >          if ( (count > MAX_BATCH_SIZE) || (count < 0) ) {
> >              ERROR("Max batch size exceeded (%d). Giving up.", count);
> > @@ -1755,6 +1766,15 @@ int xc_domain_restore(xc_interface *xch, int
> io_fd, uint32_t dom,
> >      if (pagebuf.viridian != 0)
> >          xc_set_hvm_param(xch, dom, HVM_PARAM_VIRIDIAN, 1);
> >
> > +    if ( hvm ) {
> > +        int nr_ioreq_servers = pagebuf.nr_ioreq_servers;
> > +
> > +        if ( nr_ioreq_servers == 0 )
> > +            nr_ioreq_servers = 1;
> > +
> > +        xc_set_hvm_param(xch, dom, HVM_PARAM_NR_IOREQ_SERVERS,
> nr_ioreq_servers);
> > +    }
> > +
> >      if (pagebuf.acpi_ioport_location == 1) {
> >          DBGPRINTF("Use new firmware ioport from the checkpoint\n");
> >          xc_set_hvm_param(xch, dom,
> HVM_PARAM_ACPI_IOPORTS_LOCATION, 1);
> > diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
> > index 42c4752..3293e29 100644
> > --- a/tools/libxc/xc_domain_save.c
> > +++ b/tools/libxc/xc_domain_save.c
> > @@ -1731,6 +1731,18 @@ int xc_domain_save(xc_interface *xch, int io_fd,
> uint32_t dom, uint32_t max_iter
> >              PERROR("Error when writing the viridian flag");
> >              goto out;
> >          }
> > +
> > +        chunk.id = XC_SAVE_ID_HVM_NR_IOREQ_SERVERS;
> > +        chunk.data = 0;
> > +        xc_get_hvm_param(xch, dom, HVM_PARAM_NR_IOREQ_SERVERS,
> > +                         (unsigned long *)&chunk.data);
> > +
> > +        if ( (chunk.data != 0) &&
> > +             wrexact(io_fd, &chunk, sizeof(chunk)) )
> > +        {
> > +            PERROR("Error when writing the number of IOREQ servers");
> > +            goto out;
> > +        }
> >      }
> >
> >      if ( callbacks != NULL && callbacks->toolstack_save != NULL )
> > diff --git a/tools/libxc/xc_hvm_build_x86.c
> b/tools/libxc/xc_hvm_build_x86.c
> > index b65e702..6d6328a 100644
> > --- a/tools/libxc/xc_hvm_build_x86.c
> > +++ b/tools/libxc/xc_hvm_build_x86.c
> > @@ -45,7 +45,7 @@
> >  #define SPECIALPAGE_IDENT_PT 4
> >  #define SPECIALPAGE_CONSOLE  5
> >  #define SPECIALPAGE_IOREQ    6
> > -#define NR_SPECIAL_PAGES     SPECIALPAGE_IOREQ + 2 /* ioreq server
> needs 2 pages */
> > +#define NR_SPECIAL_PAGES(n)  SPECIALPAGE_IOREQ + (2 * n) /* ioreq
> server needs 2 pages */
> >  #define special_pfn(x) (0xff000u - 1 - (x))
> >
> >  #define VGA_HOLE_SIZE (0x20)
> > @@ -85,7 +85,8 @@ static int modules_init(struct xc_hvm_build_args
> *args,
> >  }
> >
> >  static void build_hvm_info(void *hvm_info_page, uint64_t mem_size,
> > -                           uint64_t mmio_start, uint64_t mmio_size)
> > +                           uint64_t mmio_start, uint64_t mmio_size,
> > +                           int max_emulators)
> >  {
> >      struct hvm_info_table *hvm_info = (struct hvm_info_table *)
> >          (((unsigned char *)hvm_info_page) + HVM_INFO_OFFSET);
> > @@ -113,7 +114,7 @@ static void build_hvm_info(void *hvm_info_page,
> uint64_t mem_size,
> >      /* Memory parameters. */
> >      hvm_info->low_mem_pgend = lowmem_end >> PAGE_SHIFT;
> >      hvm_info->high_mem_pgend = highmem_end >> PAGE_SHIFT;
> > -    hvm_info->reserved_mem_pgstart = special_pfn(0) -
> NR_SPECIAL_PAGES;
> > +    hvm_info->reserved_mem_pgstart = special_pfn(0) -
> NR_SPECIAL_PAGES(max_emulators);
> >
> >      /* Finish with the checksum. */
> >      for ( i = 0, sum = 0; i < hvm_info->length; i++ )
> > @@ -256,6 +257,10 @@ static int setup_guest(xc_interface *xch,
> >          stat_1gb_pages = 0;
> >      int pod_mode = 0;
> >      int claim_enabled = args->claim_enabled;
> > +    int max_emulators = args->max_emulators;
> > +
> > +    if ( max_emulators < 1 )
> > +        goto error_out;
> >
> >      if ( nr_pages > target_pages )
> >          pod_mode = XENMEMF_populate_on_demand;
> > @@ -468,12 +473,13 @@ static int setup_guest(xc_interface *xch,
> >                xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE,
> >                HVM_INFO_PFN)) == NULL )
> >          goto error_out;
> > -    build_hvm_info(hvm_info_page, v_end, mmio_start, mmio_size);
> > +    build_hvm_info(hvm_info_page, v_end, mmio_start, mmio_size,
> > +                   max_emulators);
> >      munmap(hvm_info_page, PAGE_SIZE);
> >
> >      /* Allocate and clear special pages. */
> >
> > -    DPRINTF("%d SPECIAL PAGES:\n", NR_SPECIAL_PAGES);
> > +    DPRINTF("%d SPECIAL PAGES:\n",
> NR_SPECIAL_PAGES(max_emulators));
> >      DPRINTF("  PAGING:    %"PRI_xen_pfn"\n",
> >              (xen_pfn_t)special_pfn(SPECIALPAGE_PAGING));
> >      DPRINTF("  ACCESS:    %"PRI_xen_pfn"\n",
> > @@ -486,10 +492,10 @@ static int setup_guest(xc_interface *xch,
> >              (xen_pfn_t)special_pfn(SPECIALPAGE_IDENT_PT));
> >      DPRINTF("  CONSOLE:   %"PRI_xen_pfn"\n",
> >              (xen_pfn_t)special_pfn(SPECIALPAGE_CONSOLE));
> > -    DPRINTF("  IOREQ:     %"PRI_xen_pfn"\n",
> > +    DPRINTF("  IOREQ(%02d): %"PRI_xen_pfn"\n", max_emulators * 2,
> >              (xen_pfn_t)special_pfn(SPECIALPAGE_IOREQ));
> >
> > -    for ( i = 0; i < NR_SPECIAL_PAGES; i++ )
> > +    for ( i = 0; i < NR_SPECIAL_PAGES(max_emulators); i++ )
> >      {
> >          xen_pfn_t pfn = special_pfn(i);
> >          rc = xc_domain_populate_physmap_exact(xch, dom, 1, 0, 0, &pfn);
> > @@ -515,7 +521,9 @@ static int setup_guest(xc_interface *xch,
> >      xc_set_hvm_param(xch, dom, HVM_PARAM_IOREQ_PFN,
> >                       special_pfn(SPECIALPAGE_IOREQ));
> >      xc_set_hvm_param(xch, dom, HVM_PARAM_BUFIOREQ_PFN,
> > -                     special_pfn(SPECIALPAGE_IOREQ) - 1);
> > +                     special_pfn(SPECIALPAGE_IOREQ) - max_emulators);
> > +    xc_set_hvm_param(xch, dom, HVM_PARAM_NR_IOREQ_SERVERS,
> > +                     max_emulators);
> >
> >      /*
> >       * Identity-map page table is required for running with CR0.PG=0 when
> > diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
> > index 13f816b..84cab13 100644
> > --- a/tools/libxc/xenctrl.h
> > +++ b/tools/libxc/xenctrl.h
> > @@ -1801,6 +1801,47 @@ void xc_clear_last_error(xc_interface *xch);
> >  int xc_set_hvm_param(xc_interface *handle, domid_t dom, int param,
> unsigned long value);
> >  int xc_get_hvm_param(xc_interface *handle, domid_t dom, int param,
> unsigned long *value);
> >
> > +/*
> > + * IOREQ server API
> > + */
> > +int xc_hvm_create_ioreq_server(xc_interface *xch,
> > +                               domid_t domid,
> > +                               ioservid_t *id);
> > +
> > +int xc_hvm_get_ioreq_server_info(xc_interface *xch,
> > +                                 domid_t domid,
> > +                                 ioservid_t id,
> > +                                 xen_pfn_t *pfn,
> > +                                 xen_pfn_t *buf_pfn,
> > +                                 evtchn_port_t *buf_port);
> > +
> > +int xc_hvm_map_io_range_to_ioreq_server(xc_interface *xch,
> > +                                        domid_t domid,
> > +                                        ioservid_t id,
> > +                                        int is_mmio,
> > +                                        uint64_t start,
> > +                                        uint64_t end);
> > +
> > +int xc_hvm_unmap_io_range_from_ioreq_server(xc_interface *xch,
> > +                                            domid_t domid,
> > +                                            ioservid_t id,
> > +                                            int is_mmio,
> > +                                            uint64_t start);
> > +
> > +int xc_hvm_map_pcidev_to_ioreq_server(xc_interface *xch,
> > +                                      domid_t domid,
> > +                                      ioservid_t id,
> > +                                      uint16_t bdf);
> > +
> > +int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
> > +                                          domid_t domid,
> > +                                          ioservid_t id,
> > +                                          uint16_t bdf);
> > +
> > +int xc_hvm_destroy_ioreq_server(xc_interface *xch,
> > +                                domid_t domid,
> > +                                ioservid_t id);
> > +
> >  /* HVM guest pass-through */
> >  int xc_assign_device(xc_interface *xch,
> >                       uint32_t domid,
> > @@ -2428,3 +2469,13 @@ int xc_kexec_load(xc_interface *xch, uint8_t
> type, uint16_t arch,
> >  int xc_kexec_unload(xc_interface *xch, int type);
> >
> >  #endif /* XENCTRL_H */
> > +
> > +/*
> > + * Local variables:
> > + * mode: C
> > + * c-file-style: "BSD"
> > + * c-basic-offset: 4
> > + * tab-width: 4
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> > diff --git a/tools/libxc/xenguest.h b/tools/libxc/xenguest.h
> > index a0e30e1..1300933 100644
> > --- a/tools/libxc/xenguest.h
> > +++ b/tools/libxc/xenguest.h
> > @@ -234,6 +234,8 @@ struct xc_hvm_build_args {
> >      struct xc_hvm_firmware_module smbios_module;
> >      /* Whether to use claim hypercall (1 - enable, 0 - disable). */
> >      int claim_enabled;
> > +    /* Maximum number of emulators for VM */
> > +    int max_emulators;
> >  };
> >
> >  /**
> > @@ -306,3 +308,13 @@ xen_pfn_t *xc_map_m2p(xc_interface *xch,
> >                        int prot,
> >                        unsigned long *mfn0);
> >  #endif /* XENGUEST_H */
> > +
> > +/*
> > + * Local variables:
> > + * mode: C
> > + * c-file-style: "BSD"
> > + * c-basic-offset: 4
> > + * tab-width: 4
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> > diff --git a/tools/libxc/xg_save_restore.h b/tools/libxc/xg_save_restore.h
> > index f859621..5170b7f 100644
> > --- a/tools/libxc/xg_save_restore.h
> > +++ b/tools/libxc/xg_save_restore.h
> > @@ -259,6 +259,7 @@
> >  #define XC_SAVE_ID_HVM_ACCESS_RING_PFN  -16
> >  #define XC_SAVE_ID_HVM_SHARING_RING_PFN -17
> >  #define XC_SAVE_ID_TOOLSTACK          -18 /* Optional toolstack specific
> info */
> > +#define XC_SAVE_ID_HVM_NR_IOREQ_SERVERS -19
> >
> >  /*
> >  ** We process save/restore/migrate in batches of pages; the below
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index 06bbca6..5a70b76 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -95,6 +95,14 @@
> >  #define LIBXL_HAVE_BUILDINFO_EVENT_CHANNELS 1
> >
> >  /*
> > + * LIBXL_HAVE_BUILDINFO_HVM_MAX_EMULATORS indicates that the
> > + * max_emulators field is present in the hvm sections of
> > + * libxl_domain_build_info. This field can be used to reserve
> > + * extra special pages for secondary device emulators.
> > + */
> > +#define LIBXL_HAVE_BUILDINFO_HVM_MAX_EMULATORS 1
> > +
> > +/*
> >   * libxl ABI compatibility
> >   *
> >   * The only guarantee which libxl makes regarding ABI compatibility
> > diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> > index a604cd8..cce93d9 100644
> > --- a/tools/libxl/libxl_create.c
> > +++ b/tools/libxl/libxl_create.c
> > @@ -330,6 +330,9 @@ int libxl__domain_build_info_setdefault(libxl__gc
> *gc,
> >
> >          libxl_defbool_setdefault(&b_info->u.hvm.gfx_passthru, false);
> >
> > +        if (b_info->u.hvm.max_emulators < 1)
> > +            b_info->u.hvm.max_emulators = 1;
> > +
> >          break;
> >      case LIBXL_DOMAIN_TYPE_PV:
> >          libxl_defbool_setdefault(&b_info->u.pv.e820_host, false);
> > diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> > index 55f74b2..9de06f9 100644
> > --- a/tools/libxl/libxl_dom.c
> > +++ b/tools/libxl/libxl_dom.c
> > @@ -637,6 +637,7 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
> >      args.mem_size = (uint64_t)(info->max_memkb - info->video_memkb)
> << 10;
> >      args.mem_target = (uint64_t)(info->target_memkb - info-
> >video_memkb) << 10;
> >      args.claim_enabled = libxl_defbool_val(info->claim_mode);
> > +    args.max_emulators = info->u.hvm.max_emulators;
> >      if (libxl__domain_firmware(gc, info, &args)) {
> >          LOG(ERROR, "initializing domain firmware failed");
> >          goto out;
> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > index 649ce50..b707159 100644
> > --- a/tools/libxl/libxl_types.idl
> > +++ b/tools/libxl/libxl_types.idl
> > @@ -372,6 +372,7 @@ libxl_domain_build_info =
> Struct("domain_build_info",[
> >                                         ("xen_platform_pci", libxl_defbool),
> >                                         ("usbdevice_list",   
> > libxl_string_list),
> >                                         ("vendor_device",    
> > libxl_vendor_device),
> > +                                       ("max_emulators",    integer),
> >                                         ])),
> >                   ("pv", Struct(None, [("kernel", string),
> >                                        ("slack_memkb", MemKB),
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index 4fc46eb..cf9b67d 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -1750,6 +1750,9 @@ skip_vfb:
> >
> >              b_info->u.hvm.vendor_device = d;
> >          }
> > +
> > +        if (!xlu_cfg_get_long (config, "secondary_device_emulators", &l, 
> > 0))
> > +            b_info->u.hvm.max_emulators = l + 1;
> >      }
> >
> >      xlu_cfg_destroy(config);
> > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> > index fb2dd73..e8b73fa 100644
> > --- a/xen/arch/x86/hvm/hvm.c
> > +++ b/xen/arch/x86/hvm/hvm.c
> > @@ -357,14 +357,21 @@ static ioreq_t *get_ioreq(struct
> hvm_ioreq_server *s, int id)
> >  bool_t hvm_io_pending(struct vcpu *v)
> >  {
> >      struct domain *d = v->domain;
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> > -    ioreq_t *p;
> > +    struct list_head *entry;
> >
> > -    if ( !s )
> > -        return 0;
> > +    list_for_each ( entry, &d->arch.hvm_domain.ioreq_server_list )
> > +    {
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> > +        ioreq_t *p = get_ioreq(s, v->vcpu_id);
> >
> > -    p = get_ioreq(s, v->vcpu_id);
> > -    return ( p->state != STATE_IOREQ_NONE );
> > +        p = get_ioreq(s, v->vcpu_id);
> > +        if ( p->state != STATE_IOREQ_NONE )
> > +            return 1;
> > +    }
> > +
> > +    return 0;
> >  }
> >
> >  static void hvm_wait_on_io(struct domain *d, ioreq_t *p)
> > @@ -394,18 +401,20 @@ static void hvm_wait_on_io(struct domain *d,
> ioreq_t *p)
> >  void hvm_do_resume(struct vcpu *v)
> >  {
> >      struct domain *d = v->domain;
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> > +    struct list_head *entry;
> >
> >      check_wakeup_from_wait();
> >
> >      if ( is_hvm_vcpu(v) )
> >          pt_restore_timer(v);
> >
> > -    if ( s )
> > +    list_for_each ( entry, &d->arch.hvm_domain.ioreq_server_list )
> >      {
> > -        ioreq_t *p = get_ioreq(s, v->vcpu_id);
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> >
> > -        hvm_wait_on_io(d, p);
> > +        hvm_wait_on_io(d, get_ioreq(s, v->vcpu_id));
> >      }
> >
> >      /* Inject pending hw/sw trap */
> > @@ -543,6 +552,83 @@ static int hvm_print_line(
> >      return X86EMUL_OKAY;
> >  }
> >
> > +static int hvm_access_cf8(
> > +    int dir, uint32_t port, uint32_t bytes, uint32_t *val)
> > +{
> > +    struct vcpu *curr = current;
> > +    struct hvm_domain *hd = &curr->domain->arch.hvm_domain;
> > +    int rc;
> > +
> > +    BUG_ON(port < 0xcf8);
> > +    port -= 0xcf8;
> > +
> > +    spin_lock(&hd->pci_lock);
> > +
> > +    if ( dir == IOREQ_WRITE )
> > +    {
> > +        switch ( bytes )
> > +        {
> > +        case 4:
> > +            hd->pci_cf8 = *val;
> > +            break;
> > +
> > +        case 2:
> > +        {
> > +            uint32_t mask = 0xffff << (port * 8);
> > +            uint32_t subval = *val << (port * 8);
> > +
> > +            hd->pci_cf8 = (hd->pci_cf8 & ~mask) |
> > +                          (subval & mask);
> > +            break;
> > +        }
> > +
> > +        case 1:
> > +        {
> > +            uint32_t mask = 0xff << (port * 8);
> > +            uint32_t subval = *val << (port * 8);
> > +
> > +            hd->pci_cf8 = (hd->pci_cf8 & ~mask) |
> > +                          (subval & mask);
> > +            break;
> > +        }
> > +
> > +        default:
> > +            break;
> > +        }
> > +
> > +        /* We always need to fall through to the catch all emulator */
> > +        rc = X86EMUL_UNHANDLEABLE;
> > +    }
> > +    else
> > +    {
> > +        switch ( bytes )
> > +        {
> > +        case 4:
> > +            *val = hd->pci_cf8;
> > +            rc = X86EMUL_OKAY;
> > +            break;
> > +
> > +        case 2:
> > +            *val = (hd->pci_cf8 >> (port * 8)) & 0xffff;
> > +            rc = X86EMUL_OKAY;
> > +            break;
> > +
> > +        case 1:
> > +            *val = (hd->pci_cf8 >> (port * 8)) & 0xff;
> > +            rc = X86EMUL_OKAY;
> > +            break;
> > +
> > +        default:
> > +            rc = X86EMUL_UNHANDLEABLE;
> > +            break;
> > +        }
> > +    }
> > +
> > +    spin_unlock(&hd->pci_lock);
> > +
> > +    return rc;
> > +}
> > +
> >  static int handle_pvh_io(
> >      int dir, uint32_t port, uint32_t bytes, uint32_t *val)
> >  {
> > @@ -618,39 +704,53 @@ static void
> hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s, struct vcpu
> >      }
> >  }
> >
> > -static int hvm_create_ioreq_server(struct domain *d, domid_t domid)
> > +static int hvm_create_ioreq_server(struct domain *d, ioservid_t id,
> domid_t domid)
> >  {
> >      struct hvm_ioreq_server *s;
> >      unsigned long pfn;
> >      struct vcpu *v;
> >      int i, rc;
> >
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> >      rc = -EEXIST;
> > -    if ( d->arch.hvm_domain.ioreq_server != NULL )
> > -        goto fail_exist;
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        if ( s->id == id )
> > +            goto fail_exist;
> > +    }
> >
> > -    gdprintk(XENLOG_INFO, "%s: %d\n", __func__, d->domain_id);
> > +    gdprintk(XENLOG_INFO, "%s: %d:%d\n", __func__, d->domain_id, id);
> >
> >      rc = -ENOMEM;
> >      s = xzalloc(struct hvm_ioreq_server);
> >      if ( !s )
> >          goto fail_alloc;
> >
> > +    s->id = id;
> >      s->domain = d;
> >      s->domid = domid;
> > +    INIT_LIST_HEAD(&s->mmio_range_list);
> > +    INIT_LIST_HEAD(&s->portio_range_list);
> > +    INIT_LIST_HEAD(&s->pcidev_list);
> >
> >      for ( i = 0; i < MAX_HVM_VCPUS; i++ )
> >          s->ioreq_evtchn[i] = -1;
> >      s->buf_ioreq_evtchn = -1;
> >
> >      /* Initialize shared pages */
> > -    pfn = d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
> > +    pfn = d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN] - s->id;
> >
> >      hvm_init_ioreq_page(s, 0);
> >      if ( (rc = hvm_set_ioreq_page(s, 0, pfn)) < 0 )
> >          goto fail_set_ioreq;
> >
> > -    pfn = d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN];
> > +    pfn = d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] - s-
> >id;
> >
> >      hvm_init_ioreq_page(s, 1);
> >      if ( (rc = hvm_set_ioreq_page(s, 1, pfn)) < 0 )
> > @@ -664,10 +764,12 @@ static int hvm_create_ioreq_server(struct domain
> *d, domid_t domid)
> >              goto fail_add_vcpu;
> >      }
> >
> > -    d->arch.hvm_domain.ioreq_server = s;
> > +    list_add(&s->list_entry,
> > +             &d->arch.hvm_domain.ioreq_server_list);
> >
> >      domain_unpause(d);
> >
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> >      return 0;
> >
> >  fail_add_vcpu:
> > @@ -681,23 +783,33 @@ fail_set_ioreq:
> >      xfree(s);
> >  fail_alloc:
> >  fail_exist:
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> >      return rc;
> >  }
> >
> > -static void hvm_destroy_ioreq_server(struct domain *d)
> > +static void hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
> >  {
> >      struct hvm_ioreq_server *s;
> >      struct vcpu *v;
> >
> > -    gdprintk(XENLOG_INFO, "%s: %d\n", __func__, d->domain_id);
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> >
> > -    s = d->arch.hvm_domain.ioreq_server;
> > -    if ( !s )
> > -        return;
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry)
> > +    {
> > +        if ( s->id == id )
> > +            goto found;
> > +    }
> > +
> > +    goto done;
> > +
> > +found:
> > +    gdprintk(XENLOG_INFO, "%s: %d:%d\n", __func__, d->domain_id, id);
> >
> >      domain_pause(d);
> >
> > -    d->arch.hvm_domain.ioreq_server = NULL;
> > +    list_del_init(&s->list_entry);
> >
> >      for_each_vcpu ( d, v )
> >          hvm_ioreq_server_remove_vcpu(s, v);
> > @@ -708,31 +820,373 @@ static void hvm_destroy_ioreq_server(struct
> domain *d)
> >      hvm_destroy_ioreq_page(s, 0);
> >
> >      xfree(s);
> > +
> > +done:
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> >  }
> >
> > -static int hvm_get_ioreq_server_buf_port(struct domain *d,
> evtchn_port_t *port)
> > +static int hvm_get_ioreq_server_buf_port(struct domain *d, ioservid_t
> id,
> > +                                         evtchn_port_t *port)
> >  {
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> > +    struct list_head *entry;
> > +    int rc;
> >
> > -    if ( !s )
> > -        return -ENOENT;
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    rc = -ENOENT;
> > +    list_for_each ( entry,
> > +                    &d->arch.hvm_domain.ioreq_server_list )
> > +    {
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> > +
> > +        if ( s->id == id )
> > +        {
> > +            *port = s->buf_ioreq_evtchn;
> > +            rc = 0;
> > +            break;
> > +        }
> > +    }
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return rc;
> > +}
> > +
> > +static int hvm_get_ioreq_server_pfn(struct domain *d, ioservid_t id, int
> buf,
> > +                                    xen_pfn_t *pfn)
> > +{
> > +    struct list_head *entry;
> > +    int rc;
> > +
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    rc = -ENOENT;
> > +    list_for_each ( entry,
> > +                    &d->arch.hvm_domain.ioreq_server_list )
> > +    {
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> > +
> > +        if ( s->id == id )
> > +        {
> > +            int i = ( buf ) ? HVM_PARAM_BUFIOREQ_PFN :
> HVM_PARAM_IOREQ_PFN;
> > +
> > +            *pfn = d->arch.hvm_domain.params[i] - s->id;
> > +            rc = 0;
> > +            break;
> > +        }
> > +    }
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return rc;
> > +}
> > +
> > +static int hvm_map_io_range_to_ioreq_server(struct domain *d,
> ioservid_t id,
> > +                                            int is_mmio, uint64_t start, 
> > uint64_t end)
> > +{
> > +    struct hvm_ioreq_server *s;
> > +    struct hvm_io_range *x;
> > +    struct list_head *list;
> > +    int rc;
> > +
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    x = xmalloc(struct hvm_io_range);
> > +    if ( x == NULL )
> > +        return -ENOMEM;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    rc = -ENOENT;
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        if ( s->id == id )
> > +            goto found;
> > +    }
> > +
> > +    goto fail;
> > +
> > +found:
> > +    INIT_RCU_HEAD(&x->rcu);
> > +    x->start = start;
> > +    x->end = end;
> > +
> > +    list = ( is_mmio ) ? &s->mmio_range_list : &s->portio_range_list;
> > +    list_add_rcu(&x->list_entry, list);
> > +
> > +    gdprintk(XENLOG_DEBUG, "%d:%d: +%s %"PRIX64" - %"PRIX64"\n",
> > +             d->domain_id,
> > +             s->id,
> > +             ( is_mmio ) ? "MMIO" : "PORTIO",
> > +             x->start,
> > +             x->end);
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> >
> > -    *port = s->buf_ioreq_evtchn;
> >      return 0;
> > +
> > +fail:
> > +    xfree(x);
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return rc;
> >  }
> >
> > -static int hvm_get_ioreq_server_pfn(struct domain *d, int buf, xen_pfn_t
> *pfn)
> > +static void free_io_range(struct rcu_head *rcu)
> >  {
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> > -    int i;
> > +    struct hvm_io_range *x;
> >
> > -    if ( !s )
> > -        return -ENOENT;
> > +    x = container_of (rcu, struct hvm_io_range, rcu);
> > +
> > +    xfree(x);
> > +}
> > +
> > +static int hvm_unmap_io_range_from_ioreq_server(struct domain *d,
> ioservid_t id,
> > +                                                int is_mmio, uint64_t 
> > start)
> > +{
> > +    struct hvm_ioreq_server *s;
> > +    struct list_head *list, *entry;
> > +    int rc;
> > +
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    rc = -ENOENT;
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        if ( s->id == id )
> > +            goto found;
> > +    }
> > +
> > +    goto done;
> > +
> > +found:
> > +    list = ( is_mmio ) ? &s->mmio_range_list : &s->portio_range_list;
> > +
> > +    list_for_each ( entry,
> > +                    list )
> > +    {
> > +        struct hvm_io_range *x = list_entry(entry,
> > +                                            struct hvm_io_range,
> > +                                            list_entry);
> > +
> > +        if ( start == x->start )
> > +        {
> > +            gdprintk(XENLOG_DEBUG, "%d:%d: -%s %"PRIX64" - %"PRIX64"\n",
> > +                     d->domain_id,
> > +                     s->id,
> > +                     ( is_mmio ) ? "MMIO" : "PORTIO",
> > +                     x->start,
> > +                     x->end);
> > +
> > +            list_del_rcu(&x->list_entry);
> > +            call_rcu(&x->rcu, free_io_range);
> >
> > -    i = ( buf ) ? HVM_PARAM_BUFIOREQ_PFN : HVM_PARAM_IOREQ_PFN;
> > -    *pfn = d->arch.hvm_domain.params[i];
> > +            rc = 0;
> > +            break;
> > +        }
> > +    }
> > +
> > +done:
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return rc;
> > +}
> > +
> > +static int hvm_map_pcidev_to_ioreq_server(struct domain *d, ioservid_t
> id,
> > +                                          uint16_t bdf)
> > +{
> > +    struct hvm_ioreq_server *s;
> > +    struct hvm_pcidev *x;
> > +    int rc;
> > +
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    x = xmalloc(struct hvm_pcidev);
> > +    if ( x == NULL )
> > +        return -ENOMEM;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    rc = -ENOENT;
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        if ( s->id == id )
> > +            goto found;
> > +    }
> > +
> > +    goto fail;
> > +
> > +found:
> > +    INIT_RCU_HEAD(&x->rcu);
> > +    x->bdf = bdf;
> > +
> > +    list_add_rcu(&x->list_entry, &s->pcidev_list);
> > +
> > +    gdprintk(XENLOG_DEBUG, "%d:%d: +PCIDEV %04X\n",
> > +             d->domain_id,
> > +             s->id,
> > +             x->bdf);
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> >
> >      return 0;
> > +
> > +fail:
> > +    xfree(x);
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return rc;
> > +}
> > +
> > +static void free_pcidev(struct rcu_head *rcu)
> > +{
> > +    struct hvm_pcidev *x;
> > +
> > +    x = container_of (rcu, struct hvm_pcidev, rcu);
> > +
> > +    xfree(x);
> > +}
> > +
> > +static int hvm_unmap_pcidev_from_ioreq_server(struct domain *d,
> ioservid_t id,
> > +                                              uint16_t bdf)
> > +{
> > +    struct hvm_ioreq_server *s;
> > +    struct list_head *entry;
> > +    int rc;
> > +
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    rc = -ENOENT;
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        if ( s->id == id )
> > +            goto found;
> > +    }
> > +
> > +    goto done;
> > +
> > +found:
> > +    list_for_each ( entry,
> > +                    &s->pcidev_list )
> > +    {
> > +        struct hvm_pcidev *x = list_entry(entry,
> > +                                          struct hvm_pcidev,
> > +                                          list_entry);
> > +
> > +        if ( bdf == x->bdf )
> > +        {
> > +            gdprintk(XENLOG_DEBUG, "%d:%d: -PCIDEV %04X\n",
> > +                     d->domain_id,
> > +                     s->id,
> > +                     x->bdf);
> > +
> > +            list_del_rcu(&x->list_entry);
> > +            call_rcu(&x->rcu, free_pcidev);
> > +
> > +            rc = 0;
> > +            break;
> > +        }
> > +    }
> > +
> > +done:
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return rc;
> > +}
> > +
> > +static int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu
> *v)
> > +{
> > +    struct list_head *entry;
> > +    int rc;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    list_for_each ( entry,
> > +                    &d->arch.hvm_domain.ioreq_server_list )
> > +    {
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> > +
> > +        if ( (rc = hvm_ioreq_server_add_vcpu(s, v)) < 0 )
> > +            goto fail;
> > +    }
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return 0;
> > +
> > +fail:
> > +    list_for_each ( entry,
> > +                    &d->arch.hvm_domain.ioreq_server_list )
> > +    {
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> > +
> > +        hvm_ioreq_server_remove_vcpu(s, v);
> > +    }
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    return rc;
> > +}
> > +
> > +static void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct
> vcpu *v)
> > +{
> > +    struct list_head *entry;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> > +    list_for_each ( entry,
> > +                    &d->arch.hvm_domain.ioreq_server_list )
> > +    {
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> > +
> > +        hvm_ioreq_server_remove_vcpu(s, v);
> > +    }
> > +
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +}
> > +
> > +static void hvm_destroy_all_ioreq_servers(struct domain *d)
> > +{
> > +    ioservid_t id;
> > +
> > +    for ( id = 0;
> > +          id < d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS];
> > +          id++ )
> > +        hvm_destroy_ioreq_server(d, id);
> >  }
> >
> >  static int hvm_replace_event_channel(struct vcpu *v, domid_t
> remote_domid,
> > @@ -750,18 +1204,31 @@ static int hvm_replace_event_channel(struct
> vcpu *v, domid_t remote_domid,
> >      return 0;
> >  }
> >
> > -static int hvm_set_ioreq_server_domid(struct domain *d, domid_t
> domid)
> > +static int hvm_set_ioreq_server_domid(struct domain *d, ioservid_t id,
> domid_t domid)
> >  {
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> > +    struct hvm_ioreq_server *s;
> >      struct vcpu *v;
> >      int rc = 0;
> >
> > +    if ( id >= d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS] )
> > +        return -EINVAL;
> > +
> > +    spin_lock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> >      domain_pause(d);
> >
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        if ( s->id == id )
> > +            goto found;
> > +    }
> > +
> >      rc = -ENOENT;
> > -    if ( !s )
> > -        goto done;
> > +    goto done;
> >
> > +found:
> >      rc = 0;
> >      if ( s->domid == domid )
> >          goto done;
> > @@ -787,6 +1254,8 @@ static int hvm_set_ioreq_server_domid(struct
> domain *d, domid_t domid)
> >  done:
> >      domain_unpause(d);
> >
> > +    spin_unlock(&d->arch.hvm_domain.ioreq_server_lock);
> > +
> >      return rc;
> >  }
> >
> > @@ -817,6 +1286,9 @@ int hvm_domain_initialise(struct domain *d)
> >
> >      }
> >
> > +    spin_lock_init(&d->arch.hvm_domain.ioreq_server_lock);
> > +    INIT_LIST_HEAD(&d->arch.hvm_domain.ioreq_server_list);
> > +    spin_lock_init(&d->arch.hvm_domain.pci_lock);
> >      spin_lock_init(&d->arch.hvm_domain.irq_lock);
> >      spin_lock_init(&d->arch.hvm_domain.uc_lock);
> >
> > @@ -858,6 +1330,7 @@ int hvm_domain_initialise(struct domain *d)
> >      rtc_init(d);
> >
> >      register_portio_handler(d, 0xe9, 1, hvm_print_line);
> > +    register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
> >
> >      rc = hvm_funcs.domain_initialise(d);
> >      if ( rc != 0 )
> > @@ -888,7 +1361,7 @@ void hvm_domain_relinquish_resources(struct
> domain *d)
> >      if ( hvm_funcs.nhvm_domain_relinquish_resources )
> >          hvm_funcs.nhvm_domain_relinquish_resources(d);
> >
> > -    hvm_destroy_ioreq_server(d);
> > +    hvm_destroy_all_ioreq_servers(d);
> >
> >      msixtbl_pt_cleanup(d);
> >
> > @@ -1520,7 +1993,6 @@ int hvm_vcpu_initialise(struct vcpu *v)
> >  {
> >      int rc;
> >      struct domain *d = v->domain;
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> >
> >      hvm_asid_flush_vcpu(v);
> >
> > @@ -1563,12 +2035,9 @@ int hvm_vcpu_initialise(struct vcpu *v)
> >           && (rc = nestedhvm_vcpu_initialise(v)) < 0 ) /* teardown:
> nestedhvm_vcpu_destroy */
> >          goto fail5;
> >
> > -    if ( s )
> > -    {
> > -        rc = hvm_ioreq_server_add_vcpu(s, v);
> > -        if ( rc < 0 )
> > -            goto fail6;
> > -    }
> > +    rc = hvm_all_ioreq_servers_add_vcpu(d, v);
> > +    if ( rc < 0 )
> > +        goto fail6;
> >
> >      if ( v->vcpu_id == 0 )
> >      {
> > @@ -1604,10 +2073,8 @@ int hvm_vcpu_initialise(struct vcpu *v)
> >  void hvm_vcpu_destroy(struct vcpu *v)
> >  {
> >      struct domain *d = v->domain;
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> >
> > -    if ( s )
> > -        hvm_ioreq_server_remove_vcpu(s, v);
> > +    hvm_all_ioreq_servers_remove_vcpu(d, v);
> >
> >      nestedhvm_vcpu_destroy(v);
> >
> > @@ -1646,11 +2113,112 @@ void hvm_vcpu_down(struct vcpu *v)
> >      }
> >  }
> >
> > +static DEFINE_RCU_READ_LOCK(ioreq_server_rcu_lock);
> > +
> > +static struct hvm_ioreq_server *hvm_select_ioreq_server(struct vcpu *v,
> ioreq_t *p)
> > +{
> > +#define BDF(cf8) (((cf8) & 0x00ffff00) >> 8)
> > +
> > +    struct domain *d = v->domain;
> > +    struct hvm_ioreq_server *s;
> > +    uint8_t type;
> > +    uint64_t addr;
> > +
> > +    if ( p->type == IOREQ_TYPE_PIO &&
> > +         (p->addr & ~3) == 0xcfc )
> > +    {
> > +        /* PCI config data cycle */
> > +        type = IOREQ_TYPE_PCI_CONFIG;
> > +
> > +        spin_lock(&d->arch.hvm_domain.pci_lock);
> > +        addr = d->arch.hvm_domain.pci_cf8 + (p->addr & 3);
> > +        spin_unlock(&d->arch.hvm_domain.pci_lock);
> > +    }
> > +    else
> > +    {
> > +        type = p->type;
> > +        addr = p->addr;
> > +    }
> > +
> > +    rcu_read_lock(&ioreq_server_rcu_lock);
> > +
> > +    switch ( type )
> > +    {
> > +    case IOREQ_TYPE_COPY:
> > +    case IOREQ_TYPE_PIO:
> > +    case IOREQ_TYPE_PCI_CONFIG:
> > +        break;
> > +    default:
> > +        goto done;
> > +    }
> > +
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        switch ( type )
> > +        {
> > +            case IOREQ_TYPE_COPY:
> > +            case IOREQ_TYPE_PIO: {
> > +                struct list_head *list;
> > +                struct hvm_io_range *x;
> > +
> > +                list = ( type == IOREQ_TYPE_COPY ) ?
> > +                    &s->mmio_range_list :
> > +                    &s->portio_range_list;
> > +
> > +                list_for_each_entry ( x,
> > +                                      list,
> > +                                      list_entry )
> > +                {
> > +                    if ( (addr >= x->start) && (addr <= x->end) )
> > +                        goto found;
> > +                }
> > +                break;
> > +            }
> > +            case IOREQ_TYPE_PCI_CONFIG: {
> > +                struct hvm_pcidev *x;
> > +
> > +                list_for_each_entry ( x,
> > +                                      &s->pcidev_list,
> > +                                      list_entry )
> > +                {
> > +                    if ( BDF(addr) == x->bdf ) {
> > +                        p->type = type;
> > +                        p->addr = addr;
> > +                        goto found;
> > +                    }
> > +                }
> > +                break;
> > +            }
> > +        }
> > +    }
> > +
> > +done:
> > +    /* The catch-all server has id 0 */
> > +    list_for_each_entry ( s,
> > +                          &d->arch.hvm_domain.ioreq_server_list,
> > +                          list_entry )
> > +    {
> > +        if ( s->id == 0 )
> > +            goto found;
> > +    }
> > +
> > +    s = NULL;
> > +
> > +found:
> > +    rcu_read_unlock(&ioreq_server_rcu_lock);
> > +
> > +    return s;
> > +
> > +#undef BDF
> > +}
> > +
> >  int hvm_buffered_io_send(ioreq_t *p)
> >  {
> >      struct vcpu *v = current;
> >      struct domain *d = v->domain;
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> > +    struct hvm_ioreq_server *s;
> >      struct hvm_ioreq_page *iorp;
> >      buffered_iopage_t *pg;
> >      buf_ioreq_t bp;
> > @@ -1660,6 +2228,7 @@ int hvm_buffered_io_send(ioreq_t *p)
> >      /* Ensure buffered_iopage fits in a page */
> >      BUILD_BUG_ON(sizeof(buffered_iopage_t) > PAGE_SIZE);
> >
> > +    s = hvm_select_ioreq_server(v, p);
> >      if ( !s )
> >          return 0;
> >
> > @@ -1770,18 +2339,34 @@ static bool_t
> hvm_send_assist_req_to_server(struct hvm_ioreq_server *s,
> >
> >  bool_t hvm_send_assist_req(struct vcpu *v, ioreq_t *p)
> >  {
> > -    struct domain *d = v->domain;
> > -    struct hvm_ioreq_server *s = d->arch.hvm_domain.ioreq_server;
> > +    struct hvm_ioreq_server *s;
> >
> >      if ( unlikely(!vcpu_start_shutdown_deferral(v)) )
> >          return 0;
> >
> > +    s = hvm_select_ioreq_server(v, p);
> >      if ( !s )
> >          return 0;
> >
> >      return hvm_send_assist_req_to_server(s, v, p);
> >  }
> >
> > +void hvm_broadcast_assist_req(struct vcpu *v, ioreq_t *p)
> > +{
> > +    struct domain *d = v->domain;
> > +    struct list_head *entry;
> > +
> > +    list_for_each ( entry,
> > +                    &d->arch.hvm_domain.ioreq_server_list )
> > +    {
> > +        struct hvm_ioreq_server *s = list_entry(entry,
> > +                                                struct hvm_ioreq_server,
> > +                                                list_entry);
> > +
> > +        (void) hvm_send_assist_req_to_server(s, v, p);
> > +    }
> > +}
> > +
> >  void hvm_hlt(unsigned long rflags)
> >  {
> >      struct vcpu *curr = current;
> > @@ -4370,6 +4955,215 @@ static int hvmop_flush_tlb_all(void)
> >      return 0;
> >  }
> >
> > +static int hvmop_create_ioreq_server(
> > +    XEN_GUEST_HANDLE_PARAM(xen_hvm_create_ioreq_server_t) uop)
> > +{
> > +    struct domain *curr_d = current->domain;
> > +    xen_hvm_create_ioreq_server_t op;
> > +    struct domain *d;
> > +    ioservid_t id;
> > +    int rc;
> > +
> > +    if ( copy_from_guest(&op, uop, 1) )
> > +        return -EFAULT;
> > +
> > +    rc = rcu_lock_remote_domain_by_id(op.domid, &d);
> > +    if ( rc != 0 )
> > +        return rc;
> > +
> > +    rc = -EINVAL;
> > +    if ( !is_hvm_domain(d) )
> > +        goto out;
> > +
> > +    rc = -ENOSPC;
> > +    for ( id = 1;
> > +          id <  d-
> >arch.hvm_domain.params[HVM_PARAM_NR_IOREQ_SERVERS];
> > +          id++ )
> > +    {
> > +        rc = hvm_create_ioreq_server(d, id, curr_d->domain_id);
> > +        if ( rc == -EEXIST )
> > +            continue;
> > +
> > +        break;
> > +    }
> > +
> > +    if ( rc == -EEXIST )
> > +        rc = -ENOSPC;
> > +
> > +    if ( rc < 0 )
> > +        goto out;
> > +
> > +    op.id = id;
> > +
> > +    rc = copy_to_guest(uop, &op, 1) ? -EFAULT : 0;
> > +
> > +out:
> > +    rcu_unlock_domain(d);
> > +    return rc;
> > +}
> > +
> > +static int hvmop_get_ioreq_server_info(
> > +    XEN_GUEST_HANDLE_PARAM(xen_hvm_get_ioreq_server_info_t)
> uop)
> > +{
> > +    xen_hvm_get_ioreq_server_info_t op;
> > +    struct domain *d;
> > +    int rc;
> > +
> > +    if ( copy_from_guest(&op, uop, 1) )
> > +        return -EFAULT;
> > +
> > +    rc = rcu_lock_remote_domain_by_id(op.domid, &d);
> > +    if ( rc != 0 )
> > +        return rc;
> > +
> > +    rc = -EINVAL;
> > +    if ( !is_hvm_domain(d) )
> > +        goto out;
> > +
> > +    if ( (rc = hvm_get_ioreq_server_pfn(d, op.id, 0, &op.pfn)) < 0 )
> > +        goto out;
> > +
> > +    if ( (rc = hvm_get_ioreq_server_pfn(d, op.id, 1, &op.buf_pfn)) < 0 )
> > +        goto out;
> > +
> > +    if ( (rc = hvm_get_ioreq_server_buf_port(d, op.id, &op.buf_port)) < 0 )
> > +        goto out;
> > +
> > +    rc = copy_to_guest(uop, &op, 1) ? -EFAULT : 0;
> > +
> > +out:
> > +    rcu_unlock_domain(d);
> > +    return rc;
> > +}
> > +
> > +static int hvmop_map_io_range_to_ioreq_server(
> > +
> XEN_GUEST_HANDLE_PARAM(xen_hvm_map_io_range_to_ioreq_server_t
> ) uop)
> > +{
> > +    xen_hvm_map_io_range_to_ioreq_server_t op;
> > +    struct domain *d;
> > +    int rc;
> > +
> > +    if ( copy_from_guest(&op, uop, 1) )
> > +        return -EFAULT;
> > +
> > +    rc = rcu_lock_remote_domain_by_id(op.domid, &d);
> > +    if ( rc != 0 )
> > +        return rc;
> > +
> > +    rc = -EINVAL;
> > +    if ( !is_hvm_domain(d) )
> > +        goto out;
> > +
> > +    rc = hvm_map_io_range_to_ioreq_server(d, op.id, op.is_mmio,
> > +                                          op.start, op.end);
> > +
> > +out:
> > +    rcu_unlock_domain(d);
> > +    return rc;
> > +}
> > +
> > +static int hvmop_unmap_io_range_from_ioreq_server(
> > +
> XEN_GUEST_HANDLE_PARAM(xen_hvm_unmap_io_range_from_ioreq_ser
> ver_t) uop)
> > +{
> > +    xen_hvm_unmap_io_range_from_ioreq_server_t op;
> > +    struct domain *d;
> > +    int rc;
> > +
> > +    if ( copy_from_guest(&op, uop, 1) )
> > +        return -EFAULT;
> > +
> > +    rc = rcu_lock_remote_domain_by_id(op.domid, &d);
> > +    if ( rc != 0 )
> > +        return rc;
> > +
> > +    rc = -EINVAL;
> > +    if ( !is_hvm_domain(d) )
> > +        goto out;
> > +
> > +    rc = hvm_unmap_io_range_from_ioreq_server(d, op.id, op.is_mmio,
> > +                                              op.start);
> > +
> > +out:
> > +    rcu_unlock_domain(d);
> > +    return rc;
> > +}
> > +
> > +static int hvmop_map_pcidev_to_ioreq_server(
> > +
> XEN_GUEST_HANDLE_PARAM(xen_hvm_map_pcidev_to_ioreq_server_t)
> uop)
> > +{
> > +    xen_hvm_map_pcidev_to_ioreq_server_t op;
> > +    struct domain *d;
> > +    int rc;
> > +
> > +    if ( copy_from_guest(&op, uop, 1) )
> > +        return -EFAULT;
> > +
> > +    rc = rcu_lock_remote_domain_by_id(op.domid, &d);
> > +    if ( rc != 0 )
> > +        return rc;
> > +
> > +    rc = -EINVAL;
> > +    if ( !is_hvm_domain(d) )
> > +        goto out;
> > +
> > +    rc = hvm_map_pcidev_to_ioreq_server(d, op.id, op.bdf);
> > +
> > +out:
> > +    rcu_unlock_domain(d);
> > +    return rc;
> > +}
> > +
> > +static int hvmop_unmap_pcidev_from_ioreq_server(
> > +
> XEN_GUEST_HANDLE_PARAM(xen_hvm_unmap_pcidev_from_ioreq_serve
> r_t) uop)
> > +{
> > +    xen_hvm_unmap_pcidev_from_ioreq_server_t op;
> > +    struct domain *d;
> > +    int rc;
> > +
> > +    if ( copy_from_guest(&op, uop, 1) )
> > +        return -EFAULT;
> > +
> > +    rc = rcu_lock_remote_domain_by_id(op.domid, &d);
> > +    if ( rc != 0 )
> > +        return rc;
> > +
> > +    rc = -EINVAL;
> > +    if ( !is_hvm_domain(d) )
> > +        goto out;
> > +
> > +    rc = hvm_unmap_pcidev_from_ioreq_server(d, op.id, op.bdf);
> > +
> > +out:
> > +    rcu_unlock_domain(d);
> > +    return rc;
> > +}
> > +
> > +static int hvmop_destroy_ioreq_server(
> > +    XEN_GUEST_HANDLE_PARAM(xen_hvm_destroy_ioreq_server_t)
> uop)
> > +{
> > +    xen_hvm_destroy_ioreq_server_t op;
> > +    struct domain *d;
> > +    int rc;
> > +
> > +    if ( copy_from_guest(&op, uop, 1) )
> > +        return -EFAULT;
> > +
> > +    rc = rcu_lock_remote_domain_by_id(op.domid, &d);
> > +    if ( rc != 0 )
> > +        return rc;
> > +
> > +    rc = -EINVAL;
> > +    if ( !is_hvm_domain(d) )
> > +        goto out;
> > +
> > +    hvm_destroy_ioreq_server(d, op.id);
> > +    rc = 0;
> > +
> > +out:
> > +    rcu_unlock_domain(d);
> > +    return rc;
> > +}
> > +
> >  long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void)
> arg)
> >
> >  {
> > @@ -4378,6 +5172,41 @@ long do_hvm_op(unsigned long op,
> XEN_GUEST_HANDLE_PARAM(void) arg)
> >
> >      switch ( op )
> >      {
> > +    case HVMOP_create_ioreq_server:
> > +        rc = hvmop_create_ioreq_server(
> > +            guest_handle_cast(arg, xen_hvm_create_ioreq_server_t));
> > +        break;
> > +
> > +    case HVMOP_get_ioreq_server_info:
> > +        rc = hvmop_get_ioreq_server_info(
> > +            guest_handle_cast(arg, xen_hvm_get_ioreq_server_info_t));
> > +        break;
> > +
> > +    case HVMOP_map_io_range_to_ioreq_server:
> > +        rc = hvmop_map_io_range_to_ioreq_server(
> > +            guest_handle_cast(arg,
> xen_hvm_map_io_range_to_ioreq_server_t));
> > +        break;
> > +
> > +    case HVMOP_unmap_io_range_from_ioreq_server:
> > +        rc = hvmop_unmap_io_range_from_ioreq_server(
> > +            guest_handle_cast(arg,
> xen_hvm_unmap_io_range_from_ioreq_server_t));
> > +        break;
> > +
> > +    case HVMOP_map_pcidev_to_ioreq_server:
> > +        rc = hvmop_map_pcidev_to_ioreq_server(
> > +            guest_handle_cast(arg,
> xen_hvm_map_pcidev_to_ioreq_server_t));
> > +        break;
> > +
> > +    case HVMOP_unmap_pcidev_from_ioreq_server:
> > +        rc = hvmop_unmap_pcidev_from_ioreq_server(
> > +            guest_handle_cast(arg,
> xen_hvm_unmap_pcidev_from_ioreq_server_t));
> > +        break;
> > +
> > +    case HVMOP_destroy_ioreq_server:
> > +        rc = hvmop_destroy_ioreq_server(
> > +            guest_handle_cast(arg, xen_hvm_destroy_ioreq_server_t));
> > +        break;
> > +
> >      case HVMOP_set_param:
> >      case HVMOP_get_param:
> >      {
> > @@ -4466,9 +5295,9 @@ long do_hvm_op(unsigned long op,
> XEN_GUEST_HANDLE_PARAM(void) arg)
> >                  if ( a.value == DOMID_SELF )
> >                      a.value = curr_d->domain_id;
> >
> > -                rc = hvm_create_ioreq_server(d, a.value);
> > +                rc = hvm_create_ioreq_server(d, 0, a.value);
> >                  if ( rc == -EEXIST )
> > -                    rc = hvm_set_ioreq_server_domid(d, a.value);
> > +                    rc = hvm_set_ioreq_server_domid(d, 0, a.value);
> >                  break;
> >              case HVM_PARAM_ACPI_S_STATE:
> >                  /* Not reflexive, as we must domain_pause(). */
> > @@ -4533,6 +5362,10 @@ long do_hvm_op(unsigned long op,
> XEN_GUEST_HANDLE_PARAM(void) arg)
> >                  if ( a.value > SHUTDOWN_MAX )
> >                      rc = -EINVAL;
> >                  break;
> > +            case HVM_PARAM_NR_IOREQ_SERVERS:
> > +                if ( d == current->domain )
> > +                    rc = -EPERM;
> > +                break;
> >              }
> >
> >              if ( rc == 0 )
> > @@ -4567,7 +5400,7 @@ long do_hvm_op(unsigned long op,
> XEN_GUEST_HANDLE_PARAM(void) arg)
> >              case HVM_PARAM_BUFIOREQ_PFN:
> >              case HVM_PARAM_BUFIOREQ_EVTCHN:
> >                  /* May need to create server */
> > -                rc = hvm_create_ioreq_server(d, curr_d->domain_id);
> > +                rc = hvm_create_ioreq_server(d, 0, curr_d->domain_id);
> >                  if ( rc != 0 && rc != -EEXIST )
> >                      goto param_fail;
> >
> > @@ -4576,7 +5409,7 @@ long do_hvm_op(unsigned long op,
> XEN_GUEST_HANDLE_PARAM(void) arg)
> >                  case HVM_PARAM_IOREQ_PFN: {
> >                      xen_pfn_t pfn;
> >
> > -                    if ( (rc = hvm_get_ioreq_server_pfn(d, 0, &pfn)) < 0 )
> > +                    if ( (rc = hvm_get_ioreq_server_pfn(d, 0, 0, &pfn)) < 
> > 0 )
> >                          goto param_fail;
> >
> >                      a.value = pfn;
> > @@ -4585,7 +5418,7 @@ long do_hvm_op(unsigned long op,
> XEN_GUEST_HANDLE_PARAM(void) arg)
> >                  case HVM_PARAM_BUFIOREQ_PFN: {
> >                      xen_pfn_t pfn;
> >
> > -                    if ( (rc = hvm_get_ioreq_server_pfn(d, 1, &pfn)) < 0 )
> > +                    if ( (rc = hvm_get_ioreq_server_pfn(d, 0, 1, &pfn)) < 
> > 0 )
> >                          goto param_fail;
> >
> >                      a.value = pfn;
> > @@ -4594,7 +5427,7 @@ long do_hvm_op(unsigned long op,
> XEN_GUEST_HANDLE_PARAM(void) arg)
> >                  case HVM_PARAM_BUFIOREQ_EVTCHN: {
> >                      evtchn_port_t port;
> >
> > -                    if ( (rc = hvm_get_ioreq_server_buf_port(d, &port)) < 
> > 0 )
> > +                    if ( (rc = hvm_get_ioreq_server_buf_port(d, 0, &port)) 
> > < 0 )
> >                          goto param_fail;
> >
> >                      a.value = port;
> > diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
> > index c9adb94..ac0d867 100644
> > --- a/xen/arch/x86/hvm/io.c
> > +++ b/xen/arch/x86/hvm/io.c
> > @@ -75,7 +75,7 @@ void send_invalidate_req(void)
> >          .data = ~0UL, /* flush all */
> >      };
> >
> > -    (void)hvm_send_assist_req(v, &p);
> > +    hvm_broadcast_assist_req(v, &p);
> >  }
> >
> >  int handle_mmio(void)
> > diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-
> x86/hvm/domain.h
> > index a77b83d..e9da543 100644
> > --- a/xen/include/asm-x86/hvm/domain.h
> > +++ b/xen/include/asm-x86/hvm/domain.h
> > @@ -41,17 +41,38 @@ struct hvm_ioreq_page {
> >      void *va;
> >  };
> >
> > +struct hvm_io_range {
> > +    struct list_head    list_entry;
> > +    uint64_t            start, end;
> > +    struct rcu_head     rcu;
> > +};
> > +
> > +struct hvm_pcidev {
> > +    struct list_head    list_entry;
> > +    uint16_t            bdf;
> > +    struct rcu_head     rcu;
> > +};
> > +
> >  struct hvm_ioreq_server {
> > +    struct list_head       list_entry;
> > +    ioservid_t             id;
> >      struct domain          *domain;
> >      domid_t                domid;
> >      struct hvm_ioreq_page  ioreq;
> >      int                    ioreq_evtchn[MAX_HVM_VCPUS];
> >      struct hvm_ioreq_page  buf_ioreq;
> >      int                    buf_ioreq_evtchn;
> > +    struct list_head       mmio_range_list;
> > +    struct list_head       portio_range_list;
> > +    struct list_head       pcidev_list;
> >  };
> >
> >  struct hvm_domain {
> > -    struct hvm_ioreq_server *ioreq_server;
> > +    struct list_head        ioreq_server_list;
> > +    spinlock_t              ioreq_server_lock;
> > +    uint32_t                pci_cf8;
> > +    spinlock_t              pci_lock;
> > +
> >      struct pl_time         pl_time;
> >
> >      struct hvm_io_handler *io_handler;
> > diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-
> x86/hvm/hvm.h
> > index 40aeddf..4118669 100644
> > --- a/xen/include/asm-x86/hvm/hvm.h
> > +++ b/xen/include/asm-x86/hvm/hvm.h
> > @@ -229,6 +229,7 @@ int prepare_ring_for_helper(struct domain *d,
> unsigned long gmfn,
> >  void destroy_ring_for_helper(void **_va, struct page_info *page);
> >
> >  bool_t hvm_send_assist_req(struct vcpu *v, ioreq_t *p);
> > +void hvm_broadcast_assist_req(struct vcpu *v, ioreq_t *p);
> >
> >  void hvm_get_guest_pat(struct vcpu *v, u64 *guest_pat);
> >  int hvm_set_guest_pat(struct vcpu *v, u64 guest_pat);
> > diff --git a/xen/include/public/hvm/hvm_op.h
> b/xen/include/public/hvm/hvm_op.h
> > index a9aab4b..6b31189 100644
> > --- a/xen/include/public/hvm/hvm_op.h
> > +++ b/xen/include/public/hvm/hvm_op.h
> > @@ -23,6 +23,7 @@
> >
> >  #include "../xen.h"
> >  #include "../trace.h"
> > +#include "../event_channel.h"
> >
> >  /* Get/set subcommands: extra argument == pointer to xen_hvm_param
> struct. */
> >  #define HVMOP_set_param           0
> > @@ -270,6 +271,75 @@ struct xen_hvm_inject_msi {
> >  typedef struct xen_hvm_inject_msi xen_hvm_inject_msi_t;
> >  DEFINE_XEN_GUEST_HANDLE(xen_hvm_inject_msi_t);
> >
> > +typedef uint32_t ioservid_t;
> > +
> > +DEFINE_XEN_GUEST_HANDLE(ioservid_t);
> > +
> > +#define HVMOP_create_ioreq_server 17
> > +struct xen_hvm_create_ioreq_server {
> > +    domid_t domid;  /* IN - domain to be serviced */
> > +    ioservid_t id;  /* OUT - server id */
> > +};
> > +typedef struct xen_hvm_create_ioreq_server
> xen_hvm_create_ioreq_server_t;
> > +DEFINE_XEN_GUEST_HANDLE(xen_hvm_create_ioreq_server_t);
> > +
> > +#define HVMOP_get_ioreq_server_info 18
> > +struct xen_hvm_get_ioreq_server_info {
> > +    domid_t domid;          /* IN - domain to be serviced */
> > +    ioservid_t id;          /* IN - server id */
> > +    xen_pfn_t pfn;          /* OUT - ioreq pfn */
> > +    xen_pfn_t buf_pfn;      /* OUT - buf ioreq pfn */
> > +    evtchn_port_t buf_port; /* OUT - buf ioreq port */
> > +};
> > +typedef struct xen_hvm_get_ioreq_server_info
> xen_hvm_get_ioreq_server_info_t;
> > +DEFINE_XEN_GUEST_HANDLE(xen_hvm_get_ioreq_server_info_t);
> > +
> > +#define HVMOP_map_io_range_to_ioreq_server 19
> > +struct xen_hvm_map_io_range_to_ioreq_server {
> > +    domid_t domid;                  /* IN - domain to be serviced */
> > +    ioservid_t id;                  /* IN - handle from
> HVMOP_register_ioreq_server */
> > +    int is_mmio;                    /* IN - MMIO or port IO? */
> > +    uint64_aligned_t start, end;    /* IN - inclusive start and end of 
> > range */
> > +};
> > +typedef struct xen_hvm_map_io_range_to_ioreq_server
> xen_hvm_map_io_range_to_ioreq_server_t;
> >
> +DEFINE_XEN_GUEST_HANDLE(xen_hvm_map_io_range_to_ioreq_server_
> t);
> > +
> > +#define HVMOP_unmap_io_range_from_ioreq_server 20
> > +struct xen_hvm_unmap_io_range_from_ioreq_server {
> > +    domid_t domid;          /* IN - domain to be serviced */
> > +    ioservid_t id;          /* IN - handle from 
> > HVMOP_register_ioreq_server */
> > +    uint8_t is_mmio;        /* IN - MMIO or port IO? */
> > +    uint64_aligned_t start; /* IN - start address of the range to remove */
> > +};
> > +typedef struct xen_hvm_unmap_io_range_from_ioreq_server
> xen_hvm_unmap_io_range_from_ioreq_server_t;
> >
> +DEFINE_XEN_GUEST_HANDLE(xen_hvm_unmap_io_range_from_ioreq_se
> rver_t);
> > +
> > +#define HVMOP_map_pcidev_to_ioreq_server 21
> > +struct xen_hvm_map_pcidev_to_ioreq_server {
> > +    domid_t domid;      /* IN - domain to be serviced */
> > +    ioservid_t id;      /* IN - handle from HVMOP_register_ioreq_server */
> > +    uint16_t bdf;       /* IN - PCI bus/dev/func */
> > +};
> > +typedef struct xen_hvm_map_pcidev_to_ioreq_server
> xen_hvm_map_pcidev_to_ioreq_server_t;
> >
> +DEFINE_XEN_GUEST_HANDLE(xen_hvm_map_pcidev_to_ioreq_server_t);
> > +
> > +#define HVMOP_unmap_pcidev_from_ioreq_server 22
> > +struct xen_hvm_unmap_pcidev_from_ioreq_server {
> > +    domid_t domid;      /* IN - domain to be serviced */
> > +    ioservid_t id;      /* IN - handle from HVMOP_register_ioreq_server */
> > +    uint16_t bdf;       /* IN - PCI bus/dev/func */
> > +};
> > +typedef struct xen_hvm_unmap_pcidev_from_ioreq_server
> xen_hvm_unmap_pcidev_from_ioreq_server_t;
> >
> +DEFINE_XEN_GUEST_HANDLE(xen_hvm_unmap_pcidev_from_ioreq_serv
> er_t);
> > +
> > +#define HVMOP_destroy_ioreq_server 23
> > +struct xen_hvm_destroy_ioreq_server {
> > +    domid_t domid;          /* IN - domain to be serviced */
> > +    ioservid_t id;          /* IN - server id */
> > +};
> > +typedef struct xen_hvm_destroy_ioreq_server
> xen_hvm_destroy_ioreq_server_t;
> > +DEFINE_XEN_GUEST_HANDLE(xen_hvm_destroy_ioreq_server_t);
> > +
> >  #endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */
> >
> >  #endif /* __XEN_PUBLIC_HVM_HVM_OP_H__ */
> > diff --git a/xen/include/public/hvm/ioreq.h
> b/xen/include/public/hvm/ioreq.h
> > index f05d130..e84fa75 100644
> > --- a/xen/include/public/hvm/ioreq.h
> > +++ b/xen/include/public/hvm/ioreq.h
> > @@ -34,6 +34,7 @@
> >
> >  #define IOREQ_TYPE_PIO          0 /* pio */
> >  #define IOREQ_TYPE_COPY         1 /* mmio ops */
> > +#define IOREQ_TYPE_PCI_CONFIG   2 /* pci config ops */
> >  #define IOREQ_TYPE_TIMEOFFSET   7
> >  #define IOREQ_TYPE_INVALIDATE   8 /* mapcache */
> >
> > diff --git a/xen/include/public/hvm/params.h
> b/xen/include/public/hvm/params.h
> > index 517a184..4109b11 100644
> > --- a/xen/include/public/hvm/params.h
> > +++ b/xen/include/public/hvm/params.h
> > @@ -145,6 +145,8 @@
> >  /* SHUTDOWN_* action in case of a triple fault */
> >  #define HVM_PARAM_TRIPLE_FAULT_REASON 31
> >
> > -#define HVM_NR_PARAMS          32
> > +#define HVM_PARAM_NR_IOREQ_SERVERS 32
> > +
> > +#define HVM_NR_PARAMS          33
> >
> >  #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.