Re: [Xen-devel] [DOCS/PATCH] http://wiki.xen.org/wiki/X86_Paravirtualised_Memory_Management changes

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Mon, 2013-01-28 at 18:32 +0000, Konrad Rzeszutek Wilk wrote:
> These changes would allow the above mention Wiki to reference to
> more materials. This is what I had in mind to add to the Wiki:
> 
> diff --git a/intro.txt b/intro.txt
> index b902b1e..72d3ca1 100644
> --- a/intro.txt
> +++ b/intro.txt
> @@ -339,6 +339,188 @@ into the page table base register but will not be 
> explicitly pinned.
>  The initial virtual and pseudo-physical layout of a new guest is
>  described 
> [http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday
>  here].
>  
> +
> +At the start of the guest, the kernel image is read and the ELF section is

"When the guest is started...." (or "built")

> +parsed. The hypervisor looks a specific section that has the string âXenâ
> +(or âxenâ) to find out:

This is an ELF PT_NOTE program header (as opposed to a SHT_NOTE
section), right? I'm not 100% sure of the ELF terminology but it would
be good to be accurate (especially since PHDRS and Sections are
different things in ELF).

In ELF parlance the string is called the "name".

Is the name really "Xen" or "xen", or is it specifically one or the
other? I think it is "Xen" only, and note that it is NULL terminated.

>  where the kernel expects its virtual base address,
> +what type of hypervisor it can work with, certain features the kernel image
> +can support, and the location of the hypercall. There are two variants of 
> this:
                                                 ^page

(NB: A description of the hypercall page would be useful somewhere)

> +
> +; a). The lagacy ASCIIZ string with all of the keys concatenated. Each key

"legacy"

I think I would put the modern ELF note thing first and the legacy one
second, to give prominence to the ELF notes.

> +being a string and the equal sign with the value also being an string
> +(numeric values are typed as string). The delimiter is a comma.

Are they always in hex or is decimal acceptable?

> +The key can be up to 32 characters and the value up to 128 characters.
> +For example:
> +
> +     
> GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPERCALL_PAGE=0x2,LOADER=generic
> +
> +; b). A â.note.Xenâ section in ELF header conforming to the ELF .noteâ 
> format.

Extra " towards the end there?

I don't think the .note.Xen thing is part of the standard interface.
Looking at elf_xen_parse in libelf-dominfo.c it is the PT_NOTE which
matters.

(AIUI PHDRS are a loading time thing, Sections are a linking time thing)

> +This structure is a 4-byte aligned structure. First section is an numerical
> +key (aligned to 4 bytes);

The type field is part of the ELF spec I think. Whereas the specific
type values and the the rest (the descr) is defined by Xen as its
interpretation of things in the Xen namespace.

>  followed by either a string or a numerical value
> +(again, aligned to 4 bytes). The values can up to any length, if the key is
> +assumed to a string. If it is a numerical value, it is a long
> +(64-bit value - which means 8 bytes).

Could use a link to the ELF spec somewhere here.

> +
> +The parameters and its purpose are explained in
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,elfnote.h.html#incontents_elfnote
>  here].
> +
> +And the XEN_ELF_FEATURES values are explained in
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,features.h.html#incontents_elfnote_features
>  here].
> +
> +For example, if the ELF values were as so:
> +
> +{| border="1" cellpadding="2" cellspacing="0"
> + | Name of Xen ELF entry
> + | Contents
> + |-
> + | XEN_ELFNOTE_GUEST_OS (6)
> + | linux
> + |-
> + | XEN_ELFNOTE_GUEST_VERSION (7)
> + | 2.6
> + |-
> + | XEN_ELFNOTE_XEN_VERSION (5)
> + | xen-3.0
> + |-
> + | XEN_ELFNOTE_VIRT_BASE (3)
> + | 0xffffffff80000000
> + |-
> + | XEN_ELFNOTE_ENTRY (1)
> + | 0xffffffff81899200
> + |-
> + | XEN_ELFNOTE_HYPERCALL_PAGE (2)
> + | 0xffffffff81001000
> + |-
> + | XEN_ELFNOTE_FEATURES (10)
> + | !writable_page_tables|pae_pgdir_above_4gb
> + |-
> + | XEN_ELFNOTE_PAE_MODE (9)
> + | yes
> + |-
> + | XEN_ELFNOTE_LOADER (8)
> + | generic
> + |-
> + | XEN_ELFNOTE_SUSPEND_CANCEL (14)
> + | 1
> + |-
> + | XEN_ELFNOTE_HV_START_LOW
> + | 0xffff800000000000
> + |-
> + | XEN_ELFNOTE_PADDR_OFFSET
> + | 0
> + |}
> +
> +With that setup, the hypervisor constructs an initial page table that spans 
> the
> +region from virtual start address up (0xffffffff80000000) to the end of the
> +p2m map.
> +
> +For example, the layout (which then is going to be feed in

Not sure what you mean by "to be feed in". "which is used to initialise
the start of day structure <link>" ?


> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday]
> +can look as so:

Does xen make any guarantee about the pfn space layout at start of day?
I thought it was just the virtual space and the fact that pfn space was
contiguous..

I think an example is OK but you need to make it clear that this is just
an example and not part of the ABI.

> +
> +{| border="1" cellpadding="2" cellspacing="0"
> + | Page Frame (PFN)
> + | contents
> + |-
> + | 0x0
> + | location of 
> [http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_shared
>  struct shared_info]

The "Start-of-day memory layout" bit of the interface doesn't say
anything about the location of struct shared_info so I don't think you
can take this one as read?

> + |-
> + | 0x1000
> + | location of the kernel
> + |-
> + | 0x1001
> + | location of the hypercall within the kernel
> + |-
> + | 0x1E3E

This assumes kernel and ramdisk of particular sizes, might be worth
specifying and also being explicit about the fact that this is an
example layout given those inputs.

> + | ramdisk
> + |-
> + | 0xFC69
> + | phys2mach (P2M) - an array of machine frame numbers. The total size of 
> this array is dependent on the nr_pages in
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday
>  struct start_info]
> +and the architecture of the guest (each entry is four bytes
> +under 32-bit kernels and eight bytes under 64-bit kernels).
> + |-
> + | 0xFCE9
> + | location of 
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday
>  start_info structure]
> + |-
> + | 0xFCEA
> + | location of
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,xs_wire.h.html#incontents_xenstore_struct
>  XenStore structure].
> +Also refer to XXX docs/misc/xenstore.txt

http://xenbits.xen.org/docs/unstable/misc/xenstore.txt

> + |-
> + | 0xFCEB
> + | Depending on the type of the guest (initial domain or subsequent 
> domains), it can either be the
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_dom0_console
>  dom0_vga_console_info structure]
> +or 
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,console.h.html
>  XenConsole structure].
> +The parameters defining this location are in the 
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday
>  start_info structure]
> + |-
> + | 0xFCEC
> + | bootstrap page tables. These pagetables are loaded in the guest at startup
> +and cover from 0x0 up to 0xFD6f (the bootstack).
> + |-
> + | 0xFD6F
> + | bootstrap stack.
> + |}
> +
> +In terms of virtual addresses, these example PFNs are mapped to virtual 
> addresses
> +as follow:

This repeats the above quite a bit, perhaps a single table with columns
for both physical and virtual address would work better?

> +{| border="1" cellpadding="2" cellspacing="0"
> + | Virtual Address
> + | contents
> + |-
> + | 0xffffffff80000000
> + | location of 
> [http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_shared
>  struct shared_info]
> + |-
> + | 0xffffffff81000000
> + | location of the kernel
> + |-
> + | 0xffffffff81001000
> + | location of the hypercall within the kernel
> + |-
> + | 0xffffffff81e3e000 
> + | ramdisk
> + |-
> + | 0xffffffff8fc69000
> + | phys2mach (P2M) - an array of machine frame numbers. The total size of 
> this array is dependent on the nr_pages in
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday
>  struct start_info]
> +and the architecture of the guest (each entry is four bytes
> +under 32-bit kernels and eight bytes under 64-bit kernels).
> + |-
> + | 0xffffffff8fce9000
> + | location of 
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday
>  start_info structure]
> + |-
> + | 0xffffffff8fcea000
> + | location of
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,xs_wire.h.html#incontents_xenstore_struct
>  XenStore structure].
> +Also refer to XXX docs/misc/xenstore.txt
> + |-
> + | 0xffffffff8fceb000
> + | Depending on the type of the guest (initial domain or subsequent 
> domains), it can either be the
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_dom0_console
>  dom0_vga_console_info structure]
> +or 
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,console.h.html
>  XenConsole structure].
> +The parameters defining this location are in the 
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday
>  start_info structure]
> + |-
> + | 0xffffffff8fcfc000
> + | bootstrap page tables. These pagetables are loaded in the guest at startup
> +and cover from 0x0 up to 0xFD6f (the bootstack).
> + |-
> + | 0xffffffff8fd6f000
> + | bootstrap stack.
> + |}
> +
> +When the guest is launched, per
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday]
> +explanation, the register %esi contains the virtual address to the
> +start_info_t (0xffffffff8fce9000), the %cr3 points to the beginning of the
> +bootstrap page-tables (0xffffffff8fcfc000), and the %esp points to the
> +bootstrap stack (0xffffffff8fd6f000).
> +
>  = Virtual Address Space =
>  
>  Xen enforces certain restrictions on the virtual addresses which are
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel