Re: Aligning Xen to physical memory maps on embedded systems

[Bertrand Marquis <Bertrand.Marquis@xxxxxxx>]

(+ Penny, Wei and Luca)

> On 23 Feb 2021, at 01:52, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> 
> On Mon, 22 Feb 2021, Levenglick Dov wrote:
>>>> The system has 2GB of RAM (0x00000000 - 0x80000000) of which Xen and
>>>> the DomU have an allocation of 1.25GB, per this memory map:
>>>> 1. DomU1: 0x60000000 - 0x80000000
>>>> 2. DomU2: 0x40000000 - 0x60000000
>>>> 3. Xen: 0x30000000 - 0x40000000
>>> 
>>> How did you tell Xen which regions is assigned to which guests? Are your
>>> domain mapped 1:1 (i.e guest physical address == host physical address)?
>> 
>> I am working on a solution where if the "xen,domain" memory has #size-cell 
>> cells the
>> content is backward compatible. But if it contains (#address-cells + 
>> #size-cells), the 
>> address cells should be considered the physical start address.
>> During the mapping of the entire address space insetup_mm(), the carved out 
>> addresses
>> would be added to the  reserved memory address space. When the DomU is to be 
>> created,
>> this physical space would be mapped to it. The virtual addresses are less of 
>> an issue and
>> needn't be mapped 1x1 (although they could be).
> 
> As of today neither upstream Xen nor the Xilinx Xen tree come with the
> feature of allowing the specification of an address range for dom0less
> guests.
> 
> The only thing that Xilinx Xen allows, which is not upstream yet, is the
> ability of creating dom0less guests 1:1 mapped using the "direct-map"
> property. But the memory allocation is still done by Xen (you can't
> select the addresses).
> 
> Some time ago I worked on a hacky prototype to allow the specification
> of address ranges, see:
> 
> http://xenbits.xenproject.org/git-http/people/sstabellini/xen-unstable.git 
> direct-map-2
> from 7372466b21c3b6c96bb7a52754e432bac883a1e3 onward.
> 
> In particular, have a look at "xen/arm: introduce 1:1 mapping for
> domUs". The work is not complete: it might not work depending on the
> memory ranges you select for your domUs. In particular, you can't select
> top-of-RAM addresses for your domUs. However, it might help you getting
> started.
> 
> 
>>>> I am able to support True Dom0-less by means of the patch/hack
>>>> demonstrated By Stefano Stabellini at
>>> https://youtu.be/UfiP9eAV0WA?t=1746.
>>>> 
>>>> I was able to forcefully put the Xen binary at the address range
>>>> immediately below 0x40000000 by means of modifying get_xen_paddr() -
>>> in itself an ugly hack.
>>>> 
>>>> My questions are:
>>>> 1. Since Xen performs runtime allocations from its heap, it is allocating
>>>>    downwards from 0x80000000 - thereby "stealing" memory from DomU1.
>>> 
>>> In theory, any memory reserved for domains should have been carved out
>>> from the heap allocator. This would be sufficient to prevent Xen allocating
>>> memory from the ranges you described above.
>>> 
>>> Therefore, to me this looks like a bug in the tree you are using.
>> 
>> This would be a better approach, but because Xen perform allocations from its
>> heap prior to allocating memory to DomU - and since it allocates from the 
>> top of
>> the heap - it is basically taking memory that I wanted to set aside for the 
>> DomU.
> 
> Yeah, this is the main problem that my prototype above couldn't solve.
> 

Wei and Penny are working on direct map and static allocation to fit embedded 
use
cases an might have more answer there.

On the fix from Stefano explained in the video, Luca Fancellu made a patch to 
propose
a long term solution and will push it upstream next week.

Cheers
Bertrand