RE: Proposal for Porting Xen to Armv8-R64 - DraftA

[Wei Chen <Wei.Chen@xxxxxxx>]

Hi Julien,

> -----Original Message-----
> From: Julien Grall <julien@xxxxxxx>
> Sent: 2022年3月4日 3:51
> To: Wei Chen <Wei.Chen@xxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx; Stefano
> Stabellini <sstabellini@xxxxxxxxxx>
> Cc: Bertrand Marquis <Bertrand.Marquis@xxxxxxx>; Penny Zheng
> <Penny.Zheng@xxxxxxx>; Henry Wang <Henry.Wang@xxxxxxx>; nd <nd@xxxxxxx>
> Subject: Re: Proposal for Porting Xen to Armv8-R64 - DraftA
> 
> Hi Wei,
> 
> On 03/03/2022 02:06, Wei Chen wrote:
> >> -----Original Message-----
> >> From: Julien Grall <julien@xxxxxxx>
> >> Sent: 2022年3月2日 20:00
> >> To: Wei Chen <Wei.Chen@xxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx;
> Stefano
> >> Stabellini <sstabellini@xxxxxxxxxx>
> >> Cc: Bertrand Marquis <Bertrand.Marquis@xxxxxxx>; Penny Zheng
> >> <Penny.Zheng@xxxxxxx>; Henry Wang <Henry.Wang@xxxxxxx>; nd <nd@xxxxxxx>
> >> Subject: Re: Proposal for Porting Xen to Armv8-R64 - DraftA
> >>
> >>
> >>
> >> On 01/03/2022 07:51, Wei Chen wrote:
> >>> Hi Julien,
> >>
> >> Hi Wei,
> >>
> >>>> -----Original Message-----
> >>>> From: Julien Grall <julien@xxxxxxx>
> >>>> Sent: 2022年2月26日 4:55
> >>>> To: Wei Chen <Wei.Chen@xxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx;
> >> Stefano
> >>>> Stabellini <sstabellini@xxxxxxxxxx>
> >>>> Cc: Bertrand Marquis <Bertrand.Marquis@xxxxxxx>; Penny Zheng
> >>>> <Penny.Zheng@xxxxxxx>; Henry Wang <Henry.Wang@xxxxxxx>; nd
> <nd@xxxxxxx>
> >>>> Subject: Re: Proposal for Porting Xen to Armv8-R64 - DraftA
> >>>>> ### 1.2. Xen Challenges with PMSA Virtualization
> >>>>> Xen is PMSA unaware Type-1 Hypervisor, it will need modifications to
> >> run
> >>>>> with an MPU and host multiple guest OSes.
> >>>>>
> >>>>> - No MMU at EL2:
> >>>>>        - No EL2 Stage 1 address translation
> >>>>>            - Xen provides fixed ARM64 virtual memory layout as basis
> of
> >>>> EL2
> >>>>>              stage 1 address translation, which is not applicable on
> >> MPU
> >>>> system,
> >>>>>              where there is no virtual addressing. As a result, any
> >>>> operation
> >>>>>              involving transition from PA to VA, like ioremap, needs
> >>>> modification
> >>>>>              on MPU system.
> >>>>>        - Xen's run-time addresses are the same as the link time
> >> addresses.
> >>>>>            - Enable PIC (position-independent code) on a real-time
> >> target
> >>>>>              processor probably very rare.
> >>>>
> >>>> Aside the assembly boot code and UEFI stub, Xen already runs at the
> >> same
> >>>> address as it was linked.
> >>>>
> >>>
> >>> But the difference is that, base on MMU, we can use the same link
> >> address
> >>> for all platforms. But on MPU system, we can't do it in the same way.
> >>
> >> I agree that we currently use the same link address for all the
> >> platforms. But this is also a problem when using MMU because EL2 has a
> >> single TTBR.
> >>
> >> At the moment we are switching page-tables with the MMU which is not
> >> safe. Instead we need to turn out the MMU off, switch page-tables and
> >> then turn on the MMU. This means we need to have an identity mapping of
> >> Xen in the page-tables. Assuming Xen is not relocated, the identity
> >> mapping may clash with Xen (or the rest of the virtual address map).
> >>
> >
> > Is this the same reason we create a dummy reloc section for EFI loader?
> 
> The relocations for the EFI loader are necessary because IIRC it is
> running with virt == phys.
> 
> But this brings to all sort of problem:
> 
> https://lore.kernel.org/all/20171221145521.29526-1-
> julien.grall@xxxxxxxxxx/
> 

It's interesting, I will have a look into that thread.

> [...]
> 
> >>>
> >>> Some callers that want to change a memory's attribute will set them.
> >> Something like
> >>> ioremap_nocache. I am not sure is this what you had asked : )
> >>
> >> I am a bit confused. If ioremap_nocache() can change the attribute,
> then
> >> why would ioremap_attr() not be able to do it?
> >>
> >
> > MMU based iorepmap_xxxx can use a new VA and new PTE to do this. But for
> > MPU, we can't do it, except you change the whole MPU region's attribute.
> > The reasons are:
> > 1. For V8R PMSA, one physical address only be existed one MPU region.
> > 2. There's not enough MPU regions for us to split one MPU region to
> >     multiple MPU regions (changed pages region and unmodified pages
> regions).
> 
> Ok. I think we should at least check the attributes requested match the
> one in the MPU.
> 

Yes, this is what we want to do.

> >
> >>>
> >>>>
> >>>>>                if ( CACHE_ATTR_need_change )
> >>>>>                    return NULL;
> >>>>>                return (void *)pa;
> >>>>>            }
> >>>>>            static inline void __iomem *ioremap_nocache(...)
> >>>>>            {
> >>>>>                return ioremap_attr(start, len,
> PAGE_HYPERVISOR_NOCACHE);
> >>>>>            }
> >>>>>            static inline void __iomem *ioremap_cache(...)
> >>>>>            {
> >>>>>                return ioremap_attr(start, len, PAGE_HYPERVISOR);
> >>>>>            }
> >>>>>            static inline void __iomem *ioremap_wc(...)
> >>>>>            {
> >>>>>                return ioremap_attr(start, len, PAGE_HYPERVISOR_WC);
> >>>>>            }
> >>>>>            void *ioremap(...)
> >>>>>            {
> >>>>>                return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE);
> >>>>>            }
> >>>>>
> >>>>>            ```
> >>>>>        4. For `alternative`, it depends on `vmap` too.
> >>>>
> >>>> The only reason we depend on vmap() is because the map the sections
> >>>> *text read-only and we enforce WnX. For VMSA, it would be possible to
> >>>> avoid vmap() with some rework. I don't know for PMSA.
> >>>>
> >>>
> >>> For PMSA, we still enforce WnX. For your use case, I assume it's
> >> alternative.
> >>> It still may have some possibility to avoid vmap(). But there may be
> >> some
> >>> security issues. We had thought to disable MPU -> update xen text ->
> >> enable
> >>> MPU to copy VMSA alternative's behavior. The problem with this,
> however,
> >>> is that at some point, all memory is RWX. There maybe some security
> >> risk. > But because it's in init stage, it probably doesn't matter as
> much
> >> as
> >> I thought.
> >>
> >> For boot code, we need to ensure the code is compliant to the Arm Arm.
> >> Other than that, it is OK to have the memory RWX for a short period of
> >> time.
> >>
> >> In fact, when we originally boot Xen, we don't enforce WnX. We will
> >> start to enforce when initializing the memory. But there are no blocker
> >> to delay it (other than writing the code :)).
> >
> > Ah, ok, it seems we still can implement alternative on MPU system.
> > I will update it in new version proposal, but place it in TODO, I don't
> > want to include it before single CPU support be merged. Because current
> > patch series is huge enough : )
> 
> That's fine with me. I am not expecting you to implement everything we
> discussed here from day 1! :)
> 

Great! Thanks~

> Cheers,
> 
> --
> Julien Grall