RE: Linux pin_user_pages_fast fails on Xen

["NK, JESHWANTHKUMAR (JESHWANTH KUMAR)" <JESHWANTHKUMAR.NK@xxxxxxx>]

[AMD Official Use Only - General]

Missed to update the Flag details:

Flag for DMA Mapped VA - 0x0C0644BB
Flag for Local VA          -  0x08100073


VM_IO and VM_PFNMAP  - Set in DMA mapped VA but not in local VA.

Regards,
Jeshwanth

-----Original Message-----
From: NK, JESHWANTHKUMAR (JESHWANTH KUMAR) 
Sent: Tuesday, September 13, 2022 11:05 AM
To: 'Stefano Stabellini' <stefano.stabellini@xxxxxxx>
Cc: Stabellini, Stefano <stefano.stabellini@xxxxxxx>; 
boris.ostrovsky@xxxxxxxxxx; xen-devel@xxxxxxxxxxxxxxxxxxxx; Rangasamy, Devaraj 
<Devaraj.Rangasamy@xxxxxxx>; Pandeshwara krishna, Mythri 
<Mythri.Pandeshwarakrishna@xxxxxxx>; SK, SivaSangeetha (Siva Sangeetha) 
<SivaSangeetha.SK@xxxxxxx>; Thomas, Rijo-john <Rijo-john.Thomas@xxxxxxx>; 
jgross@xxxxxxxx
Subject: RE: Linux pin_user_pages_fast fails on Xen

[AMD Official Use Only - General]

Hi Stefano,

https://elixir.bootlin.com/linux/v5.16/source/mm/gup.c#L975 is the -EFAULT 
returning for our current use case.

access_ok is fine.

Regards,
Jeshwanth

-----Original Message-----
From: Stefano Stabellini <stefano.stabellini@xxxxxxx>
Sent: Tuesday, September 13, 2022 6:56 AM
To: NK, JESHWANTHKUMAR (JESHWANTH KUMAR) <JESHWANTHKUMAR.NK@xxxxxxx>
Cc: Stabellini, Stefano <stefano.stabellini@xxxxxxx>; 
boris.ostrovsky@xxxxxxxxxx; xen-devel@xxxxxxxxxxxxxxxxxxxx; NK, JESHWANTHKUMAR 
(JESHWANTH KUMAR) <JESHWANTHKUMAR.NK@xxxxxxx>; Rangasamy, Devaraj 
<Devaraj.Rangasamy@xxxxxxx>; Pandeshwara krishna, Mythri 
<Mythri.Pandeshwarakrishna@xxxxxxx>; SK, SivaSangeetha (Siva Sangeetha) 
<SivaSangeetha.SK@xxxxxxx>; Thomas, Rijo-john <Rijo-john.Thomas@xxxxxxx>; 
jgross@xxxxxxxx
Subject: Re: Linux pin_user_pages_fast fails on Xen

On Sat, 10 Sep 2022, Juergen Gross wrote:
> On 09.09.22 22:25, Stefano Stabellini wrote:
> > On Fri, 9 Sep 2022, Juergen Gross wrote:
> > > On 09.09.22 04:11, Stefano Stabellini wrote:
> > > > Adding more people in CC
> > > > 
> > > > On Thu, 8 Sep 2022, Stefano Stabellini wrote:
> > > > > Hi Juergen,
> > > > > 
> > > > > A colleague is seeing a failure on x86 in Linux Dom0. The 
> > > > > failure is pin_user_pages_fast with addresses that correspond 
> > > > > to foreign memory
> > > > > pages:
> > > > > 
> > > > > - QEMU maps a domU address using dma_memory_map
> > > > > (xen_map_cache)
> > > > > - QEMU calls an IOCTL to the TEE subsystem with the Virtual Address
> > > > >     returned by dma_memory_map
> > > > > - Linux tee_shm_register->pin_user_pages_fast Returns -14 - 
> > > > > drivers/tee/tee_shm.c
> > > > > 
> > > > > Once upon a time it used to be the case that 
> > > > > get_user_pages_fast would fail on Xen because we didn't have a 
> > > > > struct page corresponding to foreign memory mappings. But that hasn't 
> > > > > been the case for years now.
> > > > > 
> > > > > Any other ideas why it would fail?
> > > 
> > > I think we can expect that access_ok() isn't failing.
> > > 
> > > I assume the mapping was done allowing writes (sorry for paranoia mode)?
> >   I was told it was verified: QEMU could read and write to the VA 
> > returned by dma_memory_map. From /proc/<qemu-pid>/maps, the VA 
> > assigned after the mapping is pointing to /dev/xen/privcmd.
> > 
> > 
> > > Other than that I'm not having enough memory management skills. It 
> > > might be related to mmap()-ed foreign pages having _PAGE_SPECIAL 
> > > set, though.
> > 
> > Do we still set PAGE_SPECIAL for foreign mapped pages? It looks like 
> > it is not there anymore? If PAGE_SPECIAL is not there, then they 
> > really should look like regular pages?
> 
> See the call of pte_mkspecial() in remap_area_pfn_pte_fn() (mmu_pv.c).

The kernel version is 5.16 and the return code is -EFAULT. Is it the following 
-EFAULT the one that triggers?

mm/gup.c:internal_get_user_pages_fast:

        if (unlikely(!access_ok((void __user *)start, len)))
                return -EFAULT;