Re: Design session notes: GPU acceleration in Xen

[Jan Beulich <jbeulich@xxxxxxxx>]

On 17.06.2024 17:17, Demi Marie Obenour wrote:
> On Mon, Jun 17, 2024 at 11:07:54AM +0200, Jan Beulich wrote:
>> On 14.06.2024 18:44, Demi Marie Obenour wrote:
>>> On Fri, Jun 14, 2024 at 10:12:40AM +0200, Jan Beulich wrote:
>>>> On 14.06.2024 09:21, Roger Pau Monné wrote:
>>>>> I'm not sure it's possible to ensure that when using system RAM such
>>>>> memory comes from the guest rather than the host, as it would likely
>>>>> require some very intrusive hooks into the kernel logic, and
>>>>> negotiation with the guest to allocate the requested amount of
>>>>> memory and hand it over to dom0.  If the maximum size of the buffer is
>>>>> known in advance maybe dom0 can negotiate with the guest to allocate
>>>>> such a region and grant it access to dom0 at driver attachment time.
>>>>
>>>> Besides the thought of transiently converting RAM to kind-of-MMIO, this
>>>> makes me think of another possible option: Could Dom0 transfer ownership
>>>> of the RAM that wants mapping in the guest (remotely resembling
>>>> grant-transfer)? Would require the guest to have ballooned down enough
>>>> first, of course. (In both cases it would certainly need working out how
>>>> the conversion / transfer back could be made work safely and reasonably
>>>> cleanly.)
>>>
>>> The kernel driver needs to be able to reclaim the memory at any time.
>>> My understanding is that this is used to migrate memory between VRAM and
>>> system RAM.  It might also be used for other purposes.
>>
>> Except: How would the kernel driver reclaim the memory when it's mapped
>> by a DomU?
> 
> The Xen driver in dom0 will register for MMU notifier callbacks.  When
> the kernel driver reclaims the memory, the Xen driver will be notified,
> and it will issue a hypercall that tells Xen to remove the memory from
> the DomU's address space.  Subsequent accesses to the pages will trigger
> a stage 2 translation fault that is handled by an IOREQ server.

And such an ioreq server, which I assume isn't going to run in the Dom0
kernel, will then also need keeping up-to-date on holes in the (virtual)
BAR. Oh well ...

Jan