Re: [QEMU PATCH 1/1] virtgpu: do not destroy resources when guest suspend

[Robert Beckett <bob.beckett@xxxxxxxxxxxxx>]

On 21/06/2023 09:39, Gerd Hoffmann wrote:
> On Tue, Jun 20, 2023 at 01:26:15PM +0100, Robert Beckett wrote:
> > On 20/06/2023 10:41, Gerd Hoffmann wrote:
> > >     Hi,
> > >
> > > > > The guest driver should be able to restore resources after resume.
> > > > Thank you for your suggestion!
> > > > As far as I know, resources are created on host side and guest has no backup, 
> > > > if resources are destroyed, guest can't restore them.
> > > > Or do you mean guest driver need to send commands to re-create resources after 
> > > > resume?
> > > The later.  The guest driver knows which resources it has created,
> > > it can restore them after suspend.
> > Are you sure that this is viable?
> >
> > How would you propose that a guest kernel could reproduce a resource,
> > including pixel data upload during a resume?
> >
> > The kernel would not have any of the pixel data to transfer to host.
> Depends on the of resource type.  For resources which are created by
> uploading pixel data (using VIRTIO_GPU_CMD_TRANSFER_TO_HOST_*) a guest
> mirror exists which can be used for re-upload.
unfortunately this is not always the case.

https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/gallium/drivers/virgl/virgl_resource.c#L668

Often mesa will decide that it won't need to access a resource again 
after initial upload (textures etc). In this case, if it is able to copy 
back from host if needed, it will not maintain the guest shadow copy. 
Instead it will create a single page proxy object. The transfer to host 
will then over fill it to the correct size.
I think this was a fairly huge optimization for them.

> For resources filled by gl rendering ops this is indeed not the case.
>
> > Could you explain how you anticipate the guest being able to reproduce the
> > resources please?
> Same you do on physical hardware?  Suspend can poweroff your PCI
> devices, so there must be some standard way to handle that situation
> for resources stored in gpu device memory, which is very similar to
> the problem we have here.
In traditional PCI gfx card setups, TTM is used as the memory manager in 
the kernel. This is used to migrate the buffers back from VRAM to system 
pages during a suspend.
This would be suitable for use to track host blob buffers that get 
mapped to guest via the PCI BAR, though would be a significant 
re-architecting of virtio gpu driver.
It would not help with the previously mentioned proxied resources. 
Though in theory the driver could read the resources back from host to 
guest pages during suspend, this would then be potentially complicated 
by suspend time alloc failures etc.

As virtio drivers are by design paravirt drivers ,I think it is 
reasonable to accept some knowledge with and cooperation with the host 
to manage suspend/resume.
It seems to me like a lot of effort and long term maintenance to add 
support for transparent suspend/resume that would otherwise be unneeded.
Perhaps others have alternative designs for this?

> take care,
>    Gerd