Re: [Xen-devel] [Qemu-devel] [PATCH V2 5/5] vga-cirrus: Workaround during restore when using Xen.

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Wed, 4 Jan 2012, Avi Kivity wrote:
> On 01/04/2012 06:38 PM, Stefano Stabellini wrote:
> >
> > > I suggest doing the following:
> > > 
> > > 1. keep cirrus code unchanged
> > > 2. when the framebuffer is first mapped into physical memory (as known
> > > by your CPUPhysMemoryClient), copy it into a temporary buffer, map the
> > > guest memory into memory_region_get_ram_ptr(), and copy the temporary
> > > buffer into memory_region_get_ram_ptr()
> > > 3. when the framebuffer is unmapped, do the reverse: copy the
> > > framebuffer out, mmap() some anonymous memory into
> > > memory_region_get_ram_ptr(), and copy the temporary buffer into
> > > memory_region_get_ram_ptr()
> >
> > I cannot see how this is going to fix the save/restore issue we are
> > trying to solve.
> > The problem, unfortunately very complex, is that at restore time the
> > videoram is already allocated at the physical address it was mapped
> > before the save operation. If it was not mapped, it is at the end of the
> > physical memory of the guest (where qemu_ram_alloc_from_ptr decides to
> > allocate it).
> 
> Sorry, I don't follow, please be specific as to which type of address
> you're referring to:
> 
> ram_addr?
> physical address (as seen by guest - but if it is not mapped, what does
> your last sentence mean?)
> something else?

ram_addr_t as returned by qemu_ram_alloc_from_ptr.

In fact on xen qemu_ram_alloc_from_ptr asks the hypervisor to add
the specified amount of memory to the guest physmap at
new_block->offset. So in a way the videoram is always visible by the
guest, initially at new_block->offset, chosen by find_ram_offset, then
at cirrus_bank_base, when map_linear_vram_bank is called.


> > So the issue is that the videoram appears to qemu as part of the
> > physical memory of the guest at an unknown address.
> >
> > The proposal of introducing early_savevm would easily solve this last
> > problem: letting us know where the videoram is. The other problem, the
> > fact that under Xen the videoram would be already allocated while under
> > native it would not, remains unsolved. 
> > We cannot simply allocate the videoram twice because the operation
> > would fail (Xen would realize that we are trying to allocate more memory
> > than it we are supposed to, returning an error).
> > However, once we know where the videoram is, we could probably figure out
> > a smart (see hacky) way to avoid allocating it twice without changes to
> > the cirrus code.
> 
> I'm missing some context.  Can you please explain in more detail?
> Note that with the memory API changes, ram addresses are going away. 
> There will not be a linear space for guest RAM.  We'll have
> (MemoryRegion *, offset) pairs that will be mapped into discontiguous
> guest physical address ranges (perhaps with overlaps).


This is how memory is currently allocated and mapped in qemu on xen:

- qemu_ram_alloc_from_ptr asks the hypervisor to allocate memory for
the guest, the memory is added to the guest p2m (physical to machine
translation table) at the given guest physical address
(new_block->offset, as chosen by find_ram_offset);

- qemu_get_ram_ptr uses the xen mapcache to map guest physical address
ranges into qemu's address space, so that qemu can read/write guest
memory;

- xen_set_memory, called by the memory_listener interface, effectively
moves a guest physical memory address range from one address to another.
So the memory that was initially allocated at new_block->offset, as
chosen by find_ram_offset, is going to be moved to a new destination,
section->offset_within_address_space.


So the videoram lifecycle is the following:

- qemu_ram_alloc_from_ptr allocates the videoram and adds it to the end
  of the physmap;

- qemu_get_ram_ptr maps the videoram into qemu's address space;

- xen_set_memory moves the videoram to cirrus_bank_base;



Now let's introduce save/restore into the picture: the videoram is part
of the guest's memory from the hypervisor POV, so xen will take care of
saving and restoring it as part of the normal guest memory, out of
qemu's control.
At restore time, we know that the videoram is already allocated and
mapped somewhere in the guest physical address space: it could be
cirrus_bank_base, which we don't know yet, or the initial
new_block->offset.
A second videoram allocation by qemu_ram_alloc_from_ptr will fail
because of memory constraints enforced by the hypervisor. Trying to map
the already allocated videoram into qemu's address space is not easy
because we don't know where it is yet (keep in mind that machine->init
is called before the machine restore functions).

The "solution" I am proposing is introducing an early_savevm set of
save/restore functions so that at restore time we can get to know at
what address the videoram is mapped into the guest address space. Once we
know the address we can remap it into qemu's address space and/or move it
to another guest physical address.

The problem of avoiding a second allocation remains, but could be
solved by passing the "name" parameter from qemu_ram_alloc_from_ptr to
xen_ram_alloc: xen_ram_alloc could avoid doing any work for anything
called "vga.vram" at restore time, and use the reference to the already
allocated videoram instead.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel