Re: [Xen-devel] questions of vm save/restore on arm64

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Fri, 3 Jun 2016, Chenxiao Zhao wrote:
> On 6/3/2016 4:02 AM, Julien Grall wrote:
> > Hello,
> > 
> > First thing, the time in the mail headers seems to be wrong. Maybe
> > because of a wrong timezone?
> > 
> > I got: 04/06/16 02:32 however we are still the 3rd in my timezone.
> > 
> > On 04/06/16 02:32, Chenxiao Zhao wrote:
> > > 
> > > 
> > > On 6/3/2016 3:16 AM, Julien Grall wrote:
> > > > Hello,
> > > > 
> > > > On 03/06/16 18:05, Chenxiao Zhao wrote:
> > > > > I finally found out that the problem is that the toolstack did not get
> > > > > corret p2m_size while sending all pages on save(always be zero).
> > > > > After I
> > > > > fixed that, the guest could be restored but guest kernel caught
> > > > > handle_mm_fault().
> > > > > 
> > > > > where do you think I'm going to investigate, guest kernel hibernation
> > > > > restore or xen?
> > > > 
> > > > The hibernation support for ARM64 has only been merged recently in the
> > > > kernel. Which kernel are you using?
> > > 
> > > Hi Julien,
> > > 
> > > I'm using a linaro ported Linux kernel 4.1 for hikey from this link.
> > > 
> > > https://github.com/96boards/linux/tree/android-hikey-linaro-4.1
> > > 
> > > I also applied following patches to make the kernel support hibernation.
> > 
> > This looks the wrong way to do it as this series may requires some
> > patches which have been upstreamed before hand.
> > 
> > Linux upstream seems support to the hikey board [1]. Any reason to not
> > using it?
> 
> I tried a newer version of kernel 4.4, but got no luck to start dom0 with xen.
> so I decide to stay in 4.1 for now.
> 
> > 
> > > [1] http://www.spinics.net/lists/arm-kernel/msg477769.html
> > > [2] http://lists.xen.org/archives/html/xen-devel/2015-12/msg01068.html
> > > 
> > > > 
> > > > Also, what are the modifications you have made to support Xen
> > > > suspend/resume for ARM64?
> > > 
> > > I believe I have posted my modifications on xen in the first mail of
> > > this thread.
> > 
> > I mean in Linux. The patch from Ian Campbell does not have any kind of
> > support for ARM64.
> > 
> > For instance arch/arm/xen/suspend.c needs to be built for ARM64. So I am
> > wondering if your kernel has support of hibernation...
> 
> Oh, yes, I most forgot I added this file in arch/arm64/xen/Makefile to let it
> build for arm64.
> > 
> > > 
> > >  From my understanding, a kernel hibernation will cause kernel to save
> > > memories to disk(swap partition). But on guest save progress, the
> > > hibernation for domU does not make the guest save memories to disk. it's
> > > more like suspend all processes in guest, and memors actually depends on
> > > xen toolstack to save the pages to file. Am I correct?
> > 
> > You are using an older tree with a patch series based on a newer tree.
> > 
> > So I would recommend you to move to a newer tree. If it is not possible,
> > please test that hibernation works on baremetal.
> 
> I think the suspend/resume in guest is working, cause I can use pause/unpause
> command in toolstack to suspend/resume guest without problem. I can also see
> the suspend/resume kernel messages from guest's console. The only problem is
> it's can not resume from restore.

But can you still connect to the guest after resume, maybe over the network?
If you cannot, then something is likely wrong.


> One thing that confused me is that the kernel's hibernation means the guest
> kernel will save the memory state to disk and power off VM at last. The guest
> will also take care of the memory restore itself. But I do not see the
> save/restore on xen works that way. So my question is why it requires
> hibernation (aka. suspend to disk) instead of the real suspend (aka. suspend
> to RAM and standby)?

Xen suspend/resume has nothing to do with guest suspend to RAM or guest
hibernation.

Xen suspend/resume is a way for the hypervisor to save to file the
entire state of the VM, including RAM and the state of any devices.
Guest suspend to RAM and guest hibernation are two guest driven
technologies to save the state of the operating system to RAM or to
disk. The only link between Xen suspend and guest suspend is that when
Xen issues a domain suspend, it notifies the guest of it so that it can
ease the process.  The code in Linux to support Xen suspend/resume is:

drivers/xen/manage.c:do_suspend

and makes use of some of the Linux internal hooks provided for
hibernations (see CONFIG_HIBERNATE_CALLBACKS). But that's just for
better integration with the rest of Linux: hibernation is NOT what is
happening.

I hope that this clarifies things a bit, I realize that it is confusing.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel