[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCHv3] QEMU(upstream): Disable xen's use of O_DIRECT by default as it results in crashes.


--On 18 March 2013 15:49:48 +0100 Paolo Bonzini <pbonzini@xxxxxxxxxx> wrote:

The problem is that the target's page cache might host image data from a
previous run.  If you do not use O_DIRECT, it will not see the changes
made on the source.

I was under the impression that with cache=writeback, qemu doesn't
use O_DIRECT, in which case why isn't there the same danger under
kvm, i.e. that the target page cache contains data from a previous

KVM in fact only supports migration using cache=none.  This does not
apply of course if you're using cache-coherent storage, such as rbd or
gluster; or if you're using one of the userspace backends that bypass
the page cache, like NBD or libiscsi.

Well right now emulated devices under xen with qemu-upstream
dm do not use O_DIRECT. So that's going to hit this problem anyway.

Let me understand the problem a little more. Let's say a VM has a
disk backed by NFS. It runs on node A, at which point pages are
introduced to the page cache. It then migrates to node B, with
node A flushing its dirty pages to disk before the migration
completed, but not emptying the page cache completely.
It's then migrated back to node A in which case I think you are
saying that node A may still hold the page cache entries from
prior to the first migration.

However, node A has closed and reopened the file. Doesn't NFSv4
notice that the file has changed at this point and invalidate
all page cache entries? Else (re)opening a file on one NFS client
that has been changed on another would never work. I know this
is unpredictable if you haven't closed the file, but in this
instance A has closed the file.

Disabling migration seems a bit excessive when migration isn't disabled
with cache=unsafe (AFAIK)

It is not QEMU's task.  There are cases where the cache= options are
unnecessary or ignored.  But indeed libvirt warns (or blocks, I don't
remember) in this case.

Well we are kvm user too, and I understand hat kvm (as opposed to
libvirt) does not error or warn if you live migrate with cache=writeback.

I've no problem if xl or libvirt or whatever error or warn. My usage
is API based, rather than xl / libvirt based.

If libxl does migration without O_DIRECT, then that's a bug in libxl.
What about blkback?  IIRC it uses bios, so it also bypasses the page

Possibly a bug in xl rather than libxl, but as no emulated devices
use O_DIRECT, that bug is already there, and isn't in QEMU.

Stefano did ack the patch, and for a one line change it's been
through a pretty extensive discussion on xen-devel ...

It may be a one-line change, but it completely changes the paths that
I/O goes through.  Apparently the discussion was not enough.

What would you suggest?

Alex Bligh

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.