Re: [Xen-API] Storage Repository FileSR on ZFS won't work - Linux

[Dawid Kowalski <dkadds2@xxxxxxxxx>]

Hi Carlos,

Thanks for hint!
This one worked.

So what worked:
zpool -> zfs block device fomatted as ext4 and mounted as folder
storage repository created with ffs driver.
The other drivers, meaning :
- file, fileSR on mounted ext4 FS,
- ext/lvm pointed to /dev/zd0
- ffs directly on ZFS.

Failed.

Other last resort option was to create file on ZFS, losetup and create VG on it.

FFS directly on ZFS failed most probably due to unsupported O_DIRECT flag (err=22) and other failed most probably for similar reasons.

If anyone would have some questions, feel free to email me directly at love-spam-remove-me-up-to-here-dkadds2+xen-api@xxxxxxxxx

Based on my experience, there seem to be a lot to be fixed on xen side to be really user friendly/usable.

Based on ZFS list of features there should be direct connector from Xen side.

Regards,
Dawid

On 30/12/14 20:04, Carlos Reategui wrote:

Have you tried using ext4 on a ZVOL and putting your VHDs there?

Scroll down to the "Ext4 on a ZVOL" section:Âhttps://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/

On Tue, Dec 30, 2014 at 9:45 AM, Dawid Kowalski <dkadds2@xxxxxxxxx> wrote:

Just as a form of an update. I'm stuck trying to get NFS working as no other way to use ZFS in background seems to be available for me. iSCSI share option for ZFS is not available on Linux. Should I try to enable it I'd need to add third-party daemon.

I'm looking forward for your help in NFS context.

In regards to nosync - I'm happy to give it a try as it's just a test system now but I'd prefer to find working solution.

I've spent couple of days/evenings already fighting to get Xen working on Ubuntu with ZFS and I'm close to give up and try KVM. It's way to complicated.

I'll still keep trying...

Thanks,
Dawid

On 30/12/14 18:40, Dave Scott wrote:

On 30 Dec 2014, at 17:24, Dawid Kowalski <dkadds2@xxxxxxxxx> wrote:

I'll answer myself.

Modifying td.c line 258 to skip O_DIRECT and recompiling doesn't help.
For some unexplained reasons, it calls O_DIRECT flag still:

Hm. As an experiment you could try an LD_PRELOAD wrapper like this:

http://www.mcgill.org.za/stuff/software/nosync

Before using anything like this for data you care about, we need to think it through a little to make sure itâs safe. Perhaps Linux zfs support for iSCSI is safer? (Sorry to flip-flop on this, Iâm not very familiar with ZFS)

Cheers,
Dave

strace ./td-util create vhd 2048 /path/to/file
open("/path/to/file", O_WRONLY|O_CREAT|O_TRUNC|O_DIRECT, 0644) = -1 EINVAL (Invalid argument)
exit_group(22)Â Â Â Â Â Â Â Â Â Â Â Â Â = ?
+++ exited with 22 +++

Checking other files with O_WRONLY flag doesn't bring any ideas as none of them use "O_DIRECT".

lock.c:Â Â Â Â fd = open(buf, O_WRONLY | O_CREAT, 0644);
lock.c:Â Â Â Â fd = open(lockfn, O_WRONLY | O_CREAT | O_EXCL, 0644);
lock.c:Â Â Â Â Â Â Â Â fd = open(lockfn_flink, O_WRONLY | O_CREAT, 0644);
tapdisk-vbd.c:Â Â fd = open(fn, O_WRONLY | O_CREAT | O_NONBLOCK, 0666);
td.c:Â Â fd = open(name, O_WRONLY | O_CREAT | O_TRUNC, 0644);


Any ideas how to fix FileSR for ZFS on Linux?

Seems like I'm left with iScsi approach only.

Thanks,
Dawid

On 30/12/14 18:11, Dawid Kowalski wrote:

Hi Guys,

Just wanted to report that file based Storage Repository won't work on Linux systems as td-util is required O_DIRECT when creating image file not supported on ZFS (Linux).

When creating vm, i.e. vm-import it calls td-util similar to below:
td-util create vhd 2048 /path/to/file

Strace shows:
open("/path/to/file", O_WRONLY|O_CREAT|O_TRUNC|O_DIRECT, 0644) = -1 EINVAL (Invalid argument)

I'll try to patch td.c to skip O_DIRECT flag but this is rather dirty hack than proper solution of the problem.

Regards,
Dawid

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api