[Xen-devel] Create a iSCSI DomU with disks in another DomU running on the same Dom0

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

Hello,

I'm trying to use a strange setup, that consists in having a DomU
serving iSCSI targets to the Dom0, that will use this targets as disks
for other DomUs. I've tried to set up this iSCSI target DomU using both
Debian Squeeze/Wheezy (with kernels 2.6.32 and 3.2) and ISCSI
Enterprise Target (IET), and when launching the DomU I get this messages
from Xen:

(XEN) mm.c:1925:d0 Error pfn 157e68: rd=ffff83019e60c000, od=ffff830141405000, 
caf=8000000000000003, taf=7400000000000001
(XEN) Xen WARN at mm.c:1926
(XEN) ----[ Xen-4.3-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c48016ea17>] get_page+0xd5/0x101
(XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: ffff830141405000   rcx: 0000000000000000
(XEN) rdx: ffff82c480300920   rsi: 000000000000000a   rdi: ffff82c4802766e8
(XEN) rbp: ffff82c4802bfbf8   rsp: ffff82c4802bfba8   r8:  0000000000000004
(XEN) r9:  0000000000000004   r10: 0000000000000004   r11: 0000000000000001
(XEN) r12: 0000000000157e68   r13: ffff83019e60c000   r14: 7400000000000001
(XEN) r15: 8000000000000003   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 000000011c180000   cr2: 00007f668d1eb000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c4802bfba8:
(XEN)    ffff830141405000 8000000000000003 7400000000000001 0000000000145028
(XEN)    ffff82f6028a0510 ffff83019e60c000 ffff82f602afcd00 ffff82c4802bfd28
(XEN)    ffff82c4802bfd18 0000000000157e68 ffff82c4802bfc58 ffff82c480109ba3
(XEN)    ffffffffffffffff 0000000000000000 ffff83011c977fb8 0000000061dfc3f0
(XEN)    0000000000000001 ffffffffffff8000 0000000000000002 ffff83011d555000
(XEN)    ffff83019e60c000 0000000000000000 ffff82c4802bfd98 ffff82c48010c607
(XEN)    ffff82c4802bfd34 ffff82c4802bfd30 ffff82c400000001 000000000011cf90
(XEN)    0000000000000000 ffff82c4802b8000 ffff82c4802b8000 ffff82c4802b8000
(XEN)    ffff82c4802b8000 ffff82c4802bfd5c 000000029e60c000 ffff82c480300920
(XEN)    ffff82c4802b8000 ffff82c4802bfd38 00000005802bfd38 ffff82c4802b8000
(XEN)    ffff82c400000000 0000000000000001 ffffc90000028b10 ffffc90000028b10
(XEN)    ffff8300dfb03000 0000000000000000 0000000000000000 0000000000145028
(XEN)    000000000011cf7c 0000000000001000 0000000000157e68 0000000000007ff0
(XEN)    000000000000027e 000000000042000d 0000000000020b50 ffff8300dfdf0000
(XEN)    ffff82c4802bfd78 ffffc90000028ac0 ffffc90000028ac0 ffff880185f6fd58
(XEN)    ffff880185f6fd78 0000000000000005 ffff82c4802bfef8 ffff82c48010eb65
(XEN)    ffff82c4802bfdc8 ffff82c480300960 ffff82c4802bfe18 ffff82c480181831
(XEN)    000000000006df66 000032cfdc175ce6 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000005 ffff82c4802bfe28 ffff8300dfb03000
(XEN)    ffff8300dfdf0000 0000150e11a417f8 0000000000000002 ffff82c480300948
(XEN) Xen call trace:
(XEN)    [<ffff82c48016ea17>] get_page+0xd5/0x101
(XEN)    [<ffff82c480109ba3>] __get_paged_frame+0xbf/0x162
(XEN)    [<ffff82c48010c607>] gnttab_copy+0x4c6/0x91a
(XEN)    [<ffff82c48010eb65>] do_grant_table_op+0x12ad/0x1b23
(XEN)    [<ffff82c48022280b>] syscall_enter+0xeb/0x145
(XEN)    
(XEN) grant_table.c:2076:d0 source frame ffffffffffffffff invalid.
(XEN) mm.c:1925:d0 Error pfn 157e68: rd=ffff83019e60c000, od=ffff830141405000, 
caf=8000000000000003, taf=7400000000000001
(XEN) Xen WARN at mm.c:1926
(XEN) ----[ Xen-4.3-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c48016ea17>] get_page+0xd5/0x101
(XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: ffff830141405000   rcx: 0000000000000000
(XEN) rdx: ffff82c480300920   rsi: 000000000000000a   rdi: ffff82c4802766e8
(XEN) rbp: ffff82c4802bfbf8   rsp: ffff82c4802bfba8   r8:  0000000000000004
(XEN) r9:  0000000000000004   r10: 0000000000000004   r11: 0000000000000001
(XEN) r12: 0000000000157e68   r13: ffff83019e60c000   r14: 7400000000000001
(XEN) r15: 8000000000000003   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 000000011c180000   cr2: 00007f668d1eb000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c4802bfba8:
(XEN)    ffff830141405000 8000000000000003 7400000000000001 000000000014581d
(XEN)    ffff82f6028b03b0 ffff83019e60c000 ffff82f602afcd00 ffff82c4802bfd28
(XEN)    ffff82c4802bfd18 0000000000157e68 ffff82c4802bfc58 ffff82c480109ba3
(XEN)    ffffffffffffffff 0000000000000000 ffff83011c977fb8 0000000061dfc308
(XEN)    0000000000000000 ffffffffffff8000 0000000000000001 ffff83011d555000
(XEN)    ffff83019e60c000 0000000000000000 ffff82c4802bfd98 ffff82c48010c607
(XEN)    ffff82c4802bfd34 ffff82c4802bfd30 ffff82c400000001 000000000011cf90
(XEN)    0000000000000000 ffff82c4802b8000 ffff82c4802b8000 ffff82c4802b8000
(XEN)    ffff82c4802b8000 ffff82c4802bfd5c 000000029e60c000 ffff82c480300920
(XEN)    ffff82c4802b8000 ffff82c4802bfd38 00000002802bfd38 ffff82c4802b8000
(XEN)    ffffffff00000000 0000000000000001 ffffc90000028b60 ffffc90000028b60
(XEN)    ffff8300dfb03000 0000000000000000 0000000000000000 000000000014581d
(XEN)    00000000000deb3e 0000000000001000 0000000000157e68 000000000b507ff0
(XEN)    0000000000000261 000000000042000d 00000000000204b0 ffffc90000028b38
(XEN)    0000000000000002 ffffc90000028b38 ffffc90000028b38 ffff880185f6fd58
(XEN)    ffff880185f6fd78 0000000000000005 ffff82c4802bfef8 ffff82c48010eb65
(XEN)    ffff82c4802bfdc8 ffff82c480300960 ffff82c4802bfe18 ffff82c480181831
(XEN)    000000000006df66 000032cfdc175ce6 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000005 ffff82c4802bfe28 0000000000000086
(XEN)    ffff82c4802bfe28 ffff82c480125eae ffff83019e60c000 0000000000000286
(XEN) Xen call trace:
(XEN)    [<ffff82c48016ea17>] get_page+0xd5/0x101
(XEN)    [<ffff82c480109ba3>] __get_paged_frame+0xbf/0x162
(XEN)    [<ffff82c48010c607>] gnttab_copy+0x4c6/0x91a
(XEN)    [<ffff82c48010eb65>] do_grant_table_op+0x12ad/0x1b23
(XEN)    [<ffff82c48022280b>] syscall_enter+0xeb/0x145
(XEN)    
(XEN) grant_table.c:2076:d0 source frame ffffffffffffffff invalid.

(Note that I've added a WARN() to mm.c:1925 to see where the
get_page call was coming from).

Connecting the iSCSI disks to another Dom0 works fine, so this
problem only happens when trying to connect the disks to the
Dom0 where the DomU is running.

I've replaced the Linux DomU serving iSCSI targets with a
NetBSD DomU, and the problems disappears, and I'm able to
attach the targets shared by the DomU to the Dom0 without
issues.

The problem seems to come from netfront/netback, does anyone
have a clue about what might cause this bad interaction
between IET and netfront/netback?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel