[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [RFC PATCH V1 04/12] xen/arm: Introduce arch specific bits for IOREQ/DM features


On 13.08.20 23:36, Julien Grall wrote:

Hi Julien




On 13/08/2020 19:41, Oleksandr wrote:

Rebooting domain 2
root@generic-armv8-xt-dom0:~# (XEN) Xen BUG at ...tAUTOINC+bb71237a55-r0/git/xen/include/xen/mm.h:683 
(XEN) ----[ Xen-4.14.0  arm64  debug=y   Not tainted ]----
(XEN) CPU:    3
(XEN) PC:     0000000000246f28 ioreq.c#hvm_free_ioreq_mfn+0x68/0x6c
(XEN) LR:     0000000000246ef0
(XEN) SP:     0000800725eafd80
(XEN) CPSR:   60000249 MODE:64-bit EL2h (Hypervisor, handler)
(XEN)      X0: 0000000000000001  X1: 403fffffffffffff  X2: 000000000000001f (XEN)      X3: 0000000080000000  X4: 0000000000000000  X5: 0000000000400000 (XEN)      X6: 0000800725eafe24  X7: 0000ffffd1ef3e08  X8: 0000000000000020 (XEN)      X9: 0000000000000000 X10: 00e800008ecebf53 X11: 0400000000000000 (XEN)     X12: ffff7e00013b3ac0 X13: 0000000000000002 X14: 0000000000000001 (XEN)     X15: 0000000000000001 X16: 0000000000000029 X17: 0000ffff9badb3d0 (XEN)     X18: 000000000000010f X19: 0000000810e60e38 X20: 0000800725e68ec0 (XEN)     X21: 0000000000000000 X22: 00008004dc0404a0 X23: 000000005a000ea1 (XEN)     X24: ffff8000460ec280 X25: 0000000000000124 X26: 000000000000001d (XEN)     X27: ffff000008ad1000 X28: ffff800052e65100  FP: ffff0000223dbd20 
(XEN)
(XEN)   VTCR_EL2: 80023558
(XEN)  VTTBR_EL2: 0002000765f04000
(XEN)
(XEN)  SCTLR_EL2: 30cd183d
(XEN)    HCR_EL2: 000000008078663f
(XEN)  TTBR0_EL2: 00000000781c5000
(XEN)
(XEN)    ESR_EL2: f2000001
(XEN)  HPFAR_EL2: 0000000000030010
(XEN)    FAR_EL2: ffff000008005f00
(XEN)
(XEN) Xen stack trace from sp=0000800725eafd80:
(XEN)    0000800725e68ec0 0000000000247078 00008004dc040000 00000000002477c8 (XEN)    ffffffffffffffea 0000000000000001 ffff8000460ec500 0000000000000002 (XEN)    000000000024645c 00000000002462dc 0000800725eafeb0 0000800725eafeb0 (XEN)    0000800725eaff30 0000000060000145 000000000027882c 0000800725eafeb0 (XEN)    0000800725eafeb0 01ff00000935de80 00008004dc040000 0000000000000006 (XEN)    ffff800000000000 0000000000000002 000000005a000ea1 000000019bc60002 (XEN)    0000ffffd1ef3e08 0000000000000020 0000000000000004 000000000027c7d8 (XEN)    000000005a000ea1 0000800725eafeb0 000000005a000ea1 0000000000279f98 (XEN)    0000000000000000 ffff8000460ec200 0000800725eaffb8 0000000000262c58 (XEN)    0000000000262c4c 07e0000160000249 0000000000000002 0000000000000001 (XEN)    ffff8000460ec500 ffff8000460ec508 ffff8000460ec208 ffff800052e65100 (XEN)    000000005060b478 0000ffffd20f3000 ffff7e00013c77e0 0000000000000000 (XEN)    00e800008ecebf53 0400000000000000 ffff7e00013b3ac0 0000000000000002 (XEN)    0000000000000001 0000000000000001 0000000000000029 0000ffff9badb3d0 (XEN)    000000000000010f ffff8000460ec210 ffff8000460ec200 ffff8000460ec210 (XEN)    0000000000000001 ffff8000460ec500 ffff8000460ec280 0000000000000124 (XEN)    000000000000001d ffff000008ad1000 ffff800052e65100 ffff0000223dbd20 (XEN)    ffff000008537004 ffffffffffffffff ffff0000080c17e4 5a000ea160000145 (XEN)    0000000060000000 0000000000000000 0000000000000000 ffff800052e65100 (XEN)    ffff0000223dbd20 0000ffff9badb3dc 0000000000000000 0000000000000000 
(XEN) Xen call trace:
(XEN)    [<0000000000246f28>] ioreq.c#hvm_free_ioreq_mfn+0x68/0x6c (PC)
(XEN)    [<0000000000246ef0>] ioreq.c#hvm_free_ioreq_mfn+0x30/0x6c (LR)
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 3:
(XEN) Xen BUG at ...tAUTOINC+bb71237a55-r0/git/xen/include/xen/mm.h:683
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) PSCI cpu off failed for CPU0 err=-3
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
Either I did something wrong (most likely) or there is an issue with page ref-counting in the IOREQ code. I am still trying to understand what is going on.
At a first glance, the implement of set_foreign_p2m_entry() looks fine to me. 


Some notes on that:
1. I checked that put_page() was called for these pages in p2m_put_l3_page() when destroying domain. This happened before hvm_free_ioreq_mfn() execution. 2. There was no BUG detected if I passed "p2m_ram_rw" instead of "p2m_map_foreign_rw" in guest_physmap_add_entry(), but the DomU couldn't be fully destroyed because of the reference taken.
This definitely looks like a page reference issue. Would it be possible to print where the page reference are dropped? A WARN() in put_page() would help.
To avoid a lot of message, I tend to use a global variable that store the page I want to watch.
Unfortunately it is unclear from the log who calls put_page() two times. One of the call is made by p2m_put_l3_page() I assume, but who makes a second call? Needs debugging. 


Rebooting domain 2

root@generic-armv8-xt-dom0:~#

(XEN) put_page[1553] 0000000810e60e38 ---> ref = 3

(XEN) Xen WARN at mm.c:1554
(XEN) ----[ Xen-4.14.0  arm64  debug=y   Not tainted ]----
(XEN) CPU:    2
(XEN) PC:     0000000000272a48 put_page+0xa0/0xc4
(XEN) LR:     0000000000272a48
(XEN) SP:     0000800725eaf990
(XEN) CPSR:   80000249 MODE:64-bit EL2h (Hypervisor, handler)
(XEN)      X0: 0000000000310028  X1: 0000000000000001  X2: 0000800725ca4000
(XEN)      X3: 0000000000000020  X4: 0000000000000000  X5: 0000000000000020
(XEN)      X6: 0080808080808080  X7: fefefefefefeff09  X8: 7f7f7f7f7f7f7f7f
(XEN)      X9: 756e6c64513d313b X10: 7f7f7f7f7f7f7f7f X11: 0101010101010101
(XEN)     X12: 0000000000000008 X13: 000000000028b7d0 X14: 0000800725eaf6e8
(XEN)     X15: 0000000000000020 X16: 0000000000000000 X17: 0000ffffb5eaf070
(XEN)     X18: 000000000000010f X19: 8040000000000003 X20: 0000000810e60e38
(XEN)     X21: 0000800725f07208 X22: 0000800725f07208 X23: 000000000051c041
(XEN)     X24: 0000000000000001 X25: 0000800725eafa78 X26: 000000000009c041
(XEN)     X27: 0000000000000000 X28: 0000000000000007  FP: ffff00002212bd50
(XEN)
(XEN)   VTCR_EL2: 80023558
(XEN)  VTTBR_EL2: 0002000765f04000
(XEN)
(XEN)  SCTLR_EL2: 30cd183d
(XEN)    HCR_EL2: 000000008078663f
(XEN)  TTBR0_EL2: 00000000781c5000
(XEN)
(XEN)    ESR_EL2: f2000001
(XEN)  HPFAR_EL2: 0000000000030010
(XEN)    FAR_EL2: ffff000008005f00
(XEN)
(XEN) Xen stack trace from sp=0000800725eaf990:
(XEN)    034000051c0417ff 0000000000000000 00000000002743e8 0000800725f07208
(XEN)    034000051c0417ff 0000000000000000 0000800725e6d208 0000800725f07208
(XEN)    0000000000000003 0000000000274910 0000000000000000 ffffffffffffffff
(XEN)    0000000000000001 000000000009c041 0000800725f07208 0000000000000000
(XEN)    0000000000000012 0000000000000007 0000000000000001 0000000000000009
(XEN)    0000000000274e00 0000800725f07208 ffffffffffffffff 0000000025eafb0c
(XEN)    0000800725f07000 ffff000008f86528 0000000000000000 0000000200000000
(XEN)    00000041000000e0 0000800725e6d000 0000000000000001 0000800725f07208
(XEN)    000000000009c041 0000000000000000 0000800725f07000 ffff000008f86528
(XEN)    ffff8000501b6540 ffff8000501b6550 ffff8000517bdb40 ffff800052784380
(XEN)    0000000000275770 0000800725eafb08 0000000000000000 0000000025f07000
(XEN)    fffffffffffffffe 0000800725f07000 0000000810e60e38 000000000021a930
(XEN)    0000000000236d18 0000800725f1b990 0000800725eafeb0 0000800725eafeb0
(XEN)    0000800725eaff30 00000000a0000145 000000005a000ea1 ffff000008f86528
(XEN)    0000800725f566a0 00000000002a9088 ffff00000814acc0 0000000a55bb542d
(XEN)    00000000002789d8 0000800725f1b990 00000000002b6cb8 0000800725f03920
(XEN)    00000000002b6cb8 0000000c9084be29 0000000000310228 0000800725eafbf0
(XEN)    ffff00000814f160 0000800725eafbe0 0000000000310228 0000800725eafbf0
(XEN)    0000000000310228 0000000100000002 00000000002b6cb8 0000000000237aa8
(XEN)    0000000000000002 0000000000000000 0000000000000000 0000800725eafc70
(XEN)    0000800725ecd6c8 0000800725ecddf0 00000000000000a0 0000000000000240
(XEN)    000000000026c920 0000800725ecd128 0000000000000002 0000000000000000
(XEN)    0000800725ecd000 0000000000000001 000000000026bfa8 0000000c90bf0b49
(XEN)    0000000000000002 0000000000000000 0000000000276fc4 0000000000000001
(XEN)    0000800725e64000 0000800725e68a60 0000800725e64000 0000800725f566a0
(XEN)    000000000023f53c 000000000027dc14 0000000000311390 0000000000346008
(XEN)    000000000027651c 0000800725eafdd8 0000000000240e44 0000000000000004
(XEN)    0000000000311390 0000000000240e80 0000800725e64000 0000000000240f20
(XEN)    ffff000022127ff0 000000000009c041 000000005a000ea1 ffff800011ae9800
(XEN)    0000000000000124 0000000000240f2c fffffffffffffff2 0000000000000001
(XEN)    0000800725e68ec0 0000800725e68ed0 000000000024694c 00008004dc040000
(XEN)    0000800725e68ec0 00008004dc040000 0000000000247c88 0000000000247c7c
(XEN)    ffffffffffffffea 0000000000000001 ffff800011ae9e80 0000000000000002
(XEN)    000000005a000ea1 0000ffffc52989c0 00000000002463b8 000000000024613c
(XEN)    0000800725eafeb0 0000800725eafeb0 0000800725eaff30 0000000060000145
(XEN)    00000000002789d8 0000800725eafeb0 0000800725eafeb0 01ff00000935de80
(XEN)    0000800725eca000 0000000000310280 0000000000000000 0000000000000000
(XEN)    000000005a000ea1 ffff800011ae9800 0000000000000124 000000000000001d
(XEN)    0000000000000004 000000000027c984 000000005a000ea1 0000800725eafeb0
(XEN)    000000005a000ea1 000000000027a144 0000000000000000 ffff80004e381f80
(XEN) Xen call trace:
(XEN)    [<0000000000272a48>] put_page+0xa0/0xc4 (PC)
(XEN)    [<0000000000272a48>] put_page+0xa0/0xc4 (LR)
(XEN)

(XEN) put_page[1553] 0000000810e60e38 ---> ref = 2

(XEN) Xen WARN at mm.c:1554
(XEN) ----[ Xen-4.14.0  arm64  debug=y   Not tainted ]----
(XEN) CPU:    2
(XEN) PC:     0000000000272a48 put_page+0xa0/0xc4
(XEN) LR:     0000000000272a48
(XEN) SP:     0000800725eafaf0
(XEN) CPSR:   80000249 MODE:64-bit EL2h (Hypervisor, handler)
(XEN)      X0: 0000000000310028  X1: 0000000000000000  X2: 0000800725ca4000
(XEN)      X3: 0000000000000020  X4: 0000000000000000  X5: 0000000000000021
(XEN)      X6: 0080808080808080  X7: fefefefefefeff09  X8: 7f7f7f7f7f7f7f7f
(XEN)      X9: 756e6c64513d313b X10: 7f7f7f7f7f7f7f7f X11: 0101010101010101
(XEN)     X12: 0000000000000008 X13: 000000000028b7d0 X14: 0000800725eaf848
(XEN)     X15: 0000000000000020 X16: 0000000000000000 X17: 0000ffffb5eaf070
(XEN)     X18: 000000000000010f X19: 8040000000000002 X20: 0000000810e60e38
(XEN)     X21: 0000000810e60e38 X22: 0000000000000000 X23: 0000800725f07000
(XEN)     X24: ffff000008f86528 X25: ffff8000501b6540 X26: ffff8000501b6550
(XEN)     X27: ffff8000517bdb40 X28: ffff800052784380  FP: ffff00002212bd50
(XEN)
(XEN)   VTCR_EL2: 80023558
(XEN)  VTTBR_EL2: 0002000765f04000
(XEN)
(XEN)  SCTLR_EL2: 30cd183d
(XEN)    HCR_EL2: 000000008078663f
(XEN)  TTBR0_EL2: 00000000781c5000
(XEN)
(XEN)    ESR_EL2: f2000001
(XEN)  HPFAR_EL2: 0000000000030010
(XEN)    FAR_EL2: ffff000008005f00
(XEN)
(XEN) Xen stack trace from sp=0000800725eafaf0:
(XEN)    0000000000000000 0000800725f07000 000000000021a93c 000000000021a930
(XEN)    0000000000236d18 0000800725f1b990 0000800725eafeb0 0000800725eafeb0
(XEN)    0000800725eaff30 00000000a0000145 000000005a000ea1 ffff000008f86528
(XEN)    0000800725f566a0 00000000002a9088 ffff00000814acc0 0000000a55bb542d
(XEN)    00000000002789d8 0000800725f1b990 00000000002b6cb8 0000800725f03920
(XEN)    00000000002b6cb8 0000000c9084be29 0000000000310228 0000800725eafbf0
(XEN)    ffff00000814f160 0000800725eafbe0 0000000000310228 0000800725eafbf0
(XEN)    0000000000310228 0000000100000002 00000000002b6cb8 0000000000237aa8
(XEN)    0000000000000002 0000000000000000 0000000000000000 0000800725eafc70
(XEN)    0000800725ecd6c8 0000800725ecddf0 00000000000000a0 0000000000000240
(XEN)    000000000026c920 0000800725ecd128 0000000000000002 0000000000000000
(XEN)    0000800725ecd000 0000000000000001 000000000026bfa8 0000000c90bf0b49
(XEN)    0000000000000002 0000000000000000 0000000000276fc4 0000000000000001
(XEN)    0000800725e64000 0000800725e68a60 0000800725e64000 0000800725f566a0
(XEN)    000000000023f53c 000000000027dc14 0000000000311390 0000000000346008
(XEN)    000000000027651c 0000800725eafdd8 0000000000240e44 0000000000000004
(XEN)    0000000000311390 0000000000240e80 0000800725e64000 0000000000240f20
(XEN)    ffff000022127ff0 000000000009c041 000000005a000ea1 ffff800011ae9800
(XEN)    0000000000000124 0000000000240f2c fffffffffffffff2 0000000000000001
(XEN)    0000800725e68ec0 0000800725e68ed0 000000000024694c 00008004dc040000
(XEN)    0000800725e68ec0 00008004dc040000 0000000000247c88 0000000000247c7c
(XEN)    ffffffffffffffea 0000000000000001 ffff800011ae9e80 0000000000000002
(XEN)    000000005a000ea1 0000ffffc52989c0 00000000002463b8 000000000024613c
(XEN)    0000800725eafeb0 0000800725eafeb0 0000800725eaff30 0000000060000145
(XEN)    00000000002789d8 0000800725eafeb0 0000800725eafeb0 01ff00000935de80
(XEN)    0000800725eca000 0000000000310280 0000000000000000 0000000000000000
(XEN)    000000005a000ea1 ffff800011ae9800 0000000000000124 000000000000001d
(XEN)    0000000000000004 000000000027c984 000000005a000ea1 0000800725eafeb0
(XEN)    000000005a000ea1 000000000027a144 0000000000000000 ffff80004e381f80
(XEN)    0000800725eaffb8 0000000000262c58 0000000000262c4c 07e0000160000249
(XEN)    000000000000000f ffff00002212bd90 ffff80004e381f80 000000000009c041
(XEN)    ffff7dffff000000 0000ffffb603d000 0000000000000000 0000ffffb603c000
(XEN)    ffff8000521fdc08 0000000000000200 ffff8000517bdb60 0000000000000000
(XEN)    0000000000000000 0000ffffc529614f 000000000000001b 0000000000000001
(XEN)    000000000000000c 0000ffffb5eaf070 000000000000010f 0000000000000002
(XEN)    ffff80004e381f80 0000000000007ff0 ffff7e0000000000 0000000000000001
(XEN)    ffff000008f86528 ffff8000501b6540 ffff8000501b6550 ffff8000517bdb40
(XEN)    ffff800052784380 ffff00002212bd50 ffff000008537ab8 ffffffffffffffff
(XEN)    ffff0000080c1790 5a000ea1a0000145 0000000060000000 0000000000000000
(XEN)    0000000000000000 ffff800052784380 ffff00002212bd50 0000ffffb5eaf078
(XEN) Xen call trace:
(XEN)    [<0000000000272a48>] put_page+0xa0/0xc4 (PC)
(XEN)    [<0000000000272a48>] put_page+0xa0/0xc4 (LR)
(XEN)


(XEN) hvm_free_ioreq_mfn[417] ---> ref = 1

(XEN) Xen BUG at ...tAUTOINC+bb71237a55-r0/git/xen/include/xen/mm.h:683
(XEN) ----[ Xen-4.14.0  arm64  debug=y   Not tainted ]----
(XEN) CPU:    2
(XEN) PC:     0000000000246e2c ioreq.c#hvm_free_ioreq_mfn+0xbc/0xc0
(XEN) LR:     0000000000246dcc
(XEN) SP:     0000800725eafd70
(XEN) CPSR:   60000249 MODE:64-bit EL2h (Hypervisor, handler)
(XEN)      X0: 0000000000000001  X1: 403fffffffffffff  X2: 000000000000001f
(XEN)      X3: 0000000080000000  X4: 0000000000000000  X5: 0000000000400000
(XEN)      X6: 0080808080808080  X7: fefefefefefeff09  X8: 7f7f7f7f7f7f7f7f
(XEN)      X9: 756e6c64513d313b X10: 7f7f7f7f7f7f7f7f X11: 0101010101010101
(XEN)     X12: 0000000000000008 X13: 000000000028b7d0 X14: 0000800725eafac8
(XEN)     X15: 0000000000000020 X16: 0000000000000000 X17: 0000ffffb5eab3d0
(XEN)     X18: 000000000000010f X19: 0000000810e60e38 X20: 0000000810e60e48
(XEN)     X21: 0000000000000000 X22: 00008004dc0404a0 X23: 000000005a000ea1
(XEN)     X24: ffff800011ae9800 X25: 0000000000000124 X26: 000000000000001d
(XEN)     X27: ffff000008ad1000 X28: ffff800052784380  FP: ffff00002212bd20
(XEN)
(XEN)   VTCR_EL2: 80023558
(XEN)  VTTBR_EL2: 0002000765f04000
(XEN)
(XEN)  SCTLR_EL2: 30cd183d
(XEN)    HCR_EL2: 000000008078663f
(XEN)  TTBR0_EL2: 00000000781c5000
(XEN)
(XEN)    ESR_EL2: f2000001
(XEN)  HPFAR_EL2: 0000000000030010
(XEN)    FAR_EL2: ffff000008005f00
(XEN)
(XEN) Xen stack trace from sp=0000800725eafd70:
(XEN)    0000800725e68ec0 0000800725e68ec0 0000000000246f7c 0000800725e68ec0
(XEN)    00008004dc040000 00000000002476cc ffffffffffffffea 0000000000000001
(XEN)    ffff800011ae9e80 0000000000000002 00000000002462bc 000000000024613c
(XEN)    0000800725eafeb0 0000800725eafeb0 0000800725eaff30 0000000060000145
(XEN)    00000000002789d8 0000800725fb50a4 00000000ffffffff 0100000000277704
(XEN)    00008004dc040000 0000000000000006 0000000000000000 0000000000310288
(XEN)    0000000000000004 0000000125ec0002 0000ffffc52989a8 0000000000000020
(XEN)    0000000000000004 000000000027c984 000000005a000ea1 0000800725eafeb0
(XEN)    000000005a000ea1 000000000027a144 0000000000000000 ffff800011ae9100
(XEN)    0000800725eaffb8 0000000000262c58 0000000000262c4c 07e0000160000249
(XEN)    0000000000000002 0000000000000001 ffff800011ae9e80 ffff800011ae9e88
(XEN)    ffff800011ae9108 ffff800052784380 000000005068e148 0000ffffc5498000
(XEN)    ffff80005156ac10 0000000000000000 00e800008f88bf53 0400000000000000
(XEN)    ffff7e00013e22c0 0000000000000002 0000000000000001 0000000000000001
(XEN)    0000000000000029 0000ffffb5eab3d0 000000000000010f ffff800011ae9110
(XEN)    ffff800011ae9100 ffff800011ae9110 0000000000000001 ffff800011ae9e80
(XEN)    ffff800011ae9800 0000000000000124 000000000000001d ffff000008ad1000
(XEN)    ffff800052784380 ffff00002212bd20 ffff000008537004 ffffffffffffffff
(XEN)    ffff0000080c17e4 5a000ea160000145 0000000060000000 0000000000000000
(XEN)    0000000000000000 ffff800052784380 ffff00002212bd20 0000ffffb5eab3dc
(XEN)    0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<0000000000246e2c>] ioreq.c#hvm_free_ioreq_mfn+0xbc/0xc0 (PC)
(XEN)    [<0000000000246dcc>] ioreq.c#hvm_free_ioreq_mfn+0x5c/0xc0 (LR)
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 2:
(XEN) Xen BUG at ...tAUTOINC+bb71237a55-r0/git/xen/include/xen/mm.h:683
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) PSCI cpu off failed for CPU0 err=-3
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...




--
Regards,

Oleksandr Tyshchenko

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.