Re: [Xen-devel] 2.6.32.16: hvm domU boot - oops at add_pin_to_irq_node_nopanic

[Bruce Edge <bruce.edge@xxxxxxxxx>]

On Fri, Jul 9, 2010 at 3:37 AM, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> wrote:

On Thu, 8 Jul 2010, Bruce Edge wrote:
> On Thu, Jul 8, 2010 at 10:04 AM, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> wrote:
>       On Thu, 8 Jul 2010, Bruce Edge wrote:
>       > On Thu, Jul 8, 2010 at 9:55 AM, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> wrote:
>       >       On Thu, 8 Jul 2010, Bruce Edge wrote:
>       >       > This is the same problem as this post:
>       >       > http://lists.xensource.com/archives/html/xen-devel/2010-06/msg01114.html
>       >       >
>       >       > Stefano suggested I retry after Jeremy's pull.
>       >       >
>       >       > Sync'd with 2.6.32.16 last night and it looks like the identical problem.
>       >       >
>       >       > I can boot Ubuntu 10.04 with it's own native kernel, but not with a pv-ops kernel.
>       >       >
>       >       > This is with # CONFIG_XEN_PLATFORM_PCI is not set, .config is attached.
>       >       >
>       >
>       > I think that Jeremy didn't merge the up-to-date PV on HVM branch in
>       > stable-2.6.32.x but only in next.
>       >
>       >
>       > I thought that the PV on HVM code was only used if CONFIG_XEN_PLATFORM_PCI=y. 
>       > Is that not the case?
>       >
>
> For the most part yes, however the pv timer can still work even without
> the platform pci driver and with the pv timer enabled you need another
> patch to prevent the ioapic initialization code from breaking. This last
> patch is missing from stable-2.6.32.x.
>
>
> Could you give me a pointer to this patch? I'd like to test it to confirm whether it is indeed the problem.
>

you can checkout my tree from here (vanilla 2.6.34 + pv on hvm patches):

git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git

branch name 2.6.34-pvhvm-v4.

The patches that you need to have are:

- "__setup_vector_irq: handle NULL chip_data"
32b5ffd98391cb47c3cc9459c445d9490a523622

- "Fix possible NULL pointer dereference in print_IO_APIC"
a57e90eaf9c55da55e3de09ebda99cb2f8a00e40

- "Fix find_unbound_irq in presence of ioapic irqs."
d7fc454bce7b63aab43ef0e11d97b206082e4740

The above patch set didn't all apply to xen/stable-2.6.32x Some were already applied and others needed additional patches:

BTW I have a new version of the branch (2.6.34-pvhvm-v6) without those
patches because I was able to fix the issue in a better way, but I
haven't sent it to the list yet. Jeremey's PV on HVM branch is based on
v4.

So I tried the same hvm boot using the following 3 branches. 

    xen/next-2.6.32

Stefano's

    2.6.34-pvhvm-v4 

    2.6.34-pvhvm-v6

The base xen 4.0.1rc3 did not have the patch Stefano posted later in this thread.

The boot summary is inline and the complete bootlogs are attached. It has a few extra bits required by the .34 kernel, but is essentially the same one for all 3 cases.

The gist of it being:

CONFIG_XEN=y

CONFIG_XEN_MAX_DOMAIN_MEMORY=32

CONFIG_XEN_SAVE_RESTORE=y

# CONFIG_XEN_DEBUG_FS is not set

CONFIG_XEN_BLKDEV_FRONTEND=m

CONFIG_NETXEN_NIC=m

CONFIG_XEN_NETDEV_FRONTEND=m

CONFIG_XEN_KBDDEV_FRONTEND=m

CONFIG_HVC_XEN=y

CONFIG_XEN_FBDEV_FRONTEND=m

CONFIG_XEN_BALLOON=y

CONFIG_XEN_SCRUB_PAGES=y

CONFIG_XEN_DEV_EVTCHN=m

CONFIG_XENFS=m

CONFIG_XEN_COMPAT_XENFS=y

CONFIG_XEN_SYS_HYPERVISOR=y

# CONFIG_XEN_PLATFORM_PCI is not set

The .config is also attached

1) xen/next-2.6.32 - Hangs at the end of the following sequence:

[    0.464686] ACPI: bus type pci registered^M

[    0.467109] PCI: Using configuration type 1 for base access^M

[    0.470495] bio: create slab <bio-0> at 0^M

[    0.523248] ACPI: Interpreter enabled^M

[    0.525249] ACPI: (supports S0 S3 S4 S5)^M

[    0.527628] ACPI: Using IOAPIC for interrupt routing^M

[    0.624323] ACPI: No dock devices found.^M

[    0.626607] ACPI: PCI Root Bridge [PCI0] (0000:00)^M

[    0.635170] * Found PM-Timer Bug on the chipset. Due to workarounds for a bug,^M

[    0.635170] * this clock source is slow. Consider trying other clock sources^M

[    0.641158] pci 0000:00:01.3: quirk: region 1f40-1f7f claimed by PIIX4 ACPI^M

[    1.115247] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)^M

[    1.118966] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)^M

[    1.121346] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)^M

[    1.125128] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)^M

<hang>

2) 2.6.34-pvhvm-v4 - boots, with a lot of call traces on the serial console:

[    1.880528] WARNING: at arch/x86/xen/mmu.c:1953 xen_hvm_exit_mmap+0x45/0x50()^M

[    1.884168] Hardware name: HVM domU^M

[    1.885967] Modules linked in:^M

[    1.887638] Pid: 48, comm: init Not tainted 2.6.34 #1^M

[    1.890384] Call Trace:^M

[    1.891721]  [<ffffffff8105cefb>] warn_slowpath_common+0x7b/0xc0^M

[    1.894804]  [<ffffffff8105cf54>] warn_slowpath_null+0x14/0x20^M

[    1.897820]  [<ffffffff81005015>] xen_hvm_exit_mmap+0x45/0x50^M

[    1.900941]  [<ffffffff8110f7b0>] exit_mmap+0x50/0x190^M

[    1.903569]  [<ffffffff8105a5e2>] mmput+0x42/0x110^M

[    1.906059]  [<ffffffff811443a3>] flush_old_exec+0x463/0x620^M

[    1.908995]  [<ffffffff8118436e>] load_elf_binary+0x38e/0x1d20^M

[    1.912375]  [<ffffffff81108ba6>] ? follow_page+0x2d6/0x350^M

[    1.915210]  [<ffffffff81108ba6>] ? follow_page+0x2d6/0x350^M

[    1.918055]  [<ffffffff8110d7cf>] ? __get_user_pages+0x10f/0x430^M

[    1.921309]  [<ffffffff8114386c>] ? get_arg_page+0x5c/0xc0^M

[    1.924127]  [<ffffffff8114528d>] search_binary_handler+0xed/0x310^M

[    1.927336]  [<ffffffff811458e3>] do_execve+0x333/0x410^M

[    1.930170]  [<ffffffff812dff58>] ? strncpy_from_user+0x38/0x60^M

[    1.933261] Clockevents: could not switch to one-shot mode: lapic is not functional.^M

[    1.937318] Could not switch to high resolution mode on CPU 0^M

[    1.940399]  [<ffffffff810135da>] sys_execve+0x4a/0x80^M

[    1.943078]  [<ffffffff8100b48a>] stub_execve+0x6a/0xc0^M

[    1.945737] ---[ end trace 7264d4229303d265 ]---^M

but eventually gets to a login and appears functional, but the number of call traces coming out during the boot are not comforting.

3) 2.6.34-pvhvm-v6 - Same as v4

However the root fs of the domU was corrupted after this last test. Possibly the result of using partprobe to get at lvm block device partitions and mount them to install new kernels, but I had been doing that for a while with no ill effects.

Please advise, is there any other pv-ops kernel branch suitable for domU hvm operation or other patch set I should try?

Dom0 modules:

#> lsmod

Module                  Size  Used by

nf_conntrack_ipv4      12980  0 

nf_defrag_ipv4          1481  1 nf_conntrack_ipv4

xt_state                1490  0 

nf_conntrack           74126  2 nf_conntrack_ipv4,xt_state

xt_physdev              1739  0 

iptable_filter          2791  0 

ip_tables              18422  1 iptable_filter

x_tables               22493  3 xt_state,xt_physdev,ip_tables

ipmi_msghandler        37147  0 

nfs                   316736  1 

xenfs                  11748  1 

xen_evtchn              5670  1 

pci_stub                1598  0 

bridge                 53280  0 

stp                     2203  1 bridge

ppdev                   6407  0 

lp                      9336  0 

parport_pc             30246  1 

psmouse                64320  0 

parport                37744  3 ppdev,lp,parport_pc

ioatdma                42609  24 

serio_raw               4982  0 

joydev                 11296  0 

dca                     6701  1 ioatdma

usbhid                 41404  0 

hid                    83408  1 usbhid

usb_storage            50089  0 

e1000e                136653  0 

floppy                 63348  0 

#> cat /proc/misc 

 53 xen/evtchn

 54 network_throughput

 55 network_latency

 56 cpu_dma_latency

 57 device-mapper

  1 psaux

200 tun

 58 pktcdvd

228 hpet

 59 blktap-control

 60 xen/gntdev

229 fuse

 61 ecryptfs

231 snapshot

227 mcelog

 62 rfkill

Let me know if there's anything else I can provide, or other test config to run.

Thanks.

-Bruce

Attachment: screenlog.pvhvm-v4
Description: Binary data

Attachment: screenlog.pvhvm-v6
Description: Binary data

Attachment: screenlog.xen-next
Description: Binary data

Attachment: linux-2.6-pvhvm-v6.config
Description: Binary data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel