Re: [Xen-users] Trouble with VIFs using Debian xen 2.6.32-x kernels

["Joseph M. Deming" <joseph.deming@xxxxxxxxxxxxxx>]

Found this bug report, posted same info there.  Will follow up there
because I believe I've found enough similar stories to suspect an actual
bug with 2.6.32 kernel.

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1612

Any thoughts are still welcome.

On Thu, 2010-05-20 at 12:50 -0400, Joseph M. Deming wrote:
> Hello, new to list, been using Xen for a while, trying to set up my new
> third box.  Having issues.
> 
> Suspiciously similar to recent post
> http://lists.xensource.com/archives/html/xen-users/2010-05/msg00531.html
> New to list so couldn't reply directly to it.
> 
> Start a Xen DomU and get kernel oops.  All following attempts to start
> DomU's result in 'Error: Device 0 (vif) could not be connected. Hotplug
> scripts not working.'  If I start machine soon enough after boot the
> first one runs and shuts down without issue, but still causes initial
> kernel oops.  Dump seems to imply necessary drivers are loaded netfront,
> blkfront, netxen_nic, etc.  Also, if I remove VIF settings from conf
> boot is fine.  ioemu vs no ioemu makes no difference, nor does the DomU
> kernel booted, I've tried 3-4 a couple of which I know work on 2.6.26.
> 
> I use scripts to manage and build all DomUs and Dom0s so I can assure
> configuration is the same between my 3 machines.  I've tried Dom0 as
> 2.6.32-4-xen-amd64 and 2.6.32-5-xen-amd64.  Identical config works on
> other systems with 2.6.26-2-xen-amd64 as Dom0.  Can't step back to
> 2.6.26-2 or 2.6.18-6 on this box because it shipped with MegaRAID SAS6
> and there are issues found elsewhere on this list describing boot
> impossibilities with those kernels.
> 
> Ideas?  Similar experiences?  Happy to provide more info if asked, I'll
> try to keep up on this post.
> 
> - JMD
> 
> Detailed output follows:
> 
> ======== KERNEL OOPS DMESG (AND SURROUNDING) ========
> ======= THIS IS FOR A MACHINE THAT ACTUALLY STARTS AND RUNS, CAUSING
> FIRST KERNEL OOPS =====
> [   74.636071] eth0: no IPv6 routers present
> [   77.828051] eth1: no IPv6 routers present
> [   77.848047] peth1: no IPv6 routers present
> [  160.948183]   alloc irq_desc for 3236 on node 0
> [  160.948194]   alloc kstat_irqs on node 0
> [  161.120338] BUG: unable to handle kernel NULL pointer dereference at
> 0000000000000150
> [  161.120578] IP: [<ffffffff811fb5b7>] netback_uevent+0x83/0xaf
> [  161.120734] PGD 47caf3067 PUD 47db6a067 PMD 0 
> [  161.124248] Oops: 0000 [#1] SMP 
> [  161.124248] last sysfs file: /sys/devices/vif-1-0/uevent
> [  161.124248] CPU 3 
> [  161.124248] Modules linked in: bridge stp xen_evtchn xt_recent
> ipt_LOG nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_tcpudp
> iptable_filter ip_tables x_tables nfsd exportfs nfs lockd fscache
> nfs_acl auth_rpcgss sunrpc uhci_hcd snd_pcm radeon snd_timer ttm
> drm_kms_helper snd amd64_edac_mod drm i2c_algo_bit soundcore edac_core
> snd_page_alloc tpm_tis edac_mce_amd tpm i2c_nforce2 i2c_core shpchp
> tpm_bios pci_hotplug processor dcdbas joydev pcspkr button
> acpi_processor evdev psmouse serio_raw ext3 jbd mbcache dm_mod xenfs
> xen_blkfront xen_netfront netxen_nic sd_mod crc_t10dif usbhid hid sg
> sr_mod cdrom mptsas ata_generic mptscsih ohci_hcd mptbase igb sata_nv
> dca libata scsi_transport_sas thermal scsi_mod ehci_hcd thermal_sys
> usbcore nls_base [last unloaded: scsi_wait_scan]
> [  161.124248] Pid: 2701, comm: udevd Tainted: G        W
> 2.6.32-5-xen-amd64 #1 PowerEdge R805
> [  161.124248] RIP: e030:[<ffffffff811fb5b7>]  [<ffffffff811fb5b7>]
> netback_uevent+0x83/0xaf
> [  161.124248] RSP: e02b:ffff8804022d9e18  EFLAGS: 00010246
> [  161.124248] RAX: 02000000000000c1 RBX: ffff88040229f000 RCX:
> 0000000000400035
> [  161.124248] RDX: ffff88047da64500 RSI: ffffea000fb7c5e0 RDI:
> 02000000000002c0
> [  161.124248] RBP: ffff88047da64ac0 R08: 0000000000000000 R09:
> ffffffff814617e0
> [  161.124248] R10: 0000000000000200 R11: 0000000000000400 R12:
> ffff88047c9fb400
> [  161.124248] R13: 0000000000000000 R14: ffff88047dab8000 R15:
> ffff88047ca71860
> [  161.124248] FS:  00007f28978e5790(0000) GS:ffff88048bfcc000(0000)
> knlGS:0000000000000000
> [  161.124248] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  161.124248] CR2: 0000000000000150 CR3: 0000000402240000 CR4:
> 0000000000000660
> [  161.124248] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  161.124248] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  161.124248] Process udevd (pid: 2701, threadinfo ffff8804022d8000,
> task ffff88047b5446a0)
> [  161.124248] Stack:
> [  161.124248]  0000000000000908 ffff88047c9fb440 ffff88040229f000
> ffff88047cccbeb0
> [  161.124248] <0> ffff88040229f000 ffffffff812275ea ffff88047c9fb450
> ffff88047cccbeb0
> [  161.124248] <0> ffffffff814a1790 0000000000000000 ffff88047c9fb450
> ffffffff81227729
> [  161.124248] Call Trace:
> [  161.124248]  [<ffffffff812275ea>] ? dev_uevent+0x104/0x146
> [  161.124248]  [<ffffffff81227729>] ? show_uevent+0x81/0xd5
> [  161.124248]  [<ffffffff812272ce>] ? dev_attr_show+0x1f/0x42
> [  161.124248]  [<ffffffff8114055b>] ? sysfs_read_file+0xa7/0x125
> [  161.124248]  [<ffffffff810f0972>] ? vfs_read+0xa6/0xff
> [  161.124248]  [<ffffffff810f0a87>] ? sys_read+0x45/0x6e
> [  161.124248]  [<ffffffff81011b42>] ? system_call_fastpath+0x16/0x1b
> [  161.124248] Code: c7 c6 f8 a8 3e 81 31 c0 48 89 df e8 a0 0a f9 ff 85
> c0 74 0f 48 89 ef bb f4 ff ff ff e8 4d d1 ee ff eb 2a 48 89 ef e8 43 d1
> ee ff <49> 8b 95 50 01 00 00 48 89 df 31 c0 48 c7 c6 02 a9 3e 81 e8 6d 
> [  161.124248] RIP  [<ffffffff811fb5b7>] netback_uevent+0x83/0xaf
> [  161.124248]  RSP <ffff8804022d9e18>
> [  161.124248] CR2: 0000000000000150
> [  161.124248] ---[ end trace a7919e7f17c0a727 ]---
> [  161.173938] BUG: unable to handle kernel NULL pointer dereference at
> 0000000000000150
> [  161.174183] IP: [<ffffffff811fb5b7>] netback_uevent+0x83/0xaf
> [  161.174346] PGD 47d2af067 PUD 47cf7e067 PMD 0 
> [  161.174657] Oops: 0000 [#2] SMP 
> [  161.174909] last sysfs file: /sys/devices/vif-1-1/uevent
> [  161.175025] CPU 6 
> [  161.175183] Modules linked in: bridge stp xen_evtchn xt_recent
> ipt_LOG nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_tcpudp
> iptable_filter ip_tables x_tables nfsd exportfs nfs lockd fscache
> nfs_acl auth_rpcgss sunrpc uhci_hcd snd_pcm radeon snd_timer ttm
> drm_kms_helper snd amd64_edac_mod drm i2c_algo_bit soundcore edac_core
> snd_page_alloc tpm_tis edac_mce_amd tpm i2c_nforce2 i2c_core shpchp
> tpm_bios pci_hotplug processor dcdbas joydev pcspkr button
> acpi_processor evdev psmouse serio_raw ext3 jbd mbcache dm_mod xenfs
> xen_blkfront xen_netfront netxen_nic sd_mod crc_t10dif usbhid hid sg
> sr_mod cdrom mptsas ata_generic mptscsih ohci_hcd mptbase igb sata_nv
> dca libata scsi_transport_sas thermal scsi_mod ehci_hcd thermal_sys
> usbcore nls_base [last unloaded: scsi_wait_scan]
> [  161.177873] Pid: 2720, comm: udevd Tainted: G      D W
> 2.6.32-5-xen-amd64 #1 PowerEdge R805
> [  161.177873] RIP: e030:[<ffffffff811fb5b7>]  [<ffffffff811fb5b7>]
> netback_uevent+0x83/0xaf
> [  161.177873] RSP: e02b:ffff880402323e18  EFLAGS: 00010246
> [  161.177873] RAX: 02000000000000c1 RBX: ffff88047cbcf000 RCX:
> 000000000040003e
> [  161.177873] RDX: ffff880402171680 RSI: ffffea000e0750b8 RDI:
> 02000000000002c0
> [  161.177873] RBP: ffff880402171fc0 R08: 0000000000000000 R09:
> ffffffff814617e0
> [  161.177873] R10: ffff88047c314980 R11: 0000000000000001 R12:
> ffff88047cb65c00
> [  161.177873] R13: 0000000000000000 R14: ffff88047db6a000 R15:
> ffff88047c241320
> [  161.177873] FS:  00007f28978e5790(0000) GS:ffff880018889000(0000)
> knlGS:0000000000000000
> [  161.177873] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  161.177873] CR2: 0000000000000150 CR3: 000000047d142000 CR4:
> 0000000000000660
> [  161.177873] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [  161.177873] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [  161.177873] Process udevd (pid: 2720, threadinfo ffff880402322000,
> task ffff88047b40dbd0)
> [  161.177873] Stack:
> [  161.177873]  0000000000000908 ffff88047cb65c40 ffff88047cbcf000
> ffff88047cc98640
> [  161.177873] <0> ffff88047cbcf000 ffffffff812275ea ffff88047cb65c50
> ffff88047cc98640
> [  161.177873] <0> ffffffff814a1790 0000000000000000 ffff88047cb65c50
> ffffffff81227729
> [  161.177873] Call Trace:
> [  161.177873]  [<ffffffff812275ea>] ? dev_uevent+0x104/0x146
> [  161.177873]  [<ffffffff81227729>] ? show_uevent+0x81/0xd5
> [  161.177873]  [<ffffffff812272ce>] ? dev_attr_show+0x1f/0x42
> [  161.177873]  [<ffffffff8114055b>] ? sysfs_read_file+0xa7/0x125
> [  161.177873]  [<ffffffff810f0972>] ? vfs_read+0xa6/0xff
> [  161.177873]  [<ffffffff810f0a87>] ? sys_read+0x45/0x6e
> [  161.177873]  [<ffffffff81011b42>] ? system_call_fastpath+0x16/0x1b
> [  161.177873] Code: c7 c6 f8 a8 3e 81 31 c0 48 89 df e8 a0 0a f9 ff 85
> c0 74 0f 48 89 ef bb f4 ff ff ff e8 4d d1 ee ff eb 2a 48 89 ef e8 43 d1
> ee ff <49> 8b 95 50 01 00 00 48 89 df 31 c0 48 c7 c6 02 a9 3e 81 e8 6d 
> [  161.177873] RIP  [<ffffffff811fb5b7>] netback_uevent+0x83/0xaf
> [  161.177873]  RSP <ffff880402323e18>
> [  161.177873] CR2: 0000000000000150
> [  161.177873] ---[ end trace a7919e7f17c0a728 ]---
> [  161.255914]   alloc irq_desc for 3235 on node 0
> [  161.255981]   alloc kstat_irqs on node 0
> [  161.383844] device vif1.0 entered promiscuous mode
> [  161.398741] eth0: port 2(vif1.0) entering forwarding state
> [  161.493572] physdev match: using --physdev-out in the OUTPUT, FORWARD
> and POSTROUTING chains for non-bridged traffic is not supported anymore.
> [  161.513910] physdev match: using --physdev-out in the OUTPUT, FORWARD
> and POSTROUTING chains for non-bridged traffic is not supported anymore.
> [  161.525749] physdev match: using --physdev-out in the OUTPUT, FORWARD
> and POSTROUTING chains for non-bridged traffic is not supported anymore.
> [  161.525894] physdev match: using --physdev-out in the OUTPUT, FORWARD
> and POSTROUTING chains for non-bridged traffic is not supported anymore.
> [  161.632627] device vif1.1 entered promiscuous mode
> [  161.647717] eth1: port 2(vif1.1) entering forwarding state
> [  161.697347] physdev match: using --physdev-out in the OUTPUT, FORWARD
> and POSTROUTING chains for non-bridged traffic is not supported anymore.
> 
> ====== XM INFO =====
> host                   : XXXXX.matrix.msu.edu
> release                : 2.6.32-5-xen-amd64
> version                : #1 SMP Mon May 3 12:36:28 UTC 2010
> machine                : x86_64
> nr_cpus                : 12
> nr_nodes               : 1
> cores_per_socket       : 6
> threads_per_core       : 1
> cpu_mhz                : 2400
> hw_caps                :
> 178bf3ff:efd3fbff:00000000:00000310:00802001:00000000:000037ff:00000000
> virt_caps              : hvm
> total_memory           : 32762
> free_memory            : 3907
> node_to_cpu            : node0:0-11
> node_to_memory         : node0:3907
> xen_major              : 3
> xen_minor              : 4
> xen_extra              : .3-rc6
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
> hvm-3.0-x86_32p hvm-3.0-x86_64 
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> cc_compiler            : gcc version 4.4.3 (Debian 4.4.3-9) 
> cc_compile_by          : waldi
> cc_compile_domain      : debian.org
> cc_compile_date        : Sat May  8 09:35:13 UTC 2010
> xend_config_format     : 4
> 
> ===== .conf =====
> # BOOT WITH STANDARD KERNEL
> kernel      = '/boot/vmlinuz-2.6.32-4-vserver-amd64'
> ramdisk     = '/boot/initrd.img-2.6.32-4-vserver-amd64'
> memory      = '2048'
> root        = '/dev/xvda1'
> 
> # NEW SYSTEM NAME/DRIVES/NETWORK
> name        = 'wario'
> 
> disk        = [
>                 'phy:/dev/xendisks/wario-root,xvda1,w',
>                 'phy:/dev/xendisks/wario-swap,xvda2,w'
>               ]
> 
> vif         = [
>                 'type=ioemu, ip=35.9.18.249, mac=00:16:3e:09:12:F9,
> bridge=eth0',
>                 'type=ioemu, ip=10.0.0.249,  mac=00:16:3E:00:00:F9,
> bridge=eth1'
>               ]
> on_poweroff = 'destroy'
> on_reboot   = 'restart'
> on_crash    = 'restart'
> 
> ===== XEND-CONFIG.SXP =====
> (xen-api-server ((9363 pam) (unix none)))
> (network-script 'matrix-netbridge-wrapper')
> (vif-script 'matrix-vif-wrapper')
> (dom0-min-mem 196)
> (dom0-cpus 0)
> (vnc-listen '0.0.0.0')
> (vncpasswd 'matrix^2007')
> 
> ===== NEWORK WRAPPER =====
> #!/bin/sh
> 
>       /etc/xen/scripts/network-bridge $1 netdev=eth0
>       /etc/xen/scripts/network-bridge $1 netdev=eth1
> 
> 
> ===== VIF WRAPPER =====
> #!/bin/sh
> 
>       /etc/xen/scripts/vif-bridge $1 bridge=eth0
>       /etc/xen/scripts/vif-bridge $1 bridge=eth1
> 
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users