[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] domU to domU networking issues in v3.7? (netserver/netperf failing to communicate)



Wednesday, November 14, 2012, 7:24:53 AM, you wrote:

> On Mon, Nov 12, 2012 at 06:20:55PM +0100, Sander Eikelenboom wrote:
>> 
>> Monday, November 12, 2012, 5:32:04 PM, you wrote:
>> 
>> > On Mon, Nov 12, 2012 at 03:50:24PM +0100, Sander Eikelenboom wrote:
>> >> 
>> >> Monday, November 12, 2012, 3:28:35 PM, you wrote:
>> >> 
>> >> > On Mon, Nov 12, 2012 at 09:54:44AM +0000, Ian Campbell wrote:
>> >> >> On Sat, 2012-11-10 at 13:59 +0000, Konrad Rzeszutek Wilk wrote:
>> >> >> > Hey Ian, Xen-devel mailingl list,
>> >> >> > 
>> >> >> > I think the issue of 70% traffic lost was actually introduced in 
>> >> >> > v3.6 or
>> >> >> > perhaps v3.5. Annie and Marcos (CC-ed here) are looking to see which 
>> >> >> > of
>> >> >> > the releases introduced this. The issue we are seeing is that a domU
>> >> >> > to domU communication breaks - this is with netperf/netserver 
>> >> >> > talking to
>> >> >> > each other.
>> >> >> > 
>> >> >> > Anyhow, I think the 3.7 compound page exacerbated the problem and 
>> >> >> > also
>> >> >> > (at least on some of my test hardware) exposed existing issues with
>> >> >> > drivers. The issue I have is that the 'skge' driver has a bug that 
>> >> >> > has
>> >> >> > been there for ages (I tested way back to 3.0 and still saw it) were 
>> >> >> > it
>> >> >> > can not work with SWIOTLB. It is probably missing an pci_dma_sync
>> >> >> > somewhere. 
>> >> >> > 
>> >> >> > Anyhow the compound page got me to look at Xen-SWIOTLB and that looks
>> >> >> > OK. Even with synthetic driver (the fake one I posted somewhere) it
>> >> >> > dealt with compound pages properly (with debug or non-debug Xen
>> >> >> > hypervisor).
>> >> >> 
>> >> >> The debug build is probably most interesting since it deliberately
>> >> >> allocates a non 1-1 p-to-m mapping so as to catch exactly these sorts 
>> >> >> of
>> >> >> issues.
>> >> 
>> >> > Right. My test env runs with that. And so far it only has issues
>> >> > with the skge one.
>> >> >> 
>> >> >> > So was wondering if you had looked at this in more details? Any
>> >> >> > ideas? Or would it be more prudent to ask that once we know for sure
>> >> >> > which Linux release introduced the communication failures between
>> >> >> > guests?
>> >> >> 
>> >> >> I've not looked at it any further I'm afraid.
>> >> >> 
>> >> >> If these changes (be they in 3.5 or later, or earlier) are exposing
>> >> >> driver bugs then I suspect the netdev chaps would want to know about 
>> >> >> it.
>> >> 
>> >> > Right. Annie (CC-ed here) mentioned to me that v3.5 looks to work ok.
>> >> > And is off checking v3.6. v3.7 is definitly a no go.
>> >> >> 
>> >> >> FWIW I see the issue with tg3.
>> >> 
>> >> After the issues with netback where fixed, I'm seeing the issues with 
>> >> net_front reverting the single commit 
>> >> 5640f7685831e088fe6c2e1f863a6805962f8e81 (that was pointed out for 
>> >> netback) also makes these disappear.
>> 
>> > Were you ever able to trigger the BUG_ON in the patch that Ian posted?
>> 
>> What exact patch (or any other patch that can help you ) ?
>> (so i can try again to be sure)

> This one:
> http://lists.xen.org/archives/html/xen-devel/2012-10/msg00893.html


Sorry for the late response!
I have changed the bug_on in a warn_on and applied it to a kernel with and 
without commit 5640f7685831e088fe6c2e1f863a6805962f8e81 reverted.
- With commit 5640f7685831e088fe6c2e1f863a6805962f8e81 reverted I don't see any 
warnings (also after loading domU's with a few GB of network traffic)
- With a vanilla kernel and the patch, i can just ssh to a domU to see the 
warnings, after which SSH locks up. So it happens immediately, even without 
giving the domU any serious network load.


[   85.447191]  <IRQ>  [<ffffffff8106750a>] warn_slowpath_common+0x7a/0xb0
[   85.447201]  [<ffffffff81067555>] warn_slowpath_null+0x15/0x20
[   85.447207]  [<ffffffff814b2b6e>] xennet_start_xmit+0x7fe/0x860
[   85.447213]  [<ffffffff816a6169>] dev_hard_start_xmit+0x209/0x460
[   85.447220]  [<ffffffff816c1e56>] sch_direct_xmit+0xf6/0x290
[   85.447225]  [<ffffffff816a6566>] dev_queue_xmit+0x1a6/0x5a0
[   85.447231]  [<ffffffff816a63c0>] ? dev_hard_start_xmit+0x460/0x460
[   85.447238]  [<ffffffff816b0897>] neigh_resolve_output+0x127/0x250
[   85.447244]  [<ffffffff817405ad>] ip_finish_output+0x39d/0x530
[   85.447250]  [<ffffffff817402dd>] ? ip_finish_output+0xcd/0x530
[   85.447256]  [<ffffffff81740799>] ip_output+0x59/0xe0
[   85.447262]  [<ffffffff8173f2b8>] ip_local_out+0x28/0x90
[   85.447267]  [<ffffffff8173f86f>] ip_queue_xmit+0x17f/0x4a0
[   85.447273]  [<ffffffff8173f6f0>] ? ip_send_unicast_reply+0x340/0x340
[   85.447282]  [<ffffffff810a1cf7>] ? getnstimeofday+0x47/0xe0
[   85.447288]  [<ffffffff816962c9>] ? __skb_clone+0x29/0x120
[   85.447294]  [<ffffffff81755940>] tcp_transmit_skb+0x400/0x8d0
[   85.447303]  [<ffffffff8113ad55>] ? kmem_cache_free+0x125/0x1c0
[   85.450359]  [<ffffffff8175891a>] tcp_write_xmit+0x21a/0xa50
[   85.450359]  [<ffffffff81698c82>] ? __kfree_skb+0x42/0xa0
[   85.450359]  [<ffffffff817591bd>] __tcp_push_pending_frames+0x2d/0x90
[   85.450359]  [<ffffffff817545c4>] tcp_rcv_established+0x164/0x640
[   85.450359]  [<ffffffff8175cdbf>] ? tcp_v4_rcv+0x6cf/0xb10
[   85.450359]  [<ffffffff8175c3a5>] tcp_v4_do_rcv+0x135/0x480
[   85.450359]  [<ffffffff817d0dc2>] ? _raw_spin_lock_nested+0x42/0x50
[   85.450359]  [<ffffffff8175cdbf>] ? tcp_v4_rcv+0x6cf/0xb10
[   85.450359]  [<ffffffff8175d04d>] tcp_v4_rcv+0x95d/0xb10
[   85.450359]  [<ffffffff810b2158>] ? lock_acquire+0xd8/0x100
[   85.450359]  [<ffffffff81739895>] ? ip_local_deliver_finish+0x45/0x230
[   85.450359]  [<ffffffff8173996a>] ip_local_deliver_finish+0x11a/0x230
[   85.450359]  [<ffffffff81739895>] ? ip_local_deliver_finish+0x45/0x230
[   85.450359]  [<ffffffff81739ab8>] ip_local_deliver+0x38/0x80
[   85.450359]  [<ffffffff8173907a>] ip_rcv_finish+0x15a/0x630
[   85.450359]  [<ffffffff81739768>] ip_rcv+0x218/0x300
[   85.450359]  [<ffffffff816a1a9d>] __netif_receive_skb+0x65d/0x8d0
[   85.450359]  [<ffffffff816a1585>] ? __netif_receive_skb+0x145/0x8d0
[   85.450359]  [<ffffffff810ae3fd>] ? trace_hardirqs_on+0xd/0x10
[   85.450359]  [<ffffffff810faf83>] ? free_hot_cold_page+0x1b3/0x1e0
[   85.450359]  [<ffffffff816a4008>] netif_receive_skb+0x28/0xf0
[   85.450359]  [<ffffffff81699b13>] ? __pskb_pull_tail+0x253/0x340
[   85.450359]  [<ffffffff814b3795>] xennet_poll+0xad5/0xe10
[   85.450359]  [<ffffffff816a4db6>] net_rx_action+0x136/0x260
[   85.450359]  [<ffffffff8106f3a1>] ? __do_softirq+0x71/0x1a0
[   85.450359]  [<ffffffff8106f3f9>] __do_softirq+0xc9/0x1a0
[   85.450359]  [<ffffffff817d393c>] call_softirq+0x1c/0x30
[   85.450359]  [<ffffffff8100fd95>] do_softirq+0x85/0xf0
[   85.450359]  [<ffffffff8106f26e>] irq_exit+0x9e/0xd0
[   85.450359]  [<ffffffff8134659f>] xen_evtchn_do_upcall+0x2f/0x40
[   85.450359]  [<ffffffff817d399e>] xen_do_hypervisor_callback+0x1e/0x30
[   85.450359]  <EOI>  [<ffffffff8100122a>] ? xen_hypercall_xen_version+0xa/0x20
[   85.450359]  [<ffffffff8100122a>] ? xen_hypercall_xen_version+0xa/0x20
[   85.450359]  [<ffffffff8100880d>] ? xen_force_evtchn_callback+0xd/0x10
[   85.450359]  [<ffffffff810091b2>] ? check_events+0x12/0x20
[   85.450359]  [<ffffffff8100919f>] ? xen_restore_fl_direct_reloc+0x4/0x4
[   85.450359]  [<ffffffff810adfc0>] ? lockdep_trace_alloc+0xa0/0x130
[   85.450359]  [<ffffffff810fc9ad>] ? __alloc_pages_nodemask+0x8d/0x9a0
[   85.450359]  [<ffffffff810b2158>] ? lock_acquire+0xd8/0x100
[   85.450359]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.450359]  [<ffffffff812a332e>] ? check_blkcg_changed+0x8e/0x180
[   85.450359]  [<ffffffff812a32a0>] ? cfq_put_queue+0x220/0x220
[   85.450359]  [<ffffffff810b2158>] ? lock_acquire+0xd8/0x100
[   85.450359]  [<ffffffff81132f21>] ? alloc_pages_vma+0xc1/0x1d0
[   85.450359]  [<ffffffff8110752a>] ? shmem_alloc_page+0x5a/0x70
[   85.450359]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.450359]  [<ffffffff810f40fd>] ? find_get_page+0xad/0x100
[   85.450359]  [<ffffffff810f4050>] ? find_get_pages+0x190/0x190
[   85.450359]  [<ffffffff810b171b>] ? __lock_acquire+0x46b/0xdd0
[   85.450359]  [<ffffffff810f43d1>] ? find_lock_page+0x21/0x80
[   85.450359]  [<ffffffff81107c3f>] ? shmem_getpage_gfp+0x33f/0x720
[   85.450359]  [<ffffffff811080f6>] ? shmem_fault+0x46/0x70
[   85.450359]  [<ffffffff812a2d33>] ? cfq_add_cfqq_rr+0x33/0x100
[   85.450359]  [<ffffffff81269f79>] ? shm_fault+0x19/0x20
[   85.450359]  [<ffffffff81114de7>] ? __do_fault+0x67/0x4f0
[   85.450359]  [<ffffffff8111885d>] ? handle_pte_fault+0xfd/0x820
[   85.450359]  [<ffffffff8100509f>] ? pte_mfn_to_pfn+0x8f/0xf0
[   85.450359]  [<ffffffff81005159>] ? xen_pmd_val+0x9/0x10
[   85.450359]  [<ffffffff81119208>] ? handle_mm_fault+0x288/0x360
[   85.450359]  [<ffffffff8103eafc>] ? __do_page_fault+0x15c/0x510
[   85.450359]  [<ffffffff8128e2f3>] ? blk_finish_plug+0x13/0x50
[   85.450359]  [<ffffffff810ffc99>] ? __do_page_cache_readahead+0x229/0x2b0
[   85.450359]  [<ffffffff810ffb25>] ? __do_page_cache_readahead+0xb5/0x2b0
[   85.450359]  [<ffffffff810ac927>] ? trace_hardirqs_off_caller+0x77/0x150
[   85.450359]  [<ffffffff812b55ad>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[   85.450359]  [<ffffffff8103eeb9>] ? do_page_fault+0x9/0x10
[   85.450359]  [<ffffffff817d20b8>] ? page_fault+0x28/0x30
[   85.450359]  [<ffffffff810f39a6>] ? file_read_actor+0x66/0x190
[   85.450359]  [<ffffffff81100230>] ? page_cache_async_readahead+0x80/0xa0
[   85.450359]  [<ffffffff810f4050>] ? find_get_pages+0x190/0x190
[   85.450359]  [<ffffffff810f6050>] ? generic_file_aio_read+0x2e0/0x6f0
[   85.450359]  [<ffffffff81144c82>] ? do_sync_read+0xa2/0xe0
[   85.450359]  [<ffffffff811459f5>] ? vfs_read+0xc5/0x190
[   85.450359]  [<ffffffff81145bca>] ? sys_read+0x5a/0xa0
[   85.450359]  [<ffffffff812b556e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[   85.450359]  [<ffffffff817d26a9>] ? system_call_fastpath+0x16/0x1b
[   85.450359] ---[ end trace ce03efc8ae5a5e97 ]---
[   85.650129] ------------[ cut here ]------------
[   85.650147] WARNING: at drivers/net/xen-netfront.c:465 
xennet_start_xmit+0x7fe/0x860()
[   85.650154] Modules linked in:
[   85.650161] Pid: 0, comm: swapper/0 Tainted: G        W    
3.7.0-rc5-20121116-vanilla-warn #1
[   85.650167] Call Trace:
[   85.650171]  <IRQ>  [<ffffffff8106750a>] warn_slowpath_common+0x7a/0xb0
[   85.650183]  [<ffffffff81067555>] warn_slowpath_null+0x15/0x20
[   85.650189]  [<ffffffff814b2b6e>] xennet_start_xmit+0x7fe/0x860
[   85.650196]  [<ffffffff816a6169>] dev_hard_start_xmit+0x209/0x460
[   85.650203]  [<ffffffff816c1e56>] sch_direct_xmit+0xf6/0x290
[   85.650208]  [<ffffffff816a6566>] dev_queue_xmit+0x1a6/0x5a0
[   85.650214]  [<ffffffff816a63c0>] ? dev_hard_start_xmit+0x460/0x460
[   85.650221]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.650227]  [<ffffffff81740436>] ip_finish_output+0x226/0x530
[   85.650234]  [<ffffffff817402dd>] ? ip_finish_output+0xcd/0x530
[   85.650240]  [<ffffffff81740799>] ip_output+0x59/0xe0
[   85.650245]  [<ffffffff8173f2b8>] ip_local_out+0x28/0x90
[   85.650251]  [<ffffffff8173f86f>] ip_queue_xmit+0x17f/0x4a0
[   85.650256]  [<ffffffff8173f6f0>] ? ip_send_unicast_reply+0x340/0x340
[   85.650262]  [<ffffffff810a1cf7>] ? getnstimeofday+0x47/0xe0
[   85.650268]  [<ffffffff816962c9>] ? __skb_clone+0x29/0x120
[   85.650274]  [<ffffffff81755940>] tcp_transmit_skb+0x400/0x8d0
[   85.650280]  [<ffffffff81758026>] tcp_retransmit_skb+0x1c6/0x5a0
[   85.650290]  [<ffffffff8175a300>] ? tcp_write_timer_handler+0x1a0/0x1a0
[   85.650296]  [<ffffffff81759e70>] tcp_retransmit_timer+0x370/0x660
[   85.650303]  [<ffffffff8175a29d>] tcp_write_timer_handler+0x13d/0x1a0
[   85.650309]  [<ffffffff8175a378>] tcp_write_timer+0x78/0x80
[   85.650315]  [<ffffffff81074f9c>] call_timer_fn+0x7c/0x100
[   85.650321]  [<ffffffff81074f20>] ? cascade+0xa0/0xa0
[   85.650326]  [<ffffffff8175a300>] ? tcp_write_timer_handler+0x1a0/0x1a0
[   85.650332]  [<ffffffff8175a300>] ? tcp_write_timer_handler+0x1a0/0x1a0
[   85.650338]  [<ffffffff81075237>] run_timer_softirq+0x217/0x250
[   85.650344]  [<ffffffff8106f3f9>] __do_softirq+0xc9/0x1a0
[   85.650350]  [<ffffffff817d393c>] call_softirq+0x1c/0x30
[   85.650356]  [<ffffffff8100fd95>] do_softirq+0x85/0xf0
[   85.650362]  [<ffffffff8106f26e>] irq_exit+0x9e/0xd0
[   85.650369]  [<ffffffff8134659f>] xen_evtchn_do_upcall+0x2f/0x40
[   85.650374]  [<ffffffff817d399e>] xen_do_hypervisor_callback+0x1e/0x30
[   85.650379]  <EOI>  [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
[   85.650389]  [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
[   85.650396]  [<ffffffff81008850>] ? xen_safe_halt+0x10/0x20
[   85.650402]  [<ffffffff810170f0>] ? default_idle+0x40/0x90
[   85.650408]  [<ffffffff810174a6>] ? cpu_idle+0x96/0xf0
[   85.650415]  [<ffffffff817a9c8c>] ? rest_init+0xbc/0xd0
[   85.650420]  [<ffffffff817a9bd0>] ? csum_partial_copy_generic+0x170/0x170
[   85.650428]  [<ffffffff81ce4be7>] ? start_kernel+0x390/0x39d
[   85.650433]  [<ffffffff81ce4677>] ? repair_env_string+0x5b/0x5b
[   85.650440]  [<ffffffff81ce4356>] ? x86_64_start_reservations+0x131/0x136
[   85.650446]  [<ffffffff81ce7915>] ? xen_start_kernel+0x54e/0x550
[   85.650451] ---[ end trace ce03efc8ae5a5e98 ]---
[   85.806018] ------------[ cut here ]------------
[   85.806044] WARNING: at drivers/net/xen-netfront.c:465 
xennet_start_xmit+0x7fe/0x860()
[   85.806051] Modules linked in:
[   85.806058] Pid: 2019, comm: postgres Tainted: G        W    
3.7.0-rc5-20121116-vanilla-warn #1
[   85.806065] Call Trace:
[   85.806073]  [<ffffffff8106750a>] warn_slowpath_common+0x7a/0xb0
[   85.806080]  [<ffffffff81067555>] warn_slowpath_null+0x15/0x20
[   85.806086]  [<ffffffff814b2b6e>] xennet_start_xmit+0x7fe/0x860
[   85.806103]  [<ffffffff816a6169>] dev_hard_start_xmit+0x209/0x460
[   85.806109]  [<ffffffff816c1e56>] sch_direct_xmit+0xf6/0x290
[   85.806114]  [<ffffffff816a6566>] dev_queue_xmit+0x1a6/0x5a0
[   85.806119]  [<ffffffff816a63c0>] ? dev_hard_start_xmit+0x460/0x460
[   85.806125]  [<ffffffff810abab5>] ? trace_softirqs_off+0x85/0x1b0
[   85.806132]  [<ffffffff81740436>] ip_finish_output+0x226/0x530
[   85.806138]  [<ffffffff817402dd>] ? ip_finish_output+0xcd/0x530
[   85.806143]  [<ffffffff81740799>] ip_output+0x59/0xe0
[   85.806148]  [<ffffffff8173f2b8>] ip_local_out+0x28/0x90
[   85.806153]  [<ffffffff8173f86f>] ip_queue_xmit+0x17f/0x4a0
[   85.806157]  [<ffffffff8173f6f0>] ? ip_send_unicast_reply+0x340/0x340
[   85.806163]  [<ffffffff810a1cf7>] ? getnstimeofday+0x47/0xe0
[   85.806168]  [<ffffffff816962c9>] ? __skb_clone+0x29/0x120
[   85.806174]  [<ffffffff81755940>] tcp_transmit_skb+0x400/0x8d0
[   85.806179]  [<ffffffff8175891a>] tcp_write_xmit+0x21a/0xa50
[   85.806184]  [<ffffffff8175917b>] tcp_push_one+0x2b/0x40
[   85.806190]  [<ffffffff81749cdc>] tcp_sendmsg+0x8dc/0xe20
[   85.806195]  [<ffffffff8176fe09>] inet_sendmsg+0xa9/0x100
[   85.806200]  [<ffffffff8176fd60>] ? inet_autobind+0x70/0x70
[   85.806206]  [<ffffffff8168dd12>] sock_sendmsg+0x82/0xb0
[   85.806211]  [<ffffffff8168ddb5>] ? sockfd_lookup_light+0x25/0x80
[   85.806216]  [<ffffffff8168e5e4>] sys_sendto+0x104/0x140
[   85.806223]  [<ffffffff817d26a9>] system_call_fastpath+0x16/0x1b
[   85.806227] ---[ end trace ce03efc8ae5a5e99 ]---
[   85.806855] ------------[ cut here ]------------
[   85.806871] WARNING: at drivers/net/xen-netfront.c:465 
xennet_start_xmit+0x7fe/0x860()
[   85.806879] Modules linked in:
[   85.806884] Pid: 2019, comm: postgres Tainted: G        W    
3.7.0-rc5-20121116-vanilla-warn #1
[   85.806891] Call Trace:
[   85.806898]  [<ffffffff8106750a>] warn_slowpath_common+0x7a/0xb0
[   85.806905]  [<ffffffff81067555>] warn_slowpath_null+0x15/0x20
[   85.806911]  [<ffffffff814b2b6e>] xennet_start_xmit+0x7fe/0x860
[   85.806917]  [<ffffffff816a6169>] dev_hard_start_xmit+0x209/0x460
[   85.806923]  [<ffffffff816c1e56>] sch_direct_xmit+0xf6/0x290
[   85.806929]  [<ffffffff816a6566>] dev_queue_xmit+0x1a6/0x5a0
[   85.806935]  [<ffffffff816a63c0>] ? dev_hard_start_xmit+0x460/0x460
[   85.806941]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.806947]  [<ffffffff81740436>] ip_finish_output+0x226/0x530
[   85.806953]  [<ffffffff817402dd>] ? ip_finish_output+0xcd/0x530
[   85.806959]  [<ffffffff81740799>] ip_output+0x59/0xe0
[   85.806964]  [<ffffffff8173f2b8>] ip_local_out+0x28/0x90
[   85.806970]  [<ffffffff8173f86f>] ip_queue_xmit+0x17f/0x4a0
[   85.806975]  [<ffffffff8173f6f0>] ? ip_send_unicast_reply+0x340/0x340
[   85.806982]  [<ffffffff810a1cf7>] ? getnstimeofday+0x47/0xe0
[   85.806987]  [<ffffffff816962c9>] ? __skb_clone+0x29/0x120
[   85.806993]  [<ffffffff81755940>] tcp_transmit_skb+0x400/0x8d0
[   85.806999]  [<ffffffff81758026>] tcp_retransmit_skb+0x1c6/0x5a0
[   85.807005]  [<ffffffff81094806>] ? finish_task_switch+0x46/0xe0
[   85.807011]  [<ffffffff8175859e>] tcp_xmit_retransmit_queue+0x19e/0x300
[   85.807018]  [<ffffffff8175070b>] tcp_fastretrans_alert+0x76b/0xcb0
[   85.807024]  [<ffffffff817515fc>] tcp_ack+0x9ac/0x1150
[   85.807030]  [<ffffffff81752663>] ? tcp_validate_incoming+0x2d3/0x350
[   85.807036]  [<ffffffff81754778>] tcp_rcv_established+0x318/0x640
[   85.807042]  [<ffffffff8106f776>] ? local_bh_enable+0xc6/0x160
[   85.807048]  [<ffffffff8175c3a5>] tcp_v4_do_rcv+0x135/0x480
[   85.807054]  [<ffffffff8106f08f>] ? __local_bh_disable+0x8f/0xe0
[   85.807061]  [<ffffffff817483db>] tcp_prequeue_process+0x6b/0xa0
[   85.807067]  [<ffffffff81748e40>] tcp_recvmsg+0x8f0/0xb70
[   85.807072]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.807078]  [<ffffffff8176ec2a>] inet_recvmsg+0xba/0x100
[   85.807084]  [<ffffffff8176fd60>] ? inet_autobind+0x70/0x70
[   85.807090]  [<ffffffff8168dbf9>] sock_recvmsg+0x89/0xc0
[   85.807096]  [<ffffffff81116b0f>] ? tlb_flush_mmu+0x5f/0xa0
[   85.807102]  [<ffffffff810b171b>] ? __lock_acquire+0x46b/0xdd0
[   85.807108]  [<ffffffff8168ddb5>] ? sockfd_lookup_light+0x25/0x80
[   85.807114]  [<ffffffff8168e45f>] sys_recvfrom+0xbf/0x120
[   85.807120]  [<ffffffff811144db>] ? might_fault+0x3b/0x90
[   85.807126]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.807133]  [<ffffffff817d26a9>] system_call_fastpath+0x16/0x1b
[   85.807138] ---[ end trace ce03efc8ae5a5e9a ]---
[   85.807159] ------------[ cut here ]------------
[   85.807165] WARNING: at drivers/net/xen-netfront.c:465 
xennet_start_xmit+0x7fe/0x860()
[   85.807171] Modules linked in:
[   85.807176] Pid: 2019, comm: postgres Tainted: G        W    
3.7.0-rc5-20121116-vanilla-warn #1
[   85.807182] Call Trace:
[   85.807187]  [<ffffffff8106750a>] warn_slowpath_common+0x7a/0xb0
[   85.807193]  [<ffffffff81067555>] warn_slowpath_null+0x15/0x20
[   85.807199]  [<ffffffff814b2b6e>] xennet_start_xmit+0x7fe/0x860
[   85.807205]  [<ffffffff816a6169>] dev_hard_start_xmit+0x209/0x460
[   85.807211]  [<ffffffff816c1e56>] sch_direct_xmit+0xf6/0x290
[   85.807216]  [<ffffffff816a6566>] dev_queue_xmit+0x1a6/0x5a0
[   85.807222]  [<ffffffff816a63c0>] ? dev_hard_start_xmit+0x460/0x460
[   85.807228]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.807233]  [<ffffffff81740436>] ip_finish_output+0x226/0x530
[   85.807239]  [<ffffffff817402dd>] ? ip_finish_output+0xcd/0x530
[   85.807245]  [<ffffffff81740799>] ip_output+0x59/0xe0
[   85.807250]  [<ffffffff8173f2b8>] ip_local_out+0x28/0x90
[   85.807256]  [<ffffffff8173f86f>] ip_queue_xmit+0x17f/0x4a0
[   85.807261]  [<ffffffff8173f6f0>] ? ip_send_unicast_reply+0x340/0x340
[   85.807267]  [<ffffffff810a1cf7>] ? getnstimeofday+0x47/0xe0
[   85.807272]  [<ffffffff816962c9>] ? __skb_clone+0x29/0x120
[   85.807278]  [<ffffffff81755940>] tcp_transmit_skb+0x400/0x8d0
[   85.807283]  [<ffffffff81758026>] tcp_retransmit_skb+0x1c6/0x5a0
[   85.810124]  [<ffffffff81094806>] ? finish_task_switch+0x46/0xe0
[   85.810124]  [<ffffffff8175859e>] tcp_xmit_retransmit_queue+0x19e/0x300
[   85.810124]  [<ffffffff8175070b>] tcp_fastretrans_alert+0x76b/0xcb0
[   85.810124]  [<ffffffff817515fc>] tcp_ack+0x9ac/0x1150
[   85.810124]  [<ffffffff81752663>] ? tcp_validate_incoming+0x2d3/0x350
[   85.810124]  [<ffffffff81754778>] tcp_rcv_established+0x318/0x640
[   85.810124]  [<ffffffff8106f776>] ? local_bh_enable+0xc6/0x160
[   85.810124]  [<ffffffff8175c3a5>] tcp_v4_do_rcv+0x135/0x480
[   85.810124]  [<ffffffff8106f08f>] ? __local_bh_disable+0x8f/0xe0
[   85.810124]  [<ffffffff817483db>] tcp_prequeue_process+0x6b/0xa0
[   85.810124]  [<ffffffff81748e40>] tcp_recvmsg+0x8f0/0xb70
[   85.810124]  [<ffffffff810b25e7>] ? lock_release+0x117/0x250
[   85.810124]  [<ffffffff8176ec2a>] inet_recvmsg+0xba/0x100
[   85.810124]  [<ffffffff8176fd60>] ? inet_autobind+0x70/0x70
[   85.810124]  [<ffffffff8168dbf9>] sock_recvmsg+0x89/0xc0
[   85.810124]  [<ffffffff81116b0f>] ? tlb_flush_mmu+0x5f/0xa0
[   85.8


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.