RE: [Xen-devel] Writing to a ramdisk in a PV domain is SLLLOOOWWW?!?

["Santos, Jose Renato G" <joserenato.santos@xxxxxx>]

This is complete wild guess, but could it be that you are doing multi-page I/O 
with non contiguous machine pages?
I believe the DMA API for pv guests uses swioltlb which does extra copies to a 
bounce buffer if you request a dma address to a multi-page region (as the pages 
may be non contiguous).

Renato 

> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of 
> Dan Magenheimer
> Sent: Thursday, May 06, 2010 2:02 PM
> To: Jeremy Fitzhardinge
> Cc: Xen-Devel (xen-devel@xxxxxxxxxxxxxxxxxxx)
> Subject: RE: [Xen-devel] Writing to a ramdisk in a PV domain 
> is SLLLOOOWWW?!?
> 
> > Subject: Re: [Xen-devel] Writing to a ramdisk in a PV domain is 
> > SLLLOOOWWW?!?
> > 
> > > Writing to the ramdisk appears to be VERY VERY slow, 
> elapsed time in 
> > > the guest is several times larger than
> > > user+sys, and xentop shows the guest consuming vcpu
> > > seconds at about the user+sys rate.  Note that this is 
> when tmem is 
> > > turned off and there is no vhd swap disk configured.
> > >
> > > I'm suspecting that writing to ramdisk must be causing some 
> > > interesting/expensive PV pagetable behavior?
> > > Or maybe somehow /dev/ram0 is getting routed through qemu?  Or ??
> > >
> > 
> > I haven't looked at ramdisk, but I'm pretty sure there's nothing
> > special
> > about accessing it.  The only thing I can think of is that if you're
> > using a 32bit highmem system then you may be being hit by 
> lots of kmap
> > overhead.  But on a 64-bit system, AFAIK, it should just be memory
> > copies.
> 
> Thanks for the reply Jeremy.
> 
> I tried a 64-bit guest (and bare-metal) and saw the same problem.
> I guess I'll start collecting some statistics.
> 
> Maybe since ramdisk (via the swap code) is still going
> through the blockio layer, there is some kind of cache or
> TLB overhead that normally would only be necessary for
> memory used for DMA?  (I don't know much about I/O so
> this is pure speculation.)  Or maybe there is an assumption
> that since blockio is asynchronous, a timer is set to
> some minimum value and the write-to-ramdisk isn't completed
> until the timer fires?  (Hmmm... but this wouldn't explain
> why virtual is much worse than bare-metal.)
> 
> Dan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel