Re: [Xen-devel] GPU passthrough issue when VM is configured with 4G memory

[Hanweidong <hanweidong@xxxxxxxxxx>]

I assigned the nvidia quadro 2000 card as the normal pci pass-through instead of VGA pass-through. The cardâs mmio size is about 256M. whatâs the mmio size of your ATI card?

 

-weidong

 

From: David TECHER [mailto:davidtecher@xxxxxxxx]
Sent: 2013å4æ16æ 17:22
To: Pasi KÃrkkÃinen
Cc: Stefano Stabellini; George Dunlap; xudong.hao@xxxxxxxxx; Yanqiangjun; Luonengjun; Wangzhenguo; Yangxiaowei; Gonglei (Arei); Anthony Perard; xen-devel@xxxxxxxxxxxxx; Hanweidong; xiantao.zhang@xxxxxxxxx
Subject: Re: [Xen-devel] GPU passthrough issue when VM is configured with 4G memory

 

Hi Pasi,

FYI and to be sure that we understand each other  I just applied the following patches in order

- to have VGA passthrough with ATI : ftp://ftp.enjellic.com/pub/xen/xen-4.2.0.ati-passthrough.patch

- to have more that 4GB of RAM:  http://marc.info/?l=qemu-devel&m=136177475215360

Here are the details (a simple copy-paste from the prior link)..As previously noticed I have to build studom as latest step by commenting out any call to iopl() function.
To be honest I think the method I used could be improved. Unfortunately I am not a developer.

Download Xen sources
====================
rev=267773
hg clone -r $rev http://xenbits.xensource.com/staging/xen-unstable.hg/ xen-unstable.hg-rev-XXXXX
cd xen-unstable.hg-rev-XXXXX

Configure
=========
CURL=$(which curl-config) XML=$(which xml2-config) ./configure

Make a first build for tools and cleanup the folder
===================================================
 *   Make a first build for tools

    cd tools
    make -j4

 *  Clean up the folder

    make clean

Download and apply the patches
==============================
 *   Download and apply the 1st patch (patch for RAM > 3GB)

    cd qemu-xen-dir-remote/
    wget "http://marc.info/?l=qemu-devel&m=136177475215360&q=raw" -O - | patch -p1

 *   Download and apply the 2nd patch (patch for ATI)

    cd ../..
    wget ftp://ftp.enjellic.com/pub/xen/xen-4.2.0.ati-passthrough.patch -O - | sed -e "s:qemu-xen-traditional:qemu-xen-traditional-dir-remote:g" | patch -p1

 *   We have to modify tools/qemu-xen-traditional-dir-remote/hw/pt-graphics.c file so we can run build stubdom later

    for i in 0 3;do sed -i "s:^.*iopl(${i});$://iopl(${i});:g" tools/qemu-xen-traditional-dir-remote/hw/pt-graphics.c;done

    NOTICE if this workaround is not applied then you should have this error

    /opt/tmp/xen-unstable.hg-rev-XXXXX/stubdom/mini-os-x86_64-ioemu/mini-os.o: In function `ati_hw_out':
    /opt/tmp/xen-unstable.hg-rev-XXXXX/stubdom/ioemu/hw/pt-graphics.c:82: undefined reference to `iopl'
    /opt/tmp/xen-unstable.hg-rev-XXXXX/stubdom/ioemu/hw/pt-graphics.c:84: undefined reference to `iopl'
    /opt/tmp/xen-unstable.hg-rev-XXXXX/stubdom/mini-os-x86_64-ioemu/mini-os.o: In function `ati_hw_in':
    /opt/tmp/xen-unstable.hg-rev-XXXXX/stubdom/ioemu/hw/pt-graphics.c:72: undefined reference to `iopl'
    /opt/tmp/xen-unstable.hg-rev-XXXXX/stubdom/ioemu/hw/pt-graphics.c:74: undefined reference to `iopl'

Build and install xen and stubdom
=================================
  *  Build

    make -j4 xen && make -j4 stubdom

  *  Install

    make install-xen && make install-stubdom

Build and install tools
=======================
  *  Cleanup tools

    cd tools
    make clean

 *  Reverse the workaround by commenting out any call to iopl() function

    cd ..
    for i in 0 3;do sed -i "s:^.*iopl(${i});$:iopl(${i});:g" tools/qemu-xen-traditional-dir-remote/hw/pt-graphics.c;done

 *   Build and install tools

    make -j4 tools
    make install-tools PYTHON_PREFIX_ARG=

 

 

---

De : Pasi KÃrkkÃinen <pasik@xxxxxx>
à: David TECHER <davidtecher@xxxxxxxx>
Cc : Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>; George Dunlap <George.Dunlap@xxxxxxxxxxxxx>; "xudong.hao@xxxxxxxxx" <xudong.hao@xxxxxxxxx>; Yanqiangjun <yanqiangjun@xxxxxxxxxx>; Luonengjun <luonengjun@xxxxxxxxxx>; Wangzhenguo <wangzhenguo@xxxxxxxxxx>; Yangxiaowei <xiaowei.yang@xxxxxxxxxx>; Gonglei (Arei) <arei.gonglei@xxxxxxxxxx>; Anthony Perard <anthony.perard@xxxxxxxxxx>; "xen-devel@xxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxx>; Hanweidong <hanweidong@xxxxxxxxxx>; "xiantao.zhang@xxxxxxxxx" <xiantao.zhang@xxxxxxxxx>
Envoyà le : Mardi 16 avril 2013 5h54
Objet : Re: [Xen-devel] GPU passthrough issue when VM is configured with 4G memory

On Tue, Apr 16, 2013 at 01:44:23AM +0100, David TECHER wrote:
>    Those patches works fine for me with my ATI card both for Windows 7 64 bit
>    or Linux Mint 14 64-Bit.
>
>    Both were tested with 6GB and 8GB of RAM and vcpus = 6
>

Hmm.. so the patches work OK for David, but fail for Weidong.
I wonder what's the difference in your setups?

And thanks David for testing!


-- Pasi

>    To do 'make stubdom' I have to find a way to neuter iopl() call from
>    tools/qemu-xen-traditional-dir-remote/hw/pt-graphics.c
>
>    All details provided here
>    http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
>
>    --------------------------------------------------------------------------
>
>    De : Pasi KÃrkkÃinen <pasik@xxxxxx>
>    Ã : Hanweidong <hanweidong@xxxxxxxxxx>
>    Cc : Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>; George Dunlap
>    <George.Dunlap@xxxxxxxxxxxxx>; "xudong.hao@xxxxxxxxx"
>    <xudong.hao@xxxxxxxxx>; Yanqiangjun <yanqiangjun@xxxxxxxxxx>; Luonengjun
>    <luonengjun@xxxxxxxxxx>; Wangzhenguo <wangzhenguo@xxxxxxxxxx>; Yangxiaowei
>    <xiaowei.yang@xxxxxxxxxx>; Gonglei (Arei) <arei.gonglei@xxxxxxxxxx>;
>    Anthony Perard <anthony.perard@xxxxxxxxxx>; "xen-devel@xxxxxxxxxxxxx"
>    <xen-devel@xxxxxxxxxxxxx>; "xiantao.zhang@xxxxxxxxx"
>    <xiantao.zhang@xxxxxxxxx>
>    Envoyà le : Lundi 15 avril 2013 23h22
>    Objet : Re: [Xen-devel] GPU passthrough issue when VM is configured with
>    4G memory
>    On Tue, Mar 26, 2013 at 09:37:53AM +0000, Hanweidong wrote:
>    > > >
>    > > > HVM_BELOW_4G_RAM_END is f0000000. If we change HVM_BELOW_4G_RAM_END
>    > > to e0000000,
>    > > > Which it's consistent with hvmloader when assigning a GPU, and then
>    > > guest worked
>    > > > for us. So we wondering that xen_ram_init in QEMU should be
>    > > consistent with
>    > > > hvmloader.
>    > > >
>    > > > In addition, we found QEMU uses hardcode 0xe0000000 in pc_init1() as
>    > > below.
>    > > > Should keep these places handle the consistent mmio hole or not?
>    > > >
>    > > >    if (ram_size >= 0xe0000000 ) {
>    > > >        above_4g_mem_size = ram_size - 0xe0000000;
>    > > >        below_4g_mem_size = 0xe0000000;
>    > > >    } else {
>    > > >        above_4g_mem_size = 0;
>    > > >        below_4g_mem_size = ram_size;
>    > > >    }
>    > >
>    > > The guys at Intel sent a couple of patches recently to fix this issue:
>    > >
>    > > [1]http://marc.info/?l=xen-devel&m=136150317011027
>    > > [2]http://marc.info/?l=qemu-devel&m=136177475215360&w=2
>    > >
>    > > Do they solve your problem?
>    >
>    > These two patches didn't solve our problem.
>    >
>
>    Any updates on this? It'd be nice to get this fixed before Xen 4.3.
>
>    Thanks,
>
>    -- Pasi
>
>    > --weidong
>    >
>    > >
>    > > Xudong and Xiantao,
>    > > are you going to send an update of the second patch to QEMU?
>
>    _______________________________________________
>    Xen-devel mailing list
>    [3]Xen-devel@xxxxxxxxxxxxx
>    [4]http://lists.xen.org/xen-devel
>
> References
>
>    Visible links
>    1. http://marc.info/?l=xen-devel&m=136150317011027
>    2. http://marc.info/?l=qemu-devel&m=136177475215360&w=2
>    3. mailto:Xen-devel@xxxxxxxxxxxxx
>    4. http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel