Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver

[Boris Derzhavets <bderzhavets@xxxxxxxxx>]

Following  README i attemped :-

root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# make SYSSRC=/usr/src/linux-2.6-xen module
. . . . . . . .
  ld -r -m elf_x86_64  --build-id -o /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.ko /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.o /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.mod.o
NVIDIA: left KBUILD.
root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# make SYSSRC=/usr/src/linux-2.6-xen module
root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# ls -l
total 40416
-rw-r--r-- 1 root root     1148 2009-07-15 01:05 conftest.h
-rw-r--r-- 1 root root    49341 2009-05-27 13:02 conftest.sh
-rw-r--r-- 1 root root     8517 2009-05-27 13:02 cpuopsys.h
-rw-r--r-- 1 root root     9077 2009-05-27 13:02 gcc-version-check.c
-rw-r--r-- 1 root root     1834 2009-05-27 13:02 makefile
lrwxrwxrwx 1 root root       15 2009-07-15 01:05 Makefile -> Makefile.kbuild
-rw-r--r-- 1 root root     9174 2009-05-27 13:02 Makefile.kbuild
-rw-r--r-- 1 root root     3726 2009-05-27 13:02 Makefile.nvidia
-rw-r--r-- 1 root root       45 2009-07-15 01:05 Module.markers
-rw-r--r-- 1 root root       72 2009-07-15 01:05 modules.order
-rw-r--r-- 1 root root      104 2009-07-15 01:05 Module.symvers
-rw-r--r-- 1 root root    33436 2009-05-27 13:02 nvacpi.c
-rw-r--r-- 1 root root   178616 2009-07-15 01:05 nvacpi.o
-rw-r--r-- 1 root root   131773 2009-05-27 13:02 nv.c
-rw-r--r-- 1 root root       65 2009-07-15 01:05 nv_compiler.h
-rw-r--r-- 1 root root    23476 2009-05-27 13:02 nv.h
-rw-r--r-- 1 root root     9980 2009-05-27 13:02 nv-i2c.c
-rw-r--r-- 1 root root   161448 2009-07-15 01:05 nv-i2c.o
++++++++++++++++++++++++++++++++++++++
-rw-r--r-- 1 root root 13577047 2009-07-15 01:05 nvidia.ko
++++++++++++++++++++++++++++++++++++++
-rw-r--r-- 1 root root     5268 2009-07-15 01:05 nvidia.mod.c
-rw-r--r-- 1 root root    53264 2009-07-15 01:05 nvidia.mod.o
-rw-r--r-- 1 root root 13525278 2009-07-15 01:05 nvidia.o
-rw-r--r-- 1 root root 12118040 2009-05-27 13:02 nv-kernel.o
-rw-r--r-- 1 root root    50152 2009-05-27 13:02 nv-linux.h
-rw-r--r-- 1 root root     2967 2009-05-27 13:02 nv-memdbg.h
-rw-r--r-- 1 root root      714 2009-05-27 13:02 nv-misc.h
-rw-r--r-- 1 root root   403968 2009-07-15 01:05 nv.o
-rw-r--r-- 1 root root     5072 2009-05-27 13:02 nvreadme.h
-rw-r--r-- 1 root root    19284 2009-05-27 13:02 nv-reg.h
-rw-r--r-- 1 root root    12883 2009-05-27 13:02 nvtypes.h
-rw-r--r-- 1 root root    23062 2009-05-27 13:02 nv-vm.c
-rw-r--r-- 1 root root      768 2009-05-27 13:02 nv-vm.h
-rw-r--r-- 1 root root   158584 2009-07-15 01:05 nv-vm.o
-rw-r--r-- 1 root root     9249 2009-05-27 13:02 os-agp.c
-rw-r--r-- 1 root root      804 2009-05-27 13:02 os-agp.h
-rw-r--r-- 1 root root   156520 2009-07-15 01:05 os-agp.o
-rw-r--r-- 1 root root    31069 2009-05-27 13:02 os-interface.c
-rw-r--r-- 1 root root     8785 2009-05-27 13:02 os-interface.h
-rw-r--r-- 1 root root   212096 2009-07-15 01:05 os-interface.o
-rw-r--r-- 1 root root     2395 2009-05-27 13:02 os-registry.c
-rw-r--r-- 1 root root   160400 2009-07-15 01:05 os-registry.o
-rw-r--r-- 1 root root      157 2009-07-15 01:05 patches.h

Boris.

--- On Tue, 7/14/09, Boris Derzhavets <bderzhavets@xxxxxxxxx> wrote:

From: Boris Derzhavets <bderzhavets@xxxxxxxxx>
Subject: Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
To: "Jeremy Fitzhardinge" <jeremy@xxxxxxxx>
Cc: "Michael Ralston" <michael@xxxxxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, xen-users@xxxxxxxxxxxxxxxxxxx
Date: Tuesday, July 14, 2009, 4:57 PM

  By running "sh NVIDIA-Linux-x86_64-185.18.14-pkg2.run -x"
i was able to get some extraction in folder bellow. It's hard for me to decide could it be patched similar schema in http://fluxcoil.net/doku.php/xen/docs
for xenified kernel or no.
  Kernels 2.6.29.4 , 2.6.30.1 xenified via rebased patch set from Andy
do contain patched driver for GeForce 8000 series . It just works fine.

Nvidia driver extraction :-

root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2# ls -CR
.:
LICENSE  nvidia-installer  pkg-history.txt  usr

./usr:
bin  include  lib  lib32  share  src  X11R6

./usr/bin:
makeself.sh    nvidia-bug-report.sh  nvidia-smi      tls_test
mkprecompiled  nvidia-settings       nvidia-xconfig  tls_test_dso.so

./usr/include:
cuda  GL  vdpau

./usr/include/cuda:
cudaGL.h  cuda.h

./usr/include/GL:
glext.h  gl.h  glxext.h  glx.h

./usr/include/vdpau:
vdpau.h  vdpau_x11.h

./usr/lib:
libcuda.so.185.18.14    libGL.so.185.18.14          libvdpau_nvidia.so.185.18.14  tls
libGLcore.so.185.18.14  libnvidia-cfg.so.185.18.14  libvdpau.so.185.18.14
libGL.la                libnvidia-tls.so.185.18.14  libvdpau_trace.so.185.18.14

./usr/lib/tls:
libnvidia-tls.so.185.18.14

./usr/lib32:
libcuda.so.185.18.14    libGL.la            libnvidia-tls.so.185.18.14    libvdpau.so.185.18.14        tls
libGLcore.so.185.18.14  libGL.so.185.18.14  libvdpau_nvidia.so.185.18.14  libvdpau_trace.so.185.18.14

./usr/lib32/tls:
libnvidia-tls.so.185.18.14

./usr/share:
applications  doc  man  pixmaps

./usr/share/applications:
nvidia-settings.desktop

./usr/src:
nv

./usr/src/nv:
conftest.sh   Makefile.kbuild  nv.h  nv-memdbg.h  nvtypes.h  os-agp.h    README   cpuopsys.h           Makefile.nvidia  nv-i2c.c     nv-misc.h   nv-vm.c    os-interface.c  rmretval.h gcc-version-check.c  nvacpi.c     nv-kernel.o  nvreadme.h   nv-vm.h    os-interface.h
makefile   nv.c   nv-linux.h   nv-reg.h     os-agp.c   os-registry.c

./usr/X11R6:
lib

./usr/X11R6/lib:
libXvMCNVIDIA.a  libXvMCNVIDIA.so.185.18.14  modules

./usr/X11R6/lib/modules:
drivers  extensions  libnvidia-wfb.so.185.18.14

./usr/X11R6/lib/modules/drivers:
nvidia_drv.so

./usr/X11R6/lib/modules/extensions:
libglx.so.185.18.14

Boris.

--- On Tue, 7/14/09, Jeremy Fitzhardinge <jeremy@xxxxxxxx> wrote:

From: Jeremy Fitzhardinge <jeremy@xxxxxxxx>
Subject: Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
To: "Boris Derzhavets" <bderzhavets@xxxxxxxxx>
Cc: xen-users@xxxxxxxxxxxxxxxxxxx, "Michael Ralston" <michael@xxxxxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx
Date: Tuesday, July 14, 2009, 3:52 PM

On 07/11/09 23:03, Boris Derzhavets wrote:
> A while ago Jeremy responded the same question.
> Per Jeremy:-  unmodified binary Nvidia driver would crash under Xen.
> Modified driver should in the pvops kernel.
>
> It's also my major concern as well.
>
> Boris
>
> --- On *Sun, 7/12/09, Michael Ralston /<michael@xxxxxxxxxxxxx>/* wrote:
>
>
>     From: Michael Ralston <michael@xxxxxxxxxxxxx>
>     Subject: [Xen-users] pv_ops kernel and nvidia binary driver
>     To: xen-users@xxxxxxxxxxxxxxxxxxx
>     Date: Sunday, July 12, 2009, 1:48 AM
>
>     Just wondering what it will take to get the nvidia binary driver
>     working on a pv_ops kernel.
>
>     It makes it difficult to debug without the source to the nvidia
>     driver, but I think it should be possible to get it to work without
>     changing the binary driver. If the dom0 kernel had access to all the
>     resources that a bare metal kernel did, then it should work right?
>

The problem comes down to whether the nvidia driver assumes the kernel's
(pseudo-)physical addresses are really machine physical or not.  If it
doesn't do the appropriate conversions between physical and machine
addresses using the standard Linux DMA API (or similar), then it will
end up misprogramming the hardware and reading/writing random memory.
There's not a lot we can do about that if that happens within the binary
part of the nvidia driver.  If the binary code calls out to the
source-available parts of the driver to do those conversions, then it
would be possible to fix there.

There are other potential pitfalls the nvidia driver could be falling
into, but that's the most likely.

    J

-----Inline Attachment Follows-----

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users