[Xen-users] HOWTO: AOE in domU and boot from it.

[tbrown@xxxxxxxxxxxxx]

I know there was some discussion on this list a short while back where
someone was advising putting the AOE "initiator" in dom0. I don't like
that solution for a number of reasons... two of them being that pushing
the filesystem complexity into the domU makes the dom0 more stable _and_
makes domU migration trivial.

I just finished spending at least 1/2 a day trying to get a simple initrd
setup so I could boot a linux virtual machine via AOE. I had done it for
nbd, and enbd, so I thought it would be fairly simple for AOE.

Unfortunately, I could not figure out why the discovery process was not
working. Turns out that I had forgotten that I needed to bring UP the
ethernet interface first.

it is covered on
http://www.coraid.com/support/linux/contrib/vantuyl/aoeboot.html

but that page is _way_ longer than it needs to be... at least for my
taste.

On a Centos (redhat enterprise linux clone) system, the steps to build an
initrd to make it work are fairly simple. And are covered below:

--------------------------------

We can leverage the system mkinitrd to great advantage.
Basically, we run the system mkinitrd to generate a starting
boot image, then we take it apart, make some small changes, and
reassemble it. The changes we need are basically 3:
 1) ifconfig eth0 up
 2) create a /dev/etherd/discover device node
 3) write to the discover node to trigger an immediate discover


   [root@xen4 ~/tmp]# uname -r
   2.6.16-xen0
   [root@xen4 ~/tmp]# mkinitrd --nocompress tmpx `uname -r`
   [root@xen4 ~/tmp]# mkdir initrd
   [root@xen4 ~/tmp]# cd initrd
   [root@xen4 ~/tmp/initrd]# cpio -i < ../tmpx
   3144 blocks

lets have a look!

   [root@xen4 ~/tmp/initrd]# ls -F
   bin/  dev/  etc/  init*  lib/  loopfs/  proc/  sbin@  sys/  sysroot/

ok, we now have a directory tree with a good init file as
a starting point and all the appropriate support utilities...
way cool!  We need to add busybox to our toolkit, because
we need to run /bin/busybox ifconfig eth0 up  to get the
aoe discover to run. Use "yum install busybox" if the "which
busybox" command below returns a "no busybox in ..." error


   [root@xen4 ~/tmp/initrd]# which busybox
   /sbin/busybox
   [root@xen4 ~/tmp/initrd]# ldd /sbin/busybox
           not a dynamic executable
   [root@xen4 ~/tmp/initrd]# cp /sbin/busybox bin/

That ldd check is just to prove it is a statically linked file,
as dynamically linked files will not work in an initrd.

We also need to modify the init script.

OK. So my changes to the init file are to add the following commands
ahead of the "Creating root device" line... since we need to have
our root device available by the time we get to the mkrootdev line


   #TAB mods to get aoe up and running.
   # first, bring up the ethernet interface so aoe discover _can_ run

   echo /bin/busyybox ifconfig eth0 up
   /bin/busybox ifconfig eth0 up

   # give link a chance to autonegotiate etc...
   echo pausing to let ethernet settle
   sleep 2

   #next, create device nodes to trigger discovery
   mkdir /dev/etherd
   /bin/busybox rm -f /dev/etherd/discover
   mknod /dev/etherd/discover c 152 3

   echo touching /dev/etherd/discover
   echo hello world > /dev/etherd/discover

   echo pausing to aoe discovery run
   sleep 5
   echo done aoe setup... cross your fingers
   echo
   #/TAB

That fixes the init script. So we need to "burn" this into an image file
we can use for an initrd... That's easy, we'll just copy the command
used by the mkinitrd script

   find . | cpio -c -o | gzip -9 >  /boot/img.aoe

done. Boot it!

My target environment is XEN, and the relevent parameters to be changed are:

        ramdisk = "/boot/img.aoe"
        root = "/dev/etherd/e9.0 ro"

(or whatever your aoe target happens to be... I think that was the vblade
default example).

MODULES:

I avoid using kernel modules to keep life simple, and using modules means
you have to update the image file if you change kernels. If you want to
use the kernel module, we can leverage mkinitrd for that too.. It just
means you need to specify a "--preload aoe" option, and you have to be
more carefull to get the kernel version parameter correct for your target
kernel. Change the "mkinitrd --nocompress tmpx `uname -r`" command to be

   mkinitrd --nocompress --preload aoe tmpx `uname -r`

or if you're using a different target kernel, put that in where I have
`uname -r` (but don't use the back quotes... e.g. 2.6.16-xenU

-Tom











_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users