RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

[Joshua Kinard <joshua.kinard@xxxxxxxxxxxxx>]

Pretty familiar with kernel building, actually (done a bit of work on SGI MIPS 
platforms running Linux).  If you're refering to my Debian issue, that's more 
just figuring out how to get their package manager to "trust" my local 
repository so I can tweak the Xen LiveCD building scripts to pull packages 
locally to build stuff.  Not too much of a hassle, just a matter of some 
tinkering around.

Thanks though!,

--J

________________________________________
From: Robbie garrett [rgarrett@xxxxxxxxxxxxxx]
Sent: Thursday, August 20, 2009 8:07 AM
To: John Lewis; Joshua Kinard
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

try a make KERNCONF=XEN menuconfig

then you should see some xen options under the CPU option.

then isssue a make KERNCONF=XEN to make your kernel.

----- Original Message -----
From: "John Lewis" <jlewis@xxxxxxxxxxxxxx>
To: "Joshua Kinard" <joshua.kinard@xxxxxxxxxxxxx>
Cc: <xen-users@xxxxxxxxxxxxxxxxxxx>
Sent: Wednesday, August 19, 2009 1:24 PM
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?


> Two questions, probably the same answer. I am just trying to  understand
> why you are trying to do this, and what practical use this  has. Maybe I
> will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don't think qcow format is going to work with that method.   At
>> least when I tried it, the bochs bios didn't like what it was  looking
>> at.  But it's hard to tell if that was bochs, or ntldr  giving me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it  with
>> squashfs by running mksquashfs on the base image, and mounting  that via
>> loop2 to my base/ subfolder.  Then re-created the scratch  area, ran
>> dmsetup, and booted.  Loaded up great, did some changes,  shutdown,
>> unloaded the mapper device, unloaded the scratch loop,  unmounted my
>> tmpfs, and re-created that segment again, and Windows  came back in
>> pristine condition.  So that pretty much means my next  battle is
>> Debian's package manager not liking my local repository (I  can't connect
>> my Xen system to the internet, so I have to use such a  setup).  As
>> that's how the Xen LiveCD scripts are doing things, and  assuming
>> everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in  when
>> the kernel actually notices that it's detached.  If one does  losetup -d
>> <loop> followed by losetup <loop> <new img>, it seems  like in a few
>> cases, the kernel still thought it was looking at the  older loop
>> attachment.  I ran into this when using loopback to look  at the disk at
>> the disk level, and then passing -o 32256 to losetup  to look strictly at
>> the first/primary NTFS partition (while trying  to clone it to a smaller
>> disk image, which I finally gave up on  doing).
>>
>> - Haven't found the cause just yet, but dmsetup sometimes doesn't  like
>> looking at loopback devices, and possibly per the first item,  refuses to
>> create the mapped device, citing a resource is busy.   Reboot fixes this,
>> and I suspect both of these are caused by all the  experimenting I'm
>> doing with switching between the various  loopbacks, I probably confuse
>> the kernel.  From a scripted  standpoint, it should be a non-issue,
>> though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems  to be
>> fine, and that generally means any other OS as a domU should  work fine
>> as well (because NTFS is by far the pickiest, most  temperamental FS
>> known).  You get way more options with Linux/Other  UNIXes as far as
>> tricking them into thinking they're in a r/w world,  but this can be
>> extended to encapsulate quite a few domU's.   However, the only
>> limitation I can see is the number of loopback  interfaces.  I use three
>> to setup the Windows domU: /dev/loop2 for  the squashfs mount, /dev/loop0
>> for the windows image inside the  squashfs mount, and /dev/loop1 for the
>> scratch area.  I think more  loop devices can be easily created, but I've
>> not experimented that  far just yet...
>>
>> That's all for now...I've got to install my network scanner into  this
>> Windows image and re-clean it and run these tests all over again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@xxxxxxxxx]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@xxxxxxxxxxxxxxxxxxx
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing  fast
>> domU creation from RW snapshots.... good for Eucalyptus, I guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@xxxxxxxxxxxxx>
>> Think I cracked it.  Don't even need to deal with qemu's qcow stuff  (so
>> far).  Still more tests pending (including using read-only drive
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free  space,
>> then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my
>> specific case, I compacted everything down to ~2GB total installed
>> (inside the domU that is) by using straight qemu-emulated devices  (not
>> the GPLPV stuff), no page file or hibernation, stripping nearly  every
>> misc utility, and a bunch of other stuff, but NO NTFS  compression (for
>> now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/ loop1
>> n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a  scratch
>> area to /dev/loop1 (I don't even know if this is needed just  yet).
>> Needed because dmsetup only works with block devices.  Using  dmsetup, a
>> snapshot is created, so that all writes to loop0 get  redirected to
>> loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then  shutdown.
>> Re-start the domU and back at the desktop, we see that  the newly-created
>> files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/ loop1
>> n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I'm veryifying a few  things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs
>> mount, while the baseline image will be on a physical, read-only  medium.
>> By removing the mapped device and changing out the scratch  file, I was
>> seeing whether the XP image would indeed revert to the  baseline state,
>> to simulate a reboot in which /tmpfs is cleared.   Supposedly, the 'n'
>> parameter to dmsetup does this anyways, but I  wanted to be doubly sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will
>> already be pre-configured, the generation of the scratch image,  setup of
>> the loop devices, and the device-mapping will be on-the-fly  when some
>> future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with  qcow
>> with gplpv and such.  I'm probably not grasping some of it, but  I think
>> those references are aiming for different uses.  So far,  this setup
>> mostly relies wholly on the Linux dom0 to piece  everything together,
>> leaving the WinXP domU utterly oblivious to  what is going on.  And given
>> Windows' nature, that's exactly what we  want.
>>
>> Performance isn't an issue for me in this, so I'm sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment  that
>> this is actually possible), but if it works, and WinXP doesn't  complain,
>> then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
>> [xen-users-bounces@xxxxxxxxxxxxxxxxxxx ] On Behalf Of Joshua Kinard
>> [joshua.kinard@xxxxxxxxxxxxx]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@xxxxxxxxxxxxxxxxxxx
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the- box,
>> then I technically don't need any kind of UnionFS?  Just  someway to
>> specify to Xen a read-only image with a qcow2 image in a  writable zone?
>>
>> Going to dredge through some EU software patent I dug up from  Microsoft.
>> Supposed to document the /MININT switch to the NT kernel  that supposedly
>> makes Windows write all registry changes to volatile  memory instead of
>> to the registry hives.  It's mostly used in the  WinPE environment, but
>> it looks like BartPE leverages it too.  Seems  I need more to it, though,
>> than just modifying boot.ini to pass it,  as once I started to boot
>> Windows from a read-only drive mount  (after discovering Xen/qemu don't
>> properly honor the mode bit in the  Xen config for r or r/w, and even
>> file-level permissions), Windows  crashed with UNMOUNTABLE_BOOT_VOLUME as
>> its error.  But it  highlights the capability is there....just not easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/
>> Firewire drives here, with the exception of CD Burners.  Ditto on  memory
>> cards (SD, MMC, etc..).  I've got very narrow operating  parameters to
>> work in, which is why this is proving to be quite a  fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@xxxxxxxxx]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@xxxxxxxxxxxxxxxxxxx
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@xxxxxxxxxxxxx> wrote:
>> > Hmm, I didn't think of it that way.
>> >
>> > The way I read up on UnionFS and aufs' functionality, was they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>> > i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> /
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp  being
>> in tmpfs).  I guess it's not as granular as that it seems.   Would be a
>> neat trick, but I imagine it'd be complex as anything for  a kernel
>> module to have to keep track of which files have variants  loaded in the
>> writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>> > As for the application, it's a complex network security scanner,
>> made by eEye Digital Security, called "Retina".  We just don't want  to
>> setup and baby sit Windows installations on our Unix networks  strictly
>> for this one app, so I figured if I can get it to run off  of a CD, we
>> can just park some diskless hardware in a closet and  pull it out
>> whenever we need to do network testing and such.
>>
>> If it were me I'd simply setup a Windows domU on any server with
>> enough disk and RAM, make a "template" of the "good" installation
>> (preferably zfs or LVM snapshot), start it only when it's needed, shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you're absolutely sure you'll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But hey,
>> that's just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users