Re: [Xen-users] linux stubdom

[Markus Hochholdinger <Markus@xxxxxxxxxxxxxxxxx>]

Hello,

Am 29.01.2013 um 17:36 Uhr schrieb Ian Campbell <Ian.Campbell@xxxxxxxxxx>:
> On Tue, 2013-01-29 at 15:46 +0000, Markus Hochholdinger wrote:
> > I'm reading a lot about xen stub-domains, but I'm wondering if I can use
> > a linux stubdom to serve a "transformed" block device for the according
> > domU. The wiki page(s) and the stubdom-directory in the source code
> > leaves a lot of questions to me I'm hoping someone can answer me here.
> I think the thing you are looking for is a "driver domain" rather than a
> stubdomain, http://wiki.xen.org/wiki/Driver_Domain. You'll likely find
> more useful info if you google for that rather than stubdomain.

a driver domain is a great thing, but I'm wondering if I can live migrate a 
domU inclusive its driver domU? In the domU config I have to use the $domid of 
the driver domain. This $domid is probably wrong after a live migration of the 
domU and I have to migrate the driver domU in the same time while migrating 
the domU. If the driver domU is ment to stay on one dom0 there's no difference 
between doing this in dom0, but for me this doesn't work.
Do you know how I can live migrate a domU which depends on a driver domU? How 
can I migrate the driver domU?
For my understanding the block device has to be there on the destination dom0 
before live migration begins but is also used on the source dom0 from the 
migrating, but still running, domU.
Can I combine a driver domU to a normal domU like I can combine a stubdom with 
a normal domU?

I thought a stubdom live migrates with its domU so you don't have to worry 
that the driver domU live migrates while the according normal domU migrates.


> > So my question are:
> > * What are the requirements to run linux inside a stubdom? Is a current
> > pvops
> >   kernel enough or has the linux kernel to be modified for a stubdom? If
> >   yes, I would prepare a kernel and a minimal rootfs within an initrd to
> >   setup my blockdevice for the domU.
> You can use Linux as a block driver storage domain, yes.

OK, I see. But still, would it be possible to run linux in a stub-domain? I've 
read e.g. http://blog.xen.org/index.php/2012/12/12/linux-stub-domain/ which 
describes this, but I'm unsure if this will be (or is already) supported by 
current xen?


> 
> > * How can I offer a block device (created within the stubdom) from the
> > stubdom
> > 
> >   to the domU? Are there any docs how to configure this?
> > 
> > * If the above is not possible, how could I offer a blockdevice from one
> > domU
> > 
> >   to another domU as block device? Are there any docs how to do this?
> Since a driver domain is also a domU (just one which happens to provide
> services to other domains) these are basically the same question. People
> more often do this with network driver domains but block ought to be
> possible too (although there may be a certain element of having to take
> the pieces and build something yourself).

Is live migration possible with this driver domUs? What requirements are 
needed that I can live migrate a domU which depends on a driver domU?


> Essentially you just need to a) make the block device bits available in
> the driver domain b) run blkback (or some other block backend) in the
> driver domain and c) tell the toolstack that a particular device is
> provided by the driver domain when you build the guest.

Yes, I would provide two block devices (logical volumes) into the driver domU, 
create there a software raid1 device and make the md device available with 
blkback. I would do this for each domU so I can live migrate domUs 
independent. The driver domU only needs a kernel and a initrd with a rootfs 
filled with enough to build the md device and export it with blkback.

But how can I address this exported block device? As far as I've seen I need 
the $domid of the driver domain in my config file of the domU, or am I missing 
something?


> For a) one would usually use PCI passthrough to pass a storage
> controller to the driver domain and use the regular drivers in there.
> But you could also use e.g. iSCSI or NFS (I guess). If you want to also
> use this controller for dom0's disks then that's a bit more complex...

Because I'd like to "migrate" the driver domain with my normal domU I wouldn't 
do any pci passthrough but only provide the logical volumes for baking the 
block device of one domU.


> For b) that's just a case of compiling in the appropriate driver and
> installing the appropriate hotplug scripts in the domain.
> For c) I'm not entirely sure how you do that with either xend or xl in
> practice. I know there have been some patches on xen-devel not so long
> ago to improve things for xl support of disk driver domains. It possible
> that you might need to hack the toolstack a bit to get this to work, and
> depending on how and when the disk images are constructed you may need
> some out of band communication between the toolstack domain and driver
> domain to actually create the underlying devices.

I'll test a driver domain so I can test what works for me and what not.


> The biggest problem I can see is supporting Windows HVM, since the
> device model also needs to have access to the disk in order to provide
> the emulated devices (at least initially, hopefully you have PV
> drivers). The usual way to do this is to attach a PV device to the
> domain running the device model where the backend is supported by the
> driver domain as well. Again you might need to hack up a few things to
> get this working.

So this would be a dependency that the driver domain is started before the 
stubdom with qemu.

In some subdom-startup script I saw the parameter "target" while creating the 
stubdom. Is this a possibility to combine two domUs?

As far as I understand with a driver domU I need the $domid of the driver domU 
in my config, so this is the connection between the two domUs.
But with stub-domains I haven't understand how data flows between stubdom and 
domU and because I've seen a lot of nice little pictures describing that I/O 
flows between domU over stubdom to dom0 and backwards I thought stub-domains 
would be the way to go.

Many thanks so far for your informations.


> > What I'm trying to do:
> > * In my case, I make logical volumes available on all hosts with the same
> > path
> > 
> >   on each host. So I can assemble a software raid1 where each device
> >   lives on a different server while not loosing the possibility of live
> >   migration.
> > 
> > * Configure a domU with two block devices, the according (linux) stubdomu
> > 
> >   assembles a software raid1 (linux md device) and presents this md
> >   device to the domU. So the domU hasn't to handle anything with the
> >   software raid1 but has ONE redundant block device for its usage.
> > 
> > * I have two use cases, one is a HVM domU, where something like windows
> > is
> > 
> >   running and because the lack of (good) software raid1 I use the
> >   software raid1 of linux for baking the block device inside the
> >   stubdom. The other use case is a PV domU where the admin of the
> >   virtualization environment is not the admin of the domU and therefore
> >   isn't able to manage the software raid1 inside the domU. So the
> >   stubdom could be used to manage the software raid1 without interfering
> >   within the domU.

-- 
greetings

eMHa

Attachment: signature.asc
Description: This is a digitally signed message part.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users