Re: [Xen-devel] where to find blktap2 kernel module

["Dr. Greg Wettstein" <greg@xxxxxxxxxxxxxxxxx>]

On Apr 9,  8:13am, Tim Wood wrote:
} Subject: Re: [Xen-devel] where to find blktap2 kernel module

Hi, hope the day is going well for efveryone.

> --485b397dd775dd0da404bd3df0b5
> Content-Type: text/plain; charset=ISO-8859-1
> 
> On Sat, Apr 7, 2012 at 3:37 AM, Ian Campbell <Ian.Campbell@xxxxxxxxxx>wrote:
> 
> > On Fri, 2012-04-06 at 22:24 +0100, Tim Wood wrote:
> > > In xen 4.0.X I was able to write custom blktap2 drivers as a nice way
> > > to intercept VM disk traffic.  I'm now trying to take a step up to xen
> > > 4.1.2 and find that blktap2 doesn't seem to be supported anymore, or
> > > at least it requires a kernel module which I'm not sure where to
> > > find.  Is blktap2 still in use, or is there an entirely different way
> > > I should be approaching this?
> > >
> > >
> > > Previously I could run commands like:
> > >
> > >
> > > tapdisk2 -n tap:aio:/home/twood/vms/testdisk.img
> > >
> > >
> > > the new tapdisk2 doesn't support that interface anymore,
> >
> > What do you mean? I don't think the tap interface has changed (but I'm
> > not sure). In any case I'm pretty sure the functionality should be there
> > even if the command line has changed.
> >
> 
> Sorry, I wasn't clear--the functionality is still there but the command
> line interface has changed.  The xm-block attach command below seems to be
> equivalent.
> 
> 
> >
> > >  but it seems like the correct approach is now:
> > >
> > >
> > > xm block-attach 0 tap2:aio:/path/disk.img xvda w 0
> > > Error: ('create', '-aaio:/path/disk.img') failed (55808 blktap kernel
> > > module not installed )
> >
> > I think this ultimately turns into the same sort of command as the one
> > above.

The documentation in the Xen sources is out of date with respect to
userspace interfaces to blktap2.  The 'tap-ctl' command is the
simplest way to manage dom0 instances of block devices.  Its actually
a pretty well defined interface.

Here is a brief recipe:

1.) Allocate a blktap minor device:

        tap-ctl allocate

2.) Spawn a userspace driver:

        tap-ctl spawn

    The 'tap-ctl list' command is useful at this point in order to get
    the status of available devices and process numbers.  In the
    following commands substitue PPP with the PID of the userspace
    driver and MMM with the minor number of the device you are working
    with.

3.) Bind the userspace driver to a minor instance:

        tap-ctl attach -p PPP -m MMM

4.) Open a filesystem image with an appropriate driver.  In this case
    it is aio, if you are rolling your own subsitute the driver prefix
    (aio:) with the name of your driver.

        tap-ctl open -p PPP -m MMM aio:FNAME

    If this command succeeds you will see a hotplug/udev inventing
    indicating that a new block device was created.  The 'official'
    device node locations should be:

        /dev/xen/blktap-2/tapdevMMM

    Where MMM is replaced with the minor number of the block device.

5.) Close the filesystem image:

        tap-ctl close -p PPP -m MMM

6.) Detach the process from the allocated minor number:

        tap-ctl detach -p PPP -m MMM

7.) Free the minor number:

        tap-ctl free -m 0

> > > What is the "proper" way of getting the blktap kernel module
> > > installed?  I found this:
> > > http://packages.debian.org/sid/blktap-dkms
> >
> > Unfortunately the blktap2 module is not something which can be
> > upstreamed.
> >
> > The DKMS package is probably a good bet for now. The other alternative
> > is to switch to a slightly older kernel which has the blktap2 driver in
> > it, for example the 2.6.32 based xen.git kernel tree or one of the
> > classic Xen forward ports.

> OK, I will fiddle with DKMS a bit more or switch to 2.6.32.

We have a vested interest in all so in the interests of advancing this
technology we have clean kernel patches available at the following
locations:

        ftp://ftp.enjellic.com/pub/xen/blktap2-3.0.patch

        ftp://ftp.enjellic.com/pub/xen/blktap2-3.2.patch

These are basically cleanups from the last available GIT tree which
Dan Stodden had made available.

There are minimal dependencies so they should drop into other kernels
without too much effort.  We are focusing on 3.0 and 3.2 since they
seem to be odds on favorites for 'stable' kernels.

> > You mentioned writing your own blktap modules so the qemu-supplied qdisk
> > backend is probably not much use to you. This is used by the "xl"
> > toolstack by default when blktap is not present, but isn't supported by
> > xm and doesn't allow for custom modules .
> >
> > I'm actually quite interested in the fact that you are writing custom
> > blktap modules -- are you able to share the details?

> Sure, I'm a CS professor and do research on what cool stuff you can
> add in the virtualization layer.  What I did before was a module
> similar to remus's, but designed for WAN replication and
> synchronizing across groups of VMs.  The paper can be a bit tough to
> get through, but it has some of the details:
>
> http://faculty.cs.gwu.edu/~timwood/papers/SOCC11-PipeCloud.pdf

Thanks for forwarding the reference.

> Blktap is a very handy way for us to prototype these types of
> features without having to muck through the core xen code.

Indeed, we are looking at it with respect to NPIV support.  QEMU is
difficult to deal with as it is.  It seems to be an architectural
advantage to have an independent block support architecture given what
people want to do with block devices these days.

blktap2 is a very useful architecture, it seems to suffer mainly from
a lack of attention.  Hopefully this can be improved a bit.

> > Long term someone is working on a "blktap3" which is fully userspace and
> > so doesn't require a kernel module. We hope to see this for 4.3. For 4.2
> > it looks like we'll be sticking with the qdisk backend.
> >
> > Sounds great, I look forward to it.

A pure userspace implementation would be obviously an advantage.  That
would seem to be a ways out however and if our observations are useful
it seems that moving user Xen implementations forward are suffering a
bit secondary to the number of things which change a bit from release
to release and make maintenance troublesome.

Hope the above is useful.

Good luck with your efforts.

Greg

}-- End of excerpt from Tim Wood

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@xxxxxxxxxxxx
------------------------------------------------------------------------------
"In the future, company names will be a 32-character hex string."
                                -- Bruce Schneier

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel