Re: [Xen-devel] Hotplugged devices in Xen 4.5 and domain reboot

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Tue, 2015-12-01 at 18:48 +0200, Iurii Mykhalskyi wrote:
> Thanks to all for a replays, please see my answers below:
> 
> On 12/01/2015 05:29 PM, Wei Liu wrote:
> > On Tue, Dec 01, 2015 at 04:58:55PM +0200, Iurii Mykhalskyi wrote:
> > > Our real usb mass-storage device are located at driver domain (DomD).
> > > So we
> > > setup second block-device backend there.
> > > 
> > > To hotplug usb mass-storage from DomD we use follow command:
> > > 
> > > xl block-attach domU_id phy:/bla-bla,xvda10,w,backend="DomD"
> > > 
> > What happens if you run this in Dom0? I guess DomD doesn't respond to
> > the request?
> ÂÂÂ Yes, there is no responded from domD, because actual storage device
> are located there, and toolstack stuck on real device existence check.

This check is a toolstack bug which should be fixed. We've squashed some of
them at various points, but I'd not be surprised if others remain.

> > > There was no support of attaching block-device in runtime from domain
> > > other
> > > to Domain-0, so we have made some hacks to allow call block-attach
> > > command
> > > from non-dom0 privileged domain.
> > So this is a special use case. This is the first time I know people
> > actually run xl block-attach in driver domain.
> ÂÂÂ Yes, this is special case and we this by our solution design.
> > > One of patches was - don't update
> > > /var/lib/xen/userdata-d.$DOMID-$UUID.libxl-json during execution of
> > > this
> > > command (because this log located on dom0 rootfs and we don't have
> > > any
> > > access to it from DomD). So, there is no different in configs before
> > > and
> > > after hotplug.
> > > 
> > The state of $DOMID is recorded in libxl-json file. No wonder you lose
> > all state.
> > 
> > But even if you write those states, they are going to be inside driver
> > domain.ÂÂThere is no way at the moment to synthesise the state inside
> > Dom0 and DomD into one. There is also difficulty in how you can split
> > the synthesised and dispatch the states to multiple entities again when
> > rebuilding a domain.
> > 
> > So I think having multiple entities managing state of one single domain
> > is bad. I think the proper way of making it work is to make hotplug
> > device from domain other than Dom0 work.
> > 
> > There is a daemon "xl devd" in driver domain. We might be able to teach
> > it to response to Dom0 toostack request. I'm a bit surprised if it
> > doesn't do that already. Did you forget to start that daemon?
> ÂÂÂ We can't run devd in driver domain, because it failed on connect to
> xenstored socket (/var/run/xenstored/socket - we have xenstored running
> only in dom0).ÂÂ 

devd _should_ be able to talk to xenstored over the kernel provided
interface to the shared ring rather than the local socket.

It is certainly not expected that devd be colocated in the same domain as
happens to be running xenstored.

If this is not working then there is another bug.

> > In general a driver domain would not be expected to have sufficient
> > privilege over e.g. a guest domain's /local/domain/domU/devices to
> > create
> > the f.e. dirs.

> ÂÂÂ In our solution we have to create 2 full privileged domains - Dom0 and 
> DomD, so we need 2 toolstack domains.
> ÂÂ Any special privileges hack wasn't done - we need just to setup additional 
> permissions for DomD.

I'm afraid you simply cannot have 2 toolstack domains. The toolstack is a
singleton entity in a Xen system.

If you want to run toolstack operations from a non-toolstack domain then
you will need to arrange for some (likely out-of-band) mechanism for such
domains to ask the single toolstack domain to do something on their behalf.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel