Re: [Xen-devel] Driver domains and hotplug scripts, redux

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Wed, 11 Jan 2012, Roger Pau Monné wrote:
> Hello,
> 
> This comes from my experience with Xen and hotplug scripts, and it
> might be wrong, since I wasn't able to find any document explaining
> exactly how hotplug execution works and who does what. I'm gonna try
> to list the sequence of events that happens when a device is added (I
> really don't want to keep on with the discusion if this is a protocol
> or not):
> 
> 1. Toolstack writes: /local/domain/0/backend/<vbd or vif>/... with "state = 
> 1".
> 2. Kernel acks xenstore backend device creation, creates the device
> and sets backend "state = 2".
> 3. xenbackendd notices backend device with "state == 2" and launches
> hotplug script.
> 4. Hotplug script executes necessary actions and sets backend
> "hotplug-status = connected".
> 5. Kernel notices "hotplug-status == connected", plugs the device, and
> sets xenstore backend device "state = 4".
> 
> This is true on NetBSD, because there aren't any userspace hotplug
> devices, someone should probably add the missing bits if the device is
> implemented in userspace (I'm not really sure of what happens inside
> the kernel in #2 and #5, specially when using blktap or qdisk).
> 
> Regarding device shutdown/destroy:
> 
> 1. Guest sets frontend state to 6 (closed)
> 2. Kernel unplugs the device and sets backend "state = 6".
> 3. xenbackendd notices device with "state == 6", and performs the
> necessary cleanup.
> 3. Toolstack notices device with "state == 6" and removes xenstore
> backend entries.
> 
> Notice that I've used two #3, that's where the race condition happens,
> because there's no synchronization between toolstack and
> hotplug/xenbackendd to know when hotplug scripts have been executed
> (however we should be able to synchronize this watching
> "hotplug-status" instead of "state", and waiting for it to change to
> "disconnected").
> 
> Now, we have to decide how to fix the shutdown/destroy race and how to
> implement this outside of the Dom0. I'm not really sure if it's a good
> idea to try so hard to keep this flow intact, I think it's best to try
> to define a flow that solves our current problems, regardless of how
> things are now, and then try to map both flows to see what should be
> changed and how.
> 
> Since the device will be plugged from a Domain different than Dom0,
> the toolstack doesn't really (and probably shouldn't) know anything
> about which backend type will be used (phy, blktap, qdisk...). Having
> that in mind, I don't know how can we write
> /local/domain/<driverdom_id>/backend/... from Dom0, instead we should
> create something like:
> 
> /hotplug/domain/<driverdom_id>/<vbd or vif>/<domu_id>/<device_id>/params
> /hotplug/domain/<driverdom_id>/<vbd or vif>/<domu_id>/<device_id>/script
> /hotplug/domain/<driverdom_id>/<vbd or vif>/<domu_id>/<device_id>/state
> [This seem like the minimum necessary parameters, but probably there
> are others, so add what you feel necessary]
> 
> With that the driver domain should be able to create
> /local/domain/<driverdomain_id>/backend/... and the frontend also.
 
What you are proposing is good enough to solve the problem I was
describing before: xenbackendd in the driver domain would have the
liberty of running any setup script it needs to run, before writing the
backend nodes to xenstore.
Also, considering that xenbackendd would be in charge of both running
the script and writing/removing the backend nodes it would have full
control over the sequence of events, that gives us a lot of flexibility
to deal with complex scenarios.
For these reasons, I support this idea.


> I'm not sure if we should control the execution of hotplug scripts
> from Dom0, or instead let the driver domain decide when it's best to
> execute each script. 

At this point it is best to keep the driver domain in charge of its own
scripts.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel