Re: [PATCH 0/8] Force detach PCI devices for ACPI-based and PCIe native hot-unplug

["Michael S. Tsirkin" <mst@xxxxxxxxxx>]

On Mon, Sep 07, 2026 at 01:26:57AM -0700, Dongli Zhang wrote:
> 
> 
> On Thu, Sep 3, 2026 8:28:35AM -0700, Igor Mammedov wrote:
> > On Wed, 26 Aug 2026 09:15:47 -0700
> > Dongli Zhang <dongli.zhang@xxxxxxxxxx> wrote:
> > 
> >> On Mon, Aug 24, 2026 7:44:49AM -0700, Daniel P. Berrangé wrote:
> >> > On Sun, Aug 23, 2026 at 06:13:30PM -0700, Dongli Zhang wrote:  
> >> >> Hot-unplugging a PCI device can require cooperation from the guest. For
> >> >> ACPI PCI hotplug, QEMU notifies the guest through ACPI and the guest
> >> >> eventually writes the ACPI PCI eject register. For PCIe native hotplug,
> >> >> QEMU notifies the guest through the PCIe hotplug mechanism and waits for
> >> >> the slot unplug flow to complete. Only after that completion does QEMU
> >> >> unrealize the device and emit DEVICE_DELETED.
> >> >> 
> >> >> This can leave a device stuck in the unplug pending state when the guest
> >> >> does not cooperate. Examples include:
> >> >> 
> >> >> 1. The guest has panicked, or the relevant ACPI/PCI hotplug driver is
> >> >> unavailable.
> >> >> 
> >> >> 2. The guest is stalled and cannot handle the hot-unplug event. For
> >> >> example, stalling the Linux [irq/9-acpi] kernel thread can reproduce 
> >> >> this
> >> >> for ACPI-based hot-unplug.
> >> >> 
> >> >> 3. The device was attached to a slot that the guest cannot use. For
> >> >> example, a pcie-root-port only supports slot 0. If a device is added to 
> >> >> a
> >> >> non-zero slot below a pcie-root-port, the guest may never discover the
> >> >> device and therefore may never complete the unplug request.
> > 
> > all of above is actually expected, no (functioning) driver => no 
> > hotplug/unplug.
> > it's the guest problem. Once device it exposed to guest its life-cycle
> > not longer owned by QEMU.
> > 
> > That's what one would see in real hw as well, you press eject button
> > but it will not do anything if OS doesn't process it.
> > also see comment at the end.
> 
> Users are generally more tolerant of issues with real hardware.
> 
> In virtualization and cloud environments, PCI hotplug is more commonly used 
> for
> NICs and storage devices. Users are less tolerant of disruption or unexpected
> failures.


Simply put, surprise removal exists in the hardware.  Emulating that
makes sense, at a high level. Nor is it too hard.
However, guests, especially Linux, do not
handle it all that well generally. Exactly because users
would tend to impatiently reach for that tool, then
blame QEMU after a crash, we avoided emulating that.


I'd expect much more in the way of research into how guests behave,
perhaps some ways to limit it to devices that work well, and
likely some linux patches to make it work better, before we commit to
supporting such interfaces.


> > 
> >> >> 
> >> >> The non-zero slot case has also been discussed in:
> >> >> 
> 
> [snip]
> 
> >> >   
> >> 
> >> Thank you very much!
> >> 
> >> I see that the issue has been fixed. The ticket mentions the following.
> >> 
> >> "What I am observing is that it seems when the slot ID != 0, the guest OS 
> >> seems
> >> to ignore this and we never seem to hit ich9_pm_device_unplug_cb()."
> >> 
> >> Based on my experience and evaluation, ACPI-based hotplug is more likely to
> >> encounter an issue where the guest VM does not respond to an unplug 
> >> operation.
> > 
> > I'm not sure it's a good idea to delete device when guest still thinks it's 
> > there
> > (you can make guesses on QEMU side if it's in use, how useful those are is 
> > questionable).
> 
> By instrumenting the QEMU functions related to device hotplug and PCI
> initialization, we may be able to make informed assumptions.

So try. But just know that pci initialization is commonly done by firmware, not
the driver.

> > 
> > as far as I know, ACPI hotplug has no notion of surprise removal (pls 
> > educate me if it's not the case),
> > so I wouldn't do what you are proposing here at all, it's basically asking 
> > for disaster to happen.
> > And all this is basically for dealing with abused qemu flexibility.
> > 
> > Please (re)formulate usecase and make it more clear as what is eludes me
> > no matter how many times i've read this cover letter.
> 
> Here are some use cases in virtualization and cloud environments:
> 
> 1. Suppose there is a QEMU user configuration error and a PCI device is 
> attached
> to slot 1 of a pcie-root-port that uses ACPI-based hotplug. The device may not
> be detected or ejected. As a result, there is no way to detach it from the
> user's QEMU instance until the guest VM reboots.

sounds vague. if users can not configure qemu what are the chances
they will use force detach responsibly?

> 2. For an unknown reason in the customer's guest kernel (Linux, Windows, or
> BSD), a PCI device may still be referenced by the guest kernel or its 
> services.
> As a result, the guest never writes the eject register for ACPI-based hotplug,
> and QEMU cannot detach the device. The customer may blame QEMU for not 
> removing
> it. A force-detach option could provide an escape hatch, with a warning that 
> it
> may make the VM unstable or insecure.

So just reboot the guest.

> 3. Suppose a VM is stuck because of a guest kernel bug, such as a Linux kernel
> panic without a kdump kernel being triggered. The guest kernel is 
> unresponsive.
> Force detach could allow the block device to be temporarily attached to 
> another
> VM without resetting the currently panicked VM.

So just reboot the guest.

> 4. Provide a mechanism to demonstrate that force detach is unsafe. Otherwise,
> users may repeatedly attempt it and mistakenly conclude that QEMU is at fault 
> :)

We have that - we don't support unsafe detach.

> Thank you very much!
> 
> Dongli Zhang
> 
> > 
> > On positive note:
> > 
> > What you can try to implement is native PCI-E support for surprise removal.
> > How hard that would be I don't know. And I would well expect if one 
> > deviates from
> > real hw expectations/configs (such as not 0 slot/partial func removal),
> > one would quickly stumble upon issues as  that's not what what vendors 
> > write/test
> > drivers for.
> > 
> > Even if it's not likely to be used in practice (guest still might not 
> > support it),
> > it may serve as test-bed for guest drivers.
> > 
> >> Thank you very much!
> >> 
> >> Dongli Zhang
> >> 
> >