Linux kernel (drivers/pci/hotplug): A power fault on a PCIe hotplug slot latches a sticky status bit that the hardirq

Impact

A power fault on a PCIe hotplug slot latches a sticky status bit that the hardirq handler cannot clear, so the interrupt handler spins forever inside hardirq context. The CPU never returns to the scheduler - the node hard-hangs and every tenant on it goes down at once, with no clean drain and no way in to diagnose it.

Who can reach it

Not tenant-software reachable. It needs a device or sled in a PCIe hotplug slot to assert a main power fault - a failing or over-current NVMe/GPU carrier, a bad riser, or an add-in device someone with physical or smart-hands access installs. Relevant to GPU clusters because hot-pluggable NVMe and GPU sleds are exactly the population that generates power faults, and one faulty card silently converts into a whole-node outage. Requires pciehp driving the slot.

What to do

Update to 4.19.233 / 5.4.177 or later (the fix sets the power_fault_detected flag in the hardirq handler so the loop terminates). Interim: there is no software mitigation once the fault latches - track slot power-fault events in your health telemetry and drain nodes reporting them, and restrict physical/smart-hands installation of unvetted cards into hotplug slots.

References