Linux kernel (drivers/infiniband/hw/mlx5): The memory-registration engine on the primary AI-cluster NIC wedges

Impact

The memory-registration engine on the primary AI-cluster NIC wedges permanently. Recovery resets the internal UMR queue-pair without waiting for outstanding work, the firmware discards those completions, and every waiter blocks forever - registration and deregistration stall for all tenants on the node, not just the one that hit the error.

Who can reach it

A tenant container holding /dev/infiniband/uverbs* on mlx5 hardware doing normal memory-region register/deregister work; once anything pushes the shared UMR queue-pair into error and recovery runs, the node-wide stall follows. The UMR queue-pair is per-device and shared, so a single tenant's error becomes everyone's outage.

What to do

The fixed-version metadata in this record is unreliable - apply the listed stable fix commits or move to a current stable kernel. Interim: drain workloads off nodes showing hung MR-registration tasks and reboot; there is no runtime way to reset the UMR queue-pair.

References