Linux kernel (drivers/infiniband/hw/mlx5): Memory-region deregistration self-deadlocks under memory pressure. An

Impact

Memory-region deregistration self-deadlocks under memory pressure. An allocation made while holding the page-mapping lock can drive reclaim, which calls back into the same driver's invalidation handler and blocks on the lock it already holds - the tenant's task hangs unkillably and the mlx5 registration path is blocked for everything else on the node.

Who can reach it

A tenant container holding /dev/infiniband/uverbs* on mlx5 hardware deregistering an on-demand-paging memory region while the node is under memory pressure. Both halves are tenant-influenceable: the tenant chooses when to deregister and can raise memory pressure itself, which is exactly the condition a busy shared GPU node is already in.

What to do

Update to 6.12.37 / 6.14 or later. Interim: disable on-demand paging for tenant workloads so ODP regions are never created, and keep node memory headroom high enough that reclaim is not entered during RDMA teardown.

References