Lustre (mdt module, mdt_object_remote): A client sends a packet with unvalidated fields and the metadata server

Impact

A client sends a packet with unvalidated fields and the metadata server dereferences NULL and panics. Losing the MDS takes the entire filesystem namespace offline, which stalls every training job that touches shared storage - not just the tenant that sent the packet.

Who can reach it

Anything with LNet reachability to the MDS. On most clusters that is every compute node, so any tenant with a job can reach it.

What to do

Upgrade Lustre servers to 2.12.3 or later. The fix is in a kernel module on the MDS, so it means unloading and reloading Lustre modules - in practice an MDS failover or a reboot of the metadata server pair. DDN EXAScaler ships this Lustre code, so EXAScaler fleets inherit the issue and need DDN's corresponding release rather than an upstream build.

References