Lustre (mdt module, mdt_object_remote): A client sends a packet with unvalidated fields and the metadata server

Impact

A client sends a packet with unvalidated fields and the metadata server dereferences NULL and panics. Losing the MDS takes the entire filesystem namespace offline, which stalls every training job that touches shared storage - not just the tenant that sent the packet.

Who can reach it

Anything with LNet reachability to the MDS. On most clusters that is every compute node, so any tenant with a job can reach it.

What to do

Upgrade Lustre servers to 2.12.3 or later. The fix is in a kernel module on the MDS, so it means unloading and reloading Lustre modules - in practice an MDS failover or a reboot of the metadata server pair. DDN EXAScaler ships this Lustre code, so EXAScaler fleets inherit the issue and need DDN's corresponding release rather than an upstream build.

References

Related entries

• Lustre (ptlrpc module): Out-of-bounds write in the RPC layer, reachable by a client that lies about packet field sizes.CVE-2019-20425 · Lustre (ptlrpc module)High
• Lustre (ptlrpc module): A second out-of-bounds access in ptlrpc triggered by unvalidated client packet fields, endingCVE-2019-20426 · Lustre (ptlrpc module)High
• Lustre (ptlrpc module): Out-of-bounds read in ptlrpc leading to a server panic. The read primitive also means serverCVE-2019-20428 · Lustre (ptlrpc module)High
• Lustre (ptlrpc module, lm_bufcount handling): A client that modifies the lm_bufcount field walks the server off the endCVE-2019-20429 · Lustre (ptlrpc module, lm_bufcount handling)High
• Lustre (mdt module, MDT Body eadatasize): An oversized eadatasize field in an MDT request drives the metadata serverCVE-2019-20430 · Lustre (mdt module, MDT Body eadatasize)High
• Lustre (ptlrpc, osd_map_remote_to_local): Out-of-bounds access in the object-storage mapping path, reachable from aCVE-2019-20431 · Lustre (ptlrpc, osd_map_remote_to_local)High