NVIDIA NeMo Framework: Crafted data reaches code injection and privilege escalation in the job context

Impact

Crafted data reaches code injection and privilege escalation in the job context. In an AI datacenter this is the model-and-data supply chain problem: the code runs with whatever the training or inference job holds, which is usually a GPU, a service account, and mounted object storage credentials.

Who can reach it

Requires the job to load an attacker-influenced artifact - a checkpoint, .nemo file, config, tokenizer or dataset. Any pipeline that pulls from a public model hub, a customer bucket, or a tenant-supplied path is in scope.

What to do

Bump the package to the fixed version in bulletin 5736 and rebuild every training/inference image that embeds it. Cost: image rebuild and job restart; no host driver or firmware change. The durable control is refusing to deserialize untrusted checkpoints at all - prefer safetensors-style formats and treat pickle-bearing artifacts as executable code.

References