llama.cpp: unvalidated token IDs in a /completion prompt read embedding memory out of bounds or crash the server

CVSS 8.8CVE-2024-58400AI/ML frameworks & servingcurated

Impact

The /completion endpoint accepts a prompt given as an array of integer token IDs and passes them to llama_decode_internal / llama_encode_internal without checking them against n_vocab. An ID outside 0..n_vocab-1 indexes past the embedding table, so the caller can read process memory adjacent to the weights or kill the server process. On a GPU node this is an unauthenticated remote read of whatever else lives in the serving process address space, and the crash path takes the model offline until the process is restarted - a reload that costs the full weight-load time and leaves the GPUs idle. llama-server has no authentication of its own, so the exposure is whatever the network in front of it permits.

Who can reach it

Anyone who can reach the llama-server HTTP port and submit a completion request. No authentication is required by the server itself.

What to do

Upgrade llama.cpp to build b3701 or later and restart each llama-server process - a per-replica restart with a model reload, no node drain. If the build cannot be moved yet, reject or clamp integer-array prompts at the proxy in front of the server and keep the port off any tenant-reachable network.

References

https://nvd.nist.gov/vuln/detail/CVE-2024-58400 https://github.com/ggml-org/llama.cpp/commit/eae597182cb61bbde0b26e7cec5999d28b9327fe https://github.com/ggml-org/llama.cpp/releases/tag/b3701 https://github.com/ggml-org/llama.cpp/pull/9359

Related entries

All AI/ML frameworks & serving entries

This entry is curated: imported from vendor advisories with machine assistance, not yet individually verified. Confirm against your vendor's advisory before acting, and report anything wrong.