vLLM: derender endpoints process caller-supplied response objects before limits, exhausting CPU and memory

Impact

The two derender endpoints accept a caller-supplied GenerateResponse object and hand its nested structures (choices, token_ids, logprobs, top_logprobs, routed_experts) to OnlineDerenderer and tokenizer.decode before max_model_len, max_tokens, max_num_seqs or any response-size limit is applied. An authenticated API client can therefore make the server burn CPU and host memory on a request that the normal serving limits would have rejected, and get an oversized response back. On a GPU node the vLLM process is co-resident with the accelerators it owns, so degrading or OOM-killing it takes the model replica offline and the restart costs a full model load - tens of seconds to minutes of idle GPU per replica. The advisory records availability impact only; no data disclosure or code execution is claimed.

Who can reach it

Any client that can reach the vLLM HTTP server and pass whatever authentication is in front of it. No special role is needed beyond a valid API credential; where vLLM is exposed inside a cluster with no auth, any pod that can reach the service qualifies.

What to do

Upgrade to vLLM 0.26.0 (fix in commit 8e61b646) and restart the serving process. This is a rolling restart per replica plus model reload time - no node drain and no host reboot. If an upgrade cannot be scheduled immediately, block the /v1/completions/derender and /v1/chat/completions/derender routes at the gateway or ingress and enforce a request body size limit in front of the server.

References