vLLM OpenAI-compatible server (chat_template / chat_template_kwargs): NOISY-NEIGHBOUR DENIAL OF SERVICE: one tenant

Impact

NOISY-NEIGHBOUR DENIAL OF SERVICE: one tenant takes down a shared inference endpoint, and the GPU sits idle while it happens. The OpenAI-compatible server lets callers supply a Jinja template through chat_template. Jinja has loops and nesting, so a small request body can pin CPU and balloon memory until the server stops answering anyone. The operationally important detail is that blocking the chat_template parameter is not enough: the server builds its kwargs dict and then calls dict.update() with the caller's chat_template_kwargs, so an attacker simply nests a chat_template key inside chat_template_kwargs and overwrites the template anyway. Any filtering you wrote against the obvious parameter name misses the real one. On a GPU fleet the cost is the expensive resource stranded behind a wedged Python process, plus every co-tenant on that replica losing service.

Who can reach it

Network, any client authenticated to the OpenAI-compatible endpoint, if that endpoint accepts chat_template or chat_template_kwargs from untrusted callers. A single ordinary-looking chat completion request is sufficient.

What to do

Upgrade vLLM to 0.11.0 or later and restart the servers. If you front vLLM with a gateway, strip both chat_template and chat_template_kwargs from inbound requests — stripping only chat_template does not close it. Set per-pod CPU and memory limits with a restart policy so a wedged replica is recycled rather than dragging the node, and rate-limit per tenant.

References