Xinference: unauthenticated chat request reaches eval() in the Llama3 tool-call parser

Impact

Anyone who can reach the inference API can run arbitrary Python in the Xinference server process. A request to /v1/chat/completions carrying a tools field steers the model into emitting a Python expression that the tool parser hands to eval(), so the prompt itself becomes the payload and no credentials are required. On a GPU node that process usually owns the model cache, the loaded weights and whatever service tokens the serving pod was given, and it commonly runs with the GPU devices mapped in. In a multi-tenant serving fleet a single exposed endpoint turns into code execution next to other tenants' models, and the same path exists on the websocket route.

Who can reach it

Any network client that can reach the Xinference HTTP API, unauthenticated. No user interaction and no operator action beyond having a Llama3-family model with tool calling enabled.

What to do

Upgrade to 2.7.0, which removes the eval() path, then restart the Xinference workers. Model reload cost is the usual cold start for each served model; no host reboot or driver change is involved. Until the upgrade lands, keep the API off any untrusted network and disable tool calling for Llama3 models.

References