Query an LLM
POST/llms/:llm_id/query
Sends query to the LLM as a single user message, without a system message or retrieval, and returns the generated answer. The stream field selects the response format: newline-delimited JSON (NDJSON) by default, server-sent events with "sse" or plain text with false. The direct query calls the chat completions API of the model server, and works with an LLM registration without an API key.
Permissions. Execute on the LLM.
Response formats describes the three formats, and API conventions describes how a model server failure appears in each format.
Request
Responses
- 200
- 400
- 401
- 403
- 404
- 422
- 502
The answer in the format that stream selects: NDJSON lines of ApiLlmResponse objects, server-sent events with the same objects as data, or the complete answer as plain text. A streamed response has status 200 as soon as the response starts.
The body is not valid JSON, or the endpoint of the LLM names a private, loopback or link-local address while the API server configuration sets app.allow_private_endpoints to false (Invalid parameter, with the reason in details.error).
The authentication headers are missing or invalid, or the API key is inactive or expired. Errors lists the messages.
The license is not valid (Invalid license: <reason>).
No LLM with this identifier exists, or the key lacks execute permission on the LLM (LLM not found, with the identifier in details.id).
The body has no query string, or a field has the wrong type, such as a stream value other than true, false, "stream" and "sse".
The model server returned an error or could not be reached before the first part of the answer, or at any time with "stream": false (Upstream LLM error). A streamed response that has started ends with an error line or an error event instead. The API server log records the reason.