Query an LLM
POST/llms/:llm_id/query
Sends query to the LLM as a single user message, without a system message or retrieval, and returns the generated answer. The stream field selects the response format: newline-delimited JSON (NDJSON) by default, server-sent events with "sse" or plain text with false. The direct query calls the Responses API of the model server, so the LLM registration requires an API key.
Permissions. Execute on the LLM.
Response formats describes the three formats, and API conventions describes how a model server failure appears in each format.
Request
Responses
- 200
- 400
- 401
- 403
- 404
- 500
The answer in the format that stream selects: NDJSON lines of ApiLlmResponse objects, server-sent events with the same objects as data, or the complete answer as plain text. A streamed response has status 200 as soon as the response starts.
The LLM registration has no API key (Invalid parameter, with API key is required for OpenAI provider in details.error).
The authentication headers are missing or invalid, or the API key is inactive or expired. Errors lists the messages.
The license is not valid (Invalid license: <reason>).
No LLM with this identifier exists, or the key lacks execute permission on the LLM (LLM with id <id> not found).
With "stream": false, the model server returned an error or could not be reached (Stream error: <reason>).