LLMs
Language models (LLMs) registered with Foundation4, direct queries to a model and the OpenAI-compatible chat completions endpoint. LLMs describes the object type and the response formats.
List LLMs
Returns one page of the LLM registrations that the key can read, in the list envelope. Query parameters filter the list by field values and set the order and the page. The list has no filter on the model name.
Register an LLM
Registers a connection to a model server that exposes an OpenAI-compatible API. The endpoint must be an `http` or `https` URL with a host and without credentials. Private endpoints are accepted by default; when the API server configuration sets `app.allow_private_endpoints` to `false`, the host must resolve to public addresses only. Foundation4 does not contact the model server during registration, so an incorrect endpoint path, model name or API key appears only when the LLM is first used. The API key is optional. Foundation4 encrypts the API key before storage, and no response returns the key.
Get an LLM
Returns the registration of one LLM. The response never contains the API key of the registration.
Update an LLM
Changes the fields of an LLM registration that the request names, and keeps the stored value of every omitted field. A `null` value clears `description` or `api_key`, and a new `api_key` value replaces the stored key. A new endpoint is checked with the rules of registration. Foundation4 does not contact the model server during the update.
Delete an LLM
Deletes an LLM registration. Agents store no LLM, so the deletion changes no agent. Agent executions and queries that name the deleted LLM then return HTTP 404.
Query an LLM
Sends `query` to the LLM as a single user message, without a system message or retrieval, and returns the generated answer. The `stream` field selects the response format: newline-delimited JSON (NDJSON) by default, server-sent events with `'sse'` or plain text with `false`. The direct query calls the chat completions API of the model server, and works with an LLM registration without an API key.
Relay a chat completion
Relays an OpenAI chat completion request to the chat completions API of the LLM named in the `x-llm-id` header. Foundation4 forwards the body with the registered model name in the `model` field, and sends the API key of the registration, when the registration has one, as a bearer token. The `model` field of the body applies only when the registration has no model name. Foundation4 returns the model server's status and body, including error responses, and performs no retrieval, prompt templating or tracing.