Skip to main content

LLMs

Language models (LLMs) registered with Foundation4, direct queries to a model and the OpenAI-compatible chat completions endpoint. LLMs describes the object type and the response formats.

📄️Register an LLM

Registers a connection to a model server that exposes an OpenAI-compatible API. The endpoint must be an `http` or `https` URL with a host and without credentials. Private endpoints are accepted by default; when the API server configuration sets `app.allow_private_endpoints` to `false`, the host must resolve to public addresses only. Foundation4 does not contact the model server during registration, so an incorrect endpoint path, model name or API key appears only when the LLM is first used. The API key is optional. Foundation4 encrypts the API key before storage, and no response returns the key.

📄️Relay a chat completion

Relays an OpenAI chat completion request to the chat completions API of the LLM named in the `x-llm-id` header. Foundation4 forwards the body with the registered model name in the `model` field, and sends the API key of the registration, when the registration has one, as a bearer token. The `model` field of the body applies only when the registration has no model name. Foundation4 returns the model server's status and body, including error responses, and performs no retrieval, prompt templating or tracing.