Providers and models
This page lists every embedding provider and text splitter provider: where each provider runs, the parameters with their defaults, the model names that each provider accepts, the images that carry models and packages, and the network access that each provider needs. Administrators use this page to configure embedding models and text splitters, and operators use this page to plan an air-gapped installation. Embedding models and text splitters explains both object types and shows the create requests.
Where providers run
Providers run in one of two places:
- API server and workers.
FastEmbedEmbeddingsandOpenAIEmbeddingsrun inside the Foundation4 process: the API server, each worker and the command-line interface (CLI). Each process loads a model on first use and keeps the model in memory. - gRPC service. The other embedding providers and every text splitter run in the gRPC service, a Python service that runs as a second container in each API server pod and each worker pod. The gRPC service keeps each loaded model and splitter in memory.
Provider names are case-sensitive. A create request with any other provider name returns HTTP 404, with the message Embedding Provider not found for an embedding model and Text Splitter Provider not found for a text splitter, and the provider name in details.provider.
Embedding providers
provider | Runs in | Model source | Network access |
|---|---|---|---|
FastEmbedEmbeddings | API server and workers | ONNX model files in the FastEmbed model directory | None for models whose files are present. Other built-in models download from Hugging Face on first use |
OpenAIEmbeddings | API server and workers | A server that implements the OpenAI embeddings API | The server named in endpoint |
GPT4AllEmbeddings | gRPC service | GGUF model files in the GPT4All model directory | The GPT4All library can contact the GPT4All service while loading a model |
HuggingFaceEmbeddings | gRPC service | Sentence Transformers model files in the Hugging Face model directory | None for models whose files are in the Hugging Face model directory. A missing model downloads from the Hugging Face Hub, unless HF_HUB_OFFLINE=1 is set on the gRPC service |
HuggingFaceEndpointEmbeddings | gRPC service | A Hugging Face inference endpoint | The inference endpoint, on every request |
The model directories are set by the paths configuration of the deployment. The defaults are /data/fastembed, /data/gpt4all and /data/huggingface.
Embedding model limits
- Dimensions. A vector index stores vectors in a column with the dimension of the embedding model and builds a distance index on the column. The hierarchical navigable small world (HNSW) distance indexes of pgvector support up to 2,000 dimensions, so a pgvector index cannot serve a model with more dimensions. A create request for a model with more than 2,000 dimensions therefore returns HTTP 400 with
Invalid parameter, whatever the index type. New vector indexes use VectorChord, as described in Search indexes. - Distance. Every model on this page is trained for cosine similarity, so searches use the default
cosinedistance strategy. - Text as given. Foundation4 embeds the text of queries and fragments exactly as given, with no instruction or prefix. Models whose documentation asks for prefixes, such as
search_query:andsearch_document:fornomic-embed-text-v1.5, return vectors of lower quality without the prefixes.
FastEmbed
FastEmbedEmbeddings runs ONNX models with the FastEmbed library. The model field selects either a built-in model or a custom model:
- Built-in model.
parametersis empty ({}), andmodelis one of the names in the following table. - Custom ONNX model.
parametersis not empty, andmodelnames a model directory, as described in Custom ONNX models.
model | Dimensions | Files in the API server image |
|---|---|---|
Qdrant/all-MiniLM-L6-v2-onnx | 384 | Yes (seeded model) |
BAAI/bge-small-en-v1.5 | 384 | Yes |
nomic-ai/nomic-embed-text-v1.5 | 768 | Yes |
BAAI/bge-small-zh-v1.5 | 512 | No |
Snowflake/snowflake-arctic-embed-xs | 384 | No |
Xenova/all-MiniLM-L12-v2 | 384 | No |
jinaai/jina-embeddings-v2-base-en | 768 | No |
mixedbread-ai/mxbai-embed-large-v1 | 1024 | No |
intfloat/multilingual-e5-small | 384 | No |
onnx-community/embeddinggemma-300m-ONNX | 768 | No |
The API server image carries the files of the models marked Yes in /data/fastembed, and the workers run the same image. FastEmbed downloads the files of any other built-in model from Hugging Face the first time each process loads the model, into /data/fastembed or, when that directory is read-only, into the writable cache that the F4AI_FASTEMBED_CACHE environment variable names. The Helm charts set the cache to /cache/fastembed, a temporary volume of each pod. A process started without the variable uses /tmp/foundation4ai-fastembed. A download fails in an air-gapped deployment, and the files are lost when the pod restarts.
A model image mounted at /data/fastembed replaces the directory, so such an image includes every FastEmbed model that the deployment uses, including the seeded model. The mounted image is read-only, so a built-in model whose files the image does not include downloads into the writable cache, which requires internet access. A model that loads from neither directory returns HTTP 400 with Invalid parameter, and details.error names the model and the failure in each directory.
Two further names are accepted with empty parameters: sentence-transformers/all-MiniLM-L6-v2 and sentence-transformers/all-MiniLM-L12-v2, which Foundation4 loads as custom ONNX models with mean pooling. The API server image carries the files of sentence-transformers/all-MiniLM-L6-v2. Like any custom model, these two names load only from local files and are never downloaded, and no Foundation4 image carries the files of sentence-transformers/all-MiniLM-L12-v2, so creating that model returns HTTP 400 unless a model image adds the files. Any other built-in model name that the table does not list returns HTTP 400 with Unknown FastEmbed embedding model: <name>. intfloat/multilingual-e5-small is the only multilingual built-in model.
Custom ONNX models
A custom model is any ONNX sentence embedding model stored in the FastEmbed model directory in the Hugging Face cache layout. FastEmbed reads custom models from local files only and never downloads them.
| Parameter | Type | Required | Values |
|---|---|---|---|
onnx | String | Yes | The path of the ONNX file inside the model snapshot, such as onnx/model.onnx |
pooling | String | No | cls or mean |
quantization | String | No | static or dynamic |
data_files | Array of strings | No | Not supported: Foundation4 reads only the ONNX file, so a model that stores weights in separate data files cannot be loaded. A list that names a file returns HTTP 400. |
Other parameters return HTTP 400. The model files are read from the following directory, where the model value has each / replaced by --:
<paths.fastembed>/models--<model>/snapshots/<snapshot>/
The snapshot directory holds the ONNX file and the files tokenizer.json, config.json, special_tokens_map.json and tokenizer_config.json. The model directory holds exactly one snapshot. A missing file returns HTTP 400 with Error reading <file> in details.error.
OpenAI-compatible embeddings
OpenAIEmbeddings calls a server that implements the OpenAI embeddings API, such as an inference server inside the deployment's network.
| Parameter | Type | Default | Description |
|---|---|---|---|
endpoint | String | https://api.openai.com/v1 | The base URL of the embeddings server. An air-gapped deployment sets an internal server. |
api_key | String | Empty | The key that the server requires. A server without authentication needs no key. |
dimensions | Integer | The model's native dimension | The number of dimensions to request from a model that can shorten its vectors, such as text-embedding-3-large, which returns 3,072 dimensions by default. Must be greater than 0. |
The model field is the model name that the server expects. Other parameters return HTTP 400.
GPT4All
GPT4AllEmbeddings runs GGUF embedding models with the GPT4All library. The model field is the name of a model file in the GPT4All model directory.
| Artifact | Content |
|---|---|
Package image gpt4all | The GPT4All Python package, mounted into the gRPC service |
Model image gpt4all | all-MiniLM-L6-v2.gguf2.f16.gguf (384 dimensions) and nomic-embed-text-v1.5.f16.gguf (768 dimensions) |
The standard gRPC service image does not include the GPT4All package, so the provider requires the package image. Parameters are passed to the LangChain GPT4AllEmbeddings class, such as n_threads and device. Foundation4 sets the model name and the model path, so parameters cannot change them.
Hugging Face
HuggingFaceEmbeddings runs Sentence Transformers models. The model field is the name of a model in the Hugging Face model directory, such as sentence-transformers/all-mpnet-base-v2.
| Artifact | Content |
|---|---|
Package image sentence-transformers | The Sentence Transformers package with PyTorch, mounted into the gRPC service |
Model image huggingface | sentence-transformers/all-mpnet-base-v2 (768 dimensions) |
Parameters are passed to the LangChain HuggingFaceEmbeddings class: model_kwargs (options for the model, such as device), encode_kwargs (options for encoding, such as normalize_embeddings), multi_process and show_progress. Foundation4 sets the model name and the cache directory, so parameters cannot change them. The provider loads the model files from the Hugging Face model directory first, and downloads a missing model from the Hugging Face Hub. With HF_HUB_OFFLINE=1 set on the gRPC service, a missing model is not downloaded: creating the embedding model returns HTTP 500 Internal server error without details, and the gRPC service log shows Provider unavailable with the model name.
HuggingFaceEndpointEmbeddings calls a Hugging Face inference endpoint. The parameter huggingfacehub_api_token is required, and a create request without the token returns HTTP 400 with Invalid parameter, and details.error ends with API token is required for HuggingFaceEndpoint embeddings. The model field names the model that the endpoint serves.
Text splitter providers
Every text splitter runs in the gRPC service and wraps a text splitter of the LangChain text splitters library.
provider | Availability | Network access |
|---|---|---|
RecursiveCharacterTextSplitter | Available | None |
CharacterTextSplitter | Available | None |
CodeTextSplitter | Available | None |
MarkdownHeaderTextSplitter | Available. Each fragment begins with the headings of its section and of the parent sections. | None |
TokenTextSplitter | Available | None, with the encoding files in /data/tiktoken or in the directory that TIKTOKEN_CACHE_DIR names on the gRPC service. Otherwise the tiktoken library downloads the files. |
NLTKTextSplitter | Available | None, with the NLTK data from the nltk model image |
SpacyTextSplitter | Requires the spacy package image | None. The default sentencizer pipeline needs no model files, and a named pipeline is read from /data/spacy/<pipeline>, never downloaded. |
SentenceTransformersTokenTextSplitter | Requires the sentence-transformers package image | As for HuggingFaceEmbeddings: none for a model in the Hugging Face model directory |
RecursiveJsonSplitter | Not available: creation fails with HTTP 400 | Not applicable |
Parameters are passed to the LangChain class as keyword arguments. A parameter that the class does not accept returns HTTP 400 when the splitter is created. A value that the class rejects, such as an invalid language or an overlap larger than the fragment size, returns HTTP 400 with Invalid parameter, and details.error carries the message of the class. A splitter whose package or data files are missing returns HTTP 503 with Provider unavailable, and details.error names what is missing.
Common parameters
The character, recursive character, code, token, NLTK and spaCy splitters accept the following parameters. The defaults are those of the LangChain text splitters library, version 1.1.1.
| Parameter | Type | Default | Description |
|---|---|---|---|
chunk_size | Integer | 4000 | The maximum size of a fragment, in characters, or in tokens for TokenTextSplitter |
chunk_overlap | Integer | 200 | The maximum size of the overlap between consecutive fragments. Must not exceed chunk_size. |
keep_separator | Boolean, "start" or "end" | true for the recursive splitters, false for the others | Whether the separator stays in the fragments, and at which end |
strip_whitespace | Boolean | true | Whether leading and trailing whitespace is removed from each fragment |
Provider parameters
provider | Parameters |
|---|---|
RecursiveCharacterTextSplitter | separators: the list of separators, tried in order. The default is ["\n\n", "\n", " ", ""]. is_separator_regex: whether the separators are regular expressions. |
CharacterTextSplitter | separator: the separator. The default is "\n\n". Text without the separator stays one fragment, whatever the size. is_separator_regex. |
CodeTextSplitter | language (required): a LangChain language name, such as python, js, java, go, rust, markdown or html. An invalid language returns HTTP 400, and details.error ends with Unspecified or invalid language: <language>. |
MarkdownHeaderTextSplitter | headers_to_split_on (required): the headings that start a new fragment, such as [["#", "Header 1"], ["##", "Header 2"]]. Fragment size follows the sections of the document, with no size limit. |
TokenTextSplitter | encoding_name: the tiktoken encoding. The default is gpt2. model_name: selects the encoding of a model instead. |
NLTKTextSplitter | separator: the separator between sentences in a fragment. The default is "\n\n". language: the NLTK language. The default is english. |
SpacyTextSplitter | pipeline: sentencizer, the rule-based sentence splitter of spaCy, or the name of a spaCy pipeline, such as en_core_web_sm. The default is sentencizer. separator: the separator between sentences in a fragment. The default is "\n\n". |
SentenceTransformersTokenTextSplitter | model_name: the Sentence Transformers model whose tokenizer counts the tokens. The default is sentence-transformers/all-mpnet-base-v2. tokens_per_chunk: the fragment size in tokens. The default is the maximum input length of the model. chunk_overlap: the overlap in tokens. The default is 50. |
Seeded objects
Every installation includes the following objects:
| Object | Identifier | Provider | Configuration |
|---|---|---|---|
Embedding model all-MiniLM-L6-v2 | 9ba4a409-8773-415b-aa9d-ae3a2bdbc775 | FastEmbedEmbeddings | Qdrant/all-MiniLM-L6-v2-onnx, 384 dimensions |
| Text splitter | 019252e9-b4a0-7713-9a69-d701b4f4a2d1 | RecursiveCharacterTextSplitter | chunk_size 1,000 and chunk_overlap 100. An installation upgraded from an earlier release keeps the default parameters: 4,000 characters, 200 characters of overlap. |
| Text splitter | 019252e9-da22-7f12-8c2f-36f4025fb0af | CharacterTextSplitter | Default parameters |
| Text splitter | 019252e9-f2b1-7482-930f-0cf9400bdd79 | NLTKTextSplitter | Default parameters. Requires the NLTK data from the nltk model image. |
Each seeded text splitter carries a description that names its splitting method.
Model and package images
Models and Python packages that the standard images do not include are delivered as separate images, which the Helm charts mount into the containers:
| Image | Content | Mounted at |
|---|---|---|
| API server image | The Foundation4 binary and the FastEmbed models marked Yes in the FastEmbed table, in /data/fastembed | Not applicable |
| gRPC service image | The Python service, the LangChain libraries, the NLTK package and the tiktoken library | Not applicable |
Package image gpt4all | The GPT4All package | /packages/gpt4all in the gRPC service |
Package image sentence-transformers | Sentence Transformers and PyTorch | /packages/sentence-transformers in the gRPC service |
Package image spacy | The spaCy package, without language models | /packages/spacy in the gRPC service |
Model image gpt4all | Two GPT4All embedding models | /data/gpt4all |
Model image huggingface | sentence-transformers/all-mpnet-base-v2 | /data/huggingface |
Model image nltk | The NLTK sentence data punkt_tab | /data/nltk |
Model image tiktoken | The tiktoken encodings gpt2, r50k_base, p50k_base, cl100k_base and o200k_base | /data/tiktoken |
An air-gapped installation mirrors every image that the deployment uses. The CLI command foundation4ai data check-models checks that every configured embedding model and text splitter loads, without downloading FastEmbed model files, as described in Command-line interface. The API server, worker and gRPC service containers mount each model image at the same path. Models, packages and air-gapped installs describes the image configuration and a check of the files that each container sees.