Search requests
This page lists every field of a search request, every search type with the parameters and defaults of the type, the classification forms, the distance strategies, the response and the errors that a search request returns. Integrators use this page as the reference while writing search calls. Search and retrieval explains the search modes and when to use each mode.
Request
POST /pipelines/{id}/search runs one search on the pipeline with the identifier id. The API key requires read and execute permission on the pipeline (permission 5). The request carries the authentication headers described in Authenticate and a JSON body. The example searches the support-kb pipeline from First search.
curl -X POST "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/search" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "Content-Type: application/json" \
-d '{
"query": "How do customers get their money back?",
"type": "similarity",
"classification": ["internal"],
"params": {"k": 5},
"filters": {"product": {"$eq": "billing"}}
}'
Expected result: status 200 and an array of up to 5 fragments, described in Response. On support-kb, the array holds two fragments, from the kb-refund-policy and kb-invoice-export documents, because the filter excludes the kb-password-reset document.
Body fields
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
query | String | Yes | None | The search text. Similarity and MMR search convert the text into a vector, full-text search matches the words of the text, and hybrid search does both. History search ignores the text. |
type | String | Yes | None | The search type: similarity, mmr, search, hybrid or history, described in Search types. Values are lowercase and case-sensitive. |
classification | String, array or object | Yes | None | The classifications that the reader holds, described in Classification. |
params | Object | No | The defaults of the search type | The parameters of the search type, described in Parameters. |
filters | Object or null | No | No filter | A metadata filter, described in Filter operators. An empty object {} applies no restriction. |
distance_strategy | String or null | No | The first indexed strategy | The distance function for the vector indexes of the search, described in Distance strategies. Full-text and history search ignore the field. |
path | String or null | No | Documents without a path | A path pattern: a.b matches that path, a.b.* the paths one level below a.b, and a.b.> the paths at any depth below a.b. The search covers only the documents whose path matches. Without the field, or with null, the search covers only the documents without a path. Document paths describes the patterns. |
A field that the body does not define returns an error, so a misspelled or misplaced field does not change the search silently. The following rules apply:
- Top level. An unknown field at the top level of the body, such as
filterinstead offiltersorkoutsideparams, returns HTTP 400 with the messageUnknown field `<name>`, anddetails.expectedlists the fields of the body. - Parameters. A field inside
paramsthat the search type does not define, such asfetch_kin a similarity search orthresholdin a hybrid search, returns HTTP 422 with a message that names the field and the fields of the search type, as inFailed to deserialize the JSON body: unknown field `fetch_k`, expected one of `embedding`, `index`, `k`, `threshold`for a similarity search. The list includes the parameters of earlier releases that the search type held,embeddingandsearch_type, although a search that sends them returns HTTP 400, as described in Legacy parameters. - Classification object. The object form of
classificationignores fields that the object does not define.
The request has no field for pagination, for a point in time or for the full-text language. A search always reads the current version of each document and returns one list of results.
Search types
type | Search | Search indexes read | Parameters |
|---|---|---|---|
similarity | The k fragments whose vectors are closest to the query vector | One vector index | index, k, threshold |
mmr | Maximal marginal relevance (MMR) search: k fragments selected from the fetch_k closest fragments for relevance and diversity | One vector index | index, k, fetch_k, lambda_mult, threshold |
search | Full-text search: up to k fragments that match the words of the query, ranked by relevance | One text index: a full-text index or a BM25 index | index, k, threshold |
hybrid | The result lists of several search indexes, fused by reciprocal rank fusion (RRF) | Any number and mix of vector and text indexes | indexes, k |
history | The k most recent fragments, newest document version first | None | k |
The API defines one further value, query, a placeholder type for agents, which returns HTTP 400 with the message Invalid search type or parameters in a search request. Any other value returns HTTP 422. Search and retrieval explains how each type selects and ranks fragments.
Parameters
| Parameter | Type | Search types | Default | Description |
|---|---|---|---|---|
index | Identifier | similarity, mmr, search | The pipeline's default index of the kind | The identifier of the search index to read: a vector index for similarity and mmr, a full-text or BM25 index for search. |
indexes | Array of identifiers | hybrid | The default vector index and the default text index | The identifiers of the search indexes to combine. Order and repeated identifiers have no effect. |
k | Integer, 1 to 65535 | All | 5 | The number of fragments to return. For hybrid, also the number of fragments that each index contributes. |
fetch_k | Integer, 1 to 65535, at least k | mmr | 10 | The number of candidates from which MMR search selects k fragments. |
lambda_mult | Number, 0 to 1 | mmr | 0.5 | The balance between relevance and diversity. A value of 1 selects by relevance alone, and a value of 0 maximizes diversity. |
threshold | Number | similarity, mmr, search | None | For similarity and MMR search, the maximum distance from the query. For full-text search, the minimum score. Fragments outside the threshold are excluded before k applies. Hybrid and history search do not define the field. |
Every parameter that the request omits takes the default in this table, whether the request omits the whole params object or one field inside the object. The following request runs an MMR search for 5 fragments from 20 candidates on a named vector index:
{
"query": "How do customers get their money back?",
"type": "mmr",
"classification": "internal",
"params": {"index": "5d1e8c2a-3f4b-4a6c-9e7d-1b2c3d4e5f60", "k": 5, "fetch_k": 20}
}
Foundation4 checks the values before the search runs. A k or fetch_k of 0, a fetch_k smaller than k or a lambda_mult outside 0 to 1 returns HTTP 400 with Invalid parameter, and details.error names the rule, as listed in Errors. A value above 65535, or a value of another type, returns HTTP 422.
Legacy parameters
Earlier releases selected an embedding model in params.embedding and a full-text method in params.search_type. A search that sends these parameters returns HTTP 400, so that a client application written for an earlier release fails visibly instead of searching other indexes than intended. The message names the replacement:
type | Parameter sent | Message |
|---|---|---|
similarity, mmr | embedding | `embedding` is not supported in pipeline searches: pass a search indexer id in `index` |
search | search_type | `search_type` is not supported in text searches: pass the text search indexer id in `index` |
hybrid | A non-empty embedding or search_type array | `embedding` and `search_type` are not supported in hybrid searches: pass the search indexer ids to combine in `indexes` |
Classification
The classification field names the classifications that the reader holds. Search returns only fragments whose classification is resolved from the request. Classifications describes inheritance and the role of the client application.
| Form | Example | Matching |
|---|---|---|
| String | "classification": "internal" | Hierarchical |
| Array of strings | "classification": ["internal", "public"] | Hierarchical |
| Object | "classification": {"classifications": ["internal"], "search_type": 1} | As set by search_type |
In the object form, the array is named classifications, in the plural, and search_type is optional. Any other form returns HTTP 422: null or a number returns the message Failed to deserialize the JSON body: classification must be a string, an array of strings, or an object with classifications and search_type, and an object without classifications returns a message that names the missing field.
search_type | Matching | Undefined classification names |
|---|---|---|
2 (default) | Hierarchical: the named classifications and every classification that the named classifications inherit, transitively | Ignored |
1 | Exact: the named classifications only | HTTP 404 with the message Classification not found |
3 | Hierarchical with validation: as 2 | HTTP 404 with the message Classification not found |
search_type is a bit field: the value 2 adds the inherited classifications, and the value 1 rejects undefined names. Any other value, including 0, returns HTTP 422 with the message Failed to deserialize the JSON body: invalid classification search type <value>: expected 1 (exact), 2 (hierarchical) or 3 (both).
An empty array, or an array in which no name is defined in the pipeline under hierarchical matching, resolves no classification, and the search returns an empty array with status 200.
The API key bounds the classifications that a request can name. Every named classification that is defined in the pipeline must be in the classification allow-list of the API key, or in a classification that the allow-list inherits, or the search returns HTTP 403, and details.error names the first classification outside the allow-list. A key whose allow-list is * can name every classification. Access control describes the allow-list.
Distance strategies
The distance strategy determines how a vector search measures the distance between the query vector and each fragment vector. The strategy applies to every vector index that the search reads, and each strategy requires a distance index on the vector index, as described in Search indexes.
distance_strategy | Distance | Score range | Better match |
|---|---|---|---|
cosine | 1 minus the cosine similarity | 0 to 2 | Lower |
euclidean-distance | Euclidean (L2) distance | 0 upward | Lower |
dot-product | Negative inner product | Negative for similar vectors | Lower |
A request without distance_strategy, or with null, uses the first strategy that the vector index has a distance index for, in the order cosine, euclidean-distance, dot-product. A new vector index has a distance index for cosine only. A request that names a strategy without a distance index returns HTTP 400, and the message lists the indexed strategies. The value max-inner-product is defined in the API but not supported, and returns HTTP 400.
The distance strategy matches the strategy for which the embedding model was trained, which is cosine for most sentence embedding models, including the default model.
Full-text search
Full-text search ("type": "search") has the following requirements and behavior:
- Text index. The search reads the index named in
params.index, or the pipeline's default text index. A pipeline without a text index returns HTTP 400, andPOST /pipelines/{id}/search-indexersadds one, as described in Search indexes. - Full-text index. The query is plain text. After stemming and the removal of common words, every remaining word must appear in a fragment. The query syntax has no phrase, prefix,
ORorNOToperators. The API server parses the query with the language configuration that each fragment was indexed with, so a query matches fragments of every indexed language. A fragment indexed by an earlier release, which stored no configuration, is matched with the configuration of the language that the API server detects in the query. Full-text languages lists the languages and describes detection. The score is the PostgreSQL rank of the fragment, divided by 1 plus the logarithm of the fragment length. - BM25 index. The index's tokenizer splits the query into tokens, and every fragment that shares a token with the query is ranked. The score is the BM25 score of the fragment, greater than 0.
- Threshold.
thresholdsets a minimum score. - No embedding. Full-text search makes no embedding call, so
distance_strategydoes not apply.
Hybrid search
Hybrid search ("type": "hybrid") searches each index in params.indexes, or the default vector index and the default text index, for the k best fragments. Foundation4 fuses the lists by reciprocal rank fusion: a fragment at rank r in a list, counting from 1, receives 1 / (60 + r), and the fused score is the sum over the lists. The response holds the first k fragments by descending fused score. The constant 60 is fixed, every list has the same weight, and a threshold in params returns HTTP 422. Set up hybrid search describes hybrid search with a worked example.
History search
History search ("type": "history") returns the k most recent fragments that match the classifications and the filter, ordered by document version, newest first, and by position within a document version. query is required but ignored, the search reads no search index, and the fragments carry no score field.
Response
A successful search returns status 200 and a JSON array of fragments. The array is not wrapped in the list envelope described in API conventions.
| Field | Type | Description |
|---|---|---|
id | Identifier | The fragment identifier |
classification | String | The classification of the fragment's document |
metadata | Object | The metadata of the fragment's document |
path | String or null | The path of the fragment's document, or null when the document has no path |
document_id | Identifier | The identifier of the document |
page_content | String | The decrypted text of the fragment. The field is absent when the text is empty. |
version | Integer | The document version, in microseconds since the Unix epoch |
expired_at | Timestamp or null | Always null in search results, because search reads current versions only |
score | Number | The score of the fragment, described in the order rules below. Absent in history search. |
order_id | Integer | The position of the fragment within the document version, starting at 0 |
The following rules govern the order and the number of results:
- Similarity and MMR search. Fragments are ordered by ascending
score, the distance from the query. MMR search returns the selected fragments in order of distance, not in order of selection, andscoreis the distance, not the MMR score. - Full-text search. Fragments are ordered by descending
score, the rank or the BM25 score. - Hybrid search. Fragments are ordered by descending
score, the fused score. - History search. Fragments are ordered by document version, newest first, and by
order_idwithin a version. - Number of results. A search returns at most
kfragments, and MMR search at most the smaller ofkandfetch_k. A search returns fewer fragments when fewer fragments match the classifications, the filter and the threshold. - Comparison across types. Scores of vector search and full-text search run in opposite directions and cannot be compared. Hybrid search combines the two by rank position.
The following response contains the first fragment of the example in Request:
[
{
"id": "01a0ed0d-e007-7c3d-8e4f-6a7b8c9d0e1f",
"classification": "internal",
"metadata": {"product": "billing"},
"path": null,
"document_id": "01a0ed0d-df3f-7f80-9a1b-2c3d4e5f6a7b",
"page_content": "Refunds for annual plans are prorated. Support agents must record the refund reason in the ticket before approving a refund.",
"version": 1790683504447128,
"expired_at": null,
"score": 0.6445,
"order_id": 0
}
]
Each response carries the x-request-id header described in API conventions.
Errors
The following errors are specific to search requests. Errors describes every error of the API.
| Status | Message | Cause |
|---|---|---|
| 400 | Schema error | The filter names a field that the pipeline schema does not declare, or compares a field with a value of another type |
| 400 | Invalid parameter | A search parameter is out of range, or a search index or a distance strategy cannot be used. details.error names the problem, as listed below. |
| 400 | Unknown field `<name>` | The body holds a top-level field that the search request does not define. details.expected lists the fields of the body. |
| 400 | A legacy parameter message | params contains embedding or search_type, as listed in Legacy parameters |
| 400 | Invalid search type or parameters | type is query |
| 403 | Unauthorized: execute access to Pipeline required | A named classification is outside the classification allow-list of the API key. details.error names the classification, and details.id holds the pipeline identifier. |
| 404 | Pipeline not found | The pipeline does not exist, or the API key lacks read and execute permission on the pipeline. details.id holds the pipeline identifier. |
| 404 | Classification not found | search_type is 1 or 3 and a named classification is not defined in the pipeline. The response does not name the classification. |
| 409 | Conflict | A search index or a distance index is not ready. details.error names the index, as listed below. |
| 422 | Failed to deserialize the JSON body: <reason> | A required field is missing, a field has the wrong type, type or distance_strategy has an unknown value, classification has an invalid form or search_type an invalid value, params holds a field that the search type does not define or a value above 65535, path is not a valid path pattern, or the filter does not follow the filter grammar. The reason names the problem without a line or column. |
| 500 | Internal server error | The embedding model could not convert the query into a vector, the query language could not be detected, or the database rejected the query, for example because stored metadata cannot be compared as the type that the schema declares. The API server log records the cause. |
details.error of an Invalid parameter or Conflict error holds one of the following messages:
| Status | details.error | Cause |
|---|---|---|
| 400 | `k` must be greater than 0, `fetch_k` must be greater than 0 | k or fetch_k is 0. |
| 400 | `fetch_k` cannot be smaller than `k` | An MMR search sets fetch_k below k. |
| 400 | `lambda_mult` must be between 0 and 1 if specified | An MMR search sets lambda_mult outside 0 to 1. |
| 400 | No search indexer given and the pipeline has no default vector search indexer | A similarity or MMR search names no index, and the pipeline has no default vector index. |
| 400 | No search indexer given and the pipeline has no default text search indexer | A full-text search names no index, and the pipeline has no default text index. |
| 400 | No default vector search indexer configured for this pipeline, No default text search indexer configured for this pipeline | A hybrid search names no index, and the pipeline lacks the named default. |
| 400 | Not a vector search indexer of this pipeline: <id>, Not a text search indexer of this pipeline: <id>, Not a search indexer of this pipeline: <id> | The named index does not exist, belongs to another pipeline or is of the wrong kind for the search type. |
| 400 | The <distance> distance strategy has no index on search indexer <id>, so searching it would scan the whole table; ... | distance_strategy names a strategy without a distance index. The rest of the message lists the indexed strategies. |
| 400 | Search indexer <id> has no distance index, so it cannot be searched; ... | The vector index has no distance index at all. |
| 400 | The max-inner-product distance strategy is not supported | distance_strategy is max-inner-product. |
| 409 | Search indexer <id> is still being built and cannot be searched yet | The index is building. |
| 409 | Search indexer <id> failed to build and cannot be searched | The index is failed. |
| 409 | The index for the <distance> distance on search indexer <id> is still being built | The distance index of the requested strategy is being built. |
Other search operations
Two other operations retrieve fragments with the same search engine:
- Agent retrieval dry run.
POST /agents/{id}/searchruns the retrieval placeholders of an agent without calling an LLM. The search parameters come from the agent's placeholders, andfiltersmaps each placeholder name to a filter. Agents and prompt templates describes the operation. - MCP search tool. The
search_knowledgetool of the MCP server accepts a pipeline, a query, an array of classifications (matched hierarchically unlessexact_classificationsistrue), a search type, search index identifiers, a path pattern, a metadata filter and the parameters of the search type as flat arguments. The tool names full-text searchtextand history searchlatest, and accepts no distance strategy. MCP tools describes the tool.