Search requests

This page lists every field of a search request, every search type with the parameters and defaults of the type, the classification forms, the distance strategies, the response and the errors that a search request returns. Integrators use this page as the reference while writing search calls. Search and retrieval explains the search modes and when to use each mode.

Request

POST /pipelines/{id}/search runs one search on the pipeline with the identifier id. The API key requires read and execute permission on the pipeline (permission 5). The request carries the authentication headers described in Authenticate and a JSON body. The example searches the support-kb pipeline from First search.

curl -X POST "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/search" \
  -H "x-api-key: $FOUNDATION4_API_KEY" \
  -H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "How do customers get their money back?",
    "type": "similarity",
    "classification": ["internal"],
    "params": {"k": 5},
    "filters": {"product": {"$eq": "billing"}}
  }'

Expected result: status 200 and an array of up to 5 fragments, described in Response. On support-kb, the array holds two fragments, from the kb-refund-policy and kb-invoice-export documents, because the filter excludes the kb-password-reset document.

Body fields

Field	Type	Required	Default	Description
query	String	Yes	None	The search text. Similarity and MMR search convert the text into a vector. Full-text search matches the words of the text.
type	String	Yes	None	The search type: similarity, mmr or search, described in Search types. Values are lowercase and case-sensitive.
classification	String, array or object	Yes	None	The classifications that the reader holds, described in Classification.
params	Object	No	The defaults of the search type	The parameters of the search type, described in Parameters.
filters	Object or null	No	No filter	A metadata filter, described in Filter operators. An empty object {} applies no restriction.
distance_strategy	String or null	No	cosine	The distance function for similarity and MMR search, described in Distance strategies. Full-text search ignores the field.

Foundation4 ignores fields that the body does not define, at every level of the body. A field placed at the wrong level, such as k at the top level instead of inside params, therefore has no effect and returns no error. A misspelled field, such as filter instead of filters, is ignored in the same way, so the search runs without a filter.

The request has no field for pagination, for a point in time or for the full-text language. A search always reads the current version of each document and returns one list of results.

Search types

type	Search	Parameters
similarity	The k fragments whose vectors are closest to the query vector	k, threshold, embedding
mmr	Maximal marginal relevance (MMR) search: k fragments selected from the fetch_k closest fragments for relevance and diversity	k, fetch_k, lambda_mult, threshold, embedding
search	Full-text search: the k fragments that contain every word of the query, ranked by relevance	k, threshold

The API defines three further values. hybrid and history return HTTP 501 in the current release. query is a placeholder type for agents and returns HTTP 400 with the message Invalid search type or parameters in a search request. Any other value returns HTTP 422.

Parameters

Parameter	Type	Search types	Default	Description
k	Integer, 1 to 65535	All	5	The number of fragments to return. A value of 0 returns no fragments.
fetch_k	Integer, 1 to 65535	mmr	10	The number of candidates from which MMR search selects k fragments. Set to at least k, because the response contains at most fetch_k fragments.
lambda_mult	Number, 0 to 1	mmr	0.5	The balance between relevance and diversity. A value of 1 selects by relevance alone, and a value of 0 maximizes diversity. Values outside 0 to 1 are not rejected.
threshold	Number	All	None	For similarity and MMR search, the maximum distance from the query. For full-text search, the minimum rank. Fragments outside the threshold are excluded before k applies.
embedding	Identifier	similarity, mmr	The pipeline's embedding model	The identifier of the embedding model to search with. The model must belong to the pipeline.

The defaults in this table apply only when the request omits the params object. When the request includes params, every parameter that params omits takes the value 0 or no value, so k and fetch_k become 0 and the search returns no fragments. A request that sets params therefore sets k, and for MMR search also fetch_k, explicitly. The following parameters return no fragments and no error:

type	params	Result
similarity	{"threshold": 0.7}	k is 0
mmr	{"k": 5}	fetch_k is 0, so no candidates exist
search	{}	k is 0

The following parameters return up to 5 fragments:

{"type": "mmr", "params": {"k": 5, "fetch_k": 20, "lambda_mult": 0.5}}

Classification

The classification field names the classifications that the reader holds. Search returns only fragments whose classification is resolved from the request. Classifications describes inheritance and the role of the client application.

Form	Example	Matching
String	"classification": "internal"	Hierarchical
Array of strings	"classification": ["internal", "public"]	Hierarchical
Object	"classification": {"classifications": ["internal"], "search_type": 1}	As set by search_type

In the object form, the array is named classifications, in the plural, and search_type is optional. Any other form, such as null, a number or an object without classifications, returns HTTP 422.

search_type	Matching	Undefined classification names
2 (default)	Hierarchical: the named classifications and every classification that the named classifications inherit, transitively	Ignored
1	Exact: the named classifications only	HTTP 404 with the message Classification not found
3	Hierarchical with validation: as 2	HTTP 404 with the message Classification not found
0	Exact: the named classifications only	Ignored

search_type is a bit field: the value 2 adds the inherited classifications, and the value 1 rejects undefined names. Any other value returns HTTP 422.

An empty array, or an array in which no name is defined in the pipeline under hierarchical matching, resolves no classification, and the search returns an empty array with status 200.

Distance strategies

The distance strategy determines how similarity and MMR search measure the distance between the query vector and each fragment vector. Each strategy uses a separate hierarchical navigable small world (HNSW) index.

distance_strategy	Distance	Score range	Better match
cosine (default)	1 minus the cosine similarity	0 to 2	Lower
euclidean-distance	Euclidean (L2) distance	0 upward	Lower
dot-product	Negative inner product	Negative for similar vectors	Lower

The value max-inner-product is defined in the API but not supported, and a search with this value fails. The distance strategy matches the strategy for which the embedding model was trained, which is cosine for most sentence embedding models, including the default model. MMR search uses cosine or dot-product, because MMR selection with euclidean-distance does not rank candidates as intended.

Full-text search

Full-text search ("type": "search") has the following requirements and behavior:

• Pipeline setting. The pipeline was created with has_full_text_search set to true. A full-text search on any other pipeline returns HTTP 501 with the message Full text search is not enabled for this pipeline. The setting cannot be changed after the pipeline is created.
• Query parsing. The query is plain text. After stemming and the removal of common words, every remaining word must appear in a fragment. The query syntax has no phrase, prefix, OR or NOT operators. An empty query matches no fragment.
• Language. The API server detects the language of the query and parses the query with that language's configuration. The request cannot set the language. Full-text languages lists the languages and describes detection.
• Rank. The score is the PostgreSQL rank of the fragment, divided by 1 plus the logarithm of the fragment length. threshold sets a minimum rank.
• No embedding. Full-text search makes no embedding call, so the embedding model and distance_strategy do not apply.

Response

A successful search returns status 200 and a JSON array of fragments. The array is not wrapped in the list envelope described in API conventions.

Field	Type	Description
id	Identifier	The fragment identifier
classification	String	The classification of the fragment's document
metadata	Object	The metadata of the fragment's document
document_id	Identifier	The identifier of the document
page_content	String	The decrypted text of the fragment. The field is absent when the text is empty.
version	Integer	The document version, in microseconds since the Unix epoch
expired_at	Timestamp or null	Always null in search results, because search reads current versions only
score	Number	The score of the fragment, described in the order rules below
order_id	Integer	The position of the fragment within the document version, starting at 0

The following rules govern the order and the number of results:

• Similarity and MMR search. Fragments are ordered by ascending score, the distance from the query. MMR search returns the selected fragments in order of distance, not in order of selection, and score is the distance, not the MMR score.
• Full-text search. Fragments are ordered by descending score, the rank.
• Number of results. A search returns at most k fragments, and MMR search at most the smaller of k and fetch_k. A search returns fewer fragments when fewer fragments match the classifications, the filter and the threshold.
• Comparison across types. Scores of vector search and full-text search run in opposite directions and cannot be compared. A client that combines both result lists merges the lists by rank position.

The following response contains the first fragment of the example in Request:

[
  {
    "id": "01a0ed0d-e007-7c3d-8e4f-6a7b8c9d0e1f",
    "classification": "internal",
    "metadata": {"product": "billing"},
    "document_id": "01a0ed0d-df3f-7f80-9a1b-2c3d4e5f6a7b",
    "page_content": "Refunds for annual plans are prorated. Support agents must record the refund reason in the ticket before approving a refund.",
    "version": 1790683504447128,
    "expired_at": null,
    "score": 0.6445,
    "order_id": 0
  }
]

Each response carries the x-request-id header described in API conventions.

Errors

The following errors are specific to search requests. Errors describes every error of the API.

Status	Message	Cause
400	Schema error	The filter names a field that the pipeline schema does not declare, or compares a field with a value of another type
400	Vector embedding not found	params.embedding names an embedding model that the pipeline does not use, or the pipeline has no embedding model. details.embedding_id holds the requested identifier.
400	Invalid search type or parameters	type is query
404	Pipeline not found	The pipeline does not exist, or the API key lacks read and execute permission on the pipeline. details.id holds the pipeline identifier.
404	Classification not found	search_type is 1 or 3 and a named classification is not defined in the pipeline. The response does not name the classification.
422	Plain-text parsing message	A required field is missing, a field has the wrong type, type or distance_strategy has an unknown value, or the filter does not follow the filter grammar
500	Internal error	The embedding model could not convert the query into a vector, or the query language could not be detected. details.error holds the cause.
500	A database error message	The filter names a nested metadata field, distance_strategy is max-inner-product, or stored metadata cannot be compared as the type that the schema declares
501	Full text search is not enabled for this pipeline	type is search on a pipeline without full-text search
501	Hybrid placeholder is not implemented yet, History placeholder is not implemented yet	type is hybrid or history

Other search operations

Two other operations retrieve fragments with the same search engine:

• Agent retrieval dry run. POST /agents/{id}/search runs the retrieval placeholders of an agent without calling an LLM. The search parameters come from the agent's placeholders, and filters maps each placeholder name to a filter. Agents and prompt templates describes the operation.
• MCP search tool. The search_knowledge tool of the MCP server accepts a pipeline, a query, an array of classifications (matched hierarchically), a search type and parameters. The tool accepts no filter and no distance strategy, and always uses cosine. MCP tools describes the tool.