Pipelines
Pipelines, the classifications of a pipeline and the metadata indexes of a pipeline. Pipelines describes the settings of a pipeline.
List pipelines
Returns a page of pipelines that the key can read, each with its search indexes and their statuses. The response is a paginated envelope. Results can be filtered by field and ordered, as described in [API conventions](../01-api-conventions.md).
Create a pipeline
Creates a pipeline with its first search indexes. `embedding_model_id` creates a vector index on VectorChord for similarity and maximal marginal relevance (MMR) search, and `full_text_search` creates a full-text index (`fts`) or a BM25 index (`bm25`) for full-text search, with the tokenizer in `bm25_tokenizer`. Each index becomes the default index of its kind. A pipeline created with neither field has no search index and supports only history search. The search indexes are not fixed at creation: the operations under `/pipelines/{pipeline_id}/search-indexers` add, switch and remove indexes later.
Get a pipeline
Returns a pipeline by identifier. The response body is the pipeline with its search indexes: the identifiers of the default vector index and the default text index, every search index of the pipeline by identifier in `indexers`, and the status of each index in `indexer_status`.
Update a pipeline
Changes the name, description, default text splitter or metadata schema of a pipeline. Fields that the request omits keep their values, and `'description': null` clears the description. A new default text splitter applies to the documents submitted after the change. Search indexes and classifications are not changed through this operation: a request with an index field, or any other unknown field, returns HTTP 422, the operations under `/pipelines/{pipeline_id}/search-indexers` add, switch and remove indexes, and `POST /pipelines/{pipeline_id}/classifications` changes the classifications. The response body is the pipeline with the new settings.
Delete a pipeline
Deletes a pipeline, every document and version in the pipeline, the search indexes of the pipeline and the pipeline's tables. The deletion is permanent. The response has no body.
List pipeline classifications
Returns the classifications of the pipeline that the classification allow-list of the key covers, and the inheritance relationships that the key can reach from those classifications, at any depth. A key with the `*` allow-list receives every classification and every relationship, and a key with an empty allow-list receives two empty lists. The `hierarchy` field is a list of pairs, each holding a classification and the inherited classification.
Modify pipeline classifications
Adds and removes classifications and inheritance relationships in one transaction. The request body holds an `add` list and a `remove` list, each using the classification forms of the create request. Additions are processed before removals. Removing a classification permanently deletes every document that carries the classification. The response has no body.
List metadata indexes
Returns the metadata indexes defined on a pipeline: database indexes on metadata fields that speed up metadata filters. Metadata indexes are distinct from the search indexes of the pipeline. The response body is a JSON array. Each element holds the index identifier and the metadata fields the index covers.
Create a metadata index
Creates a metadata index on one or more top-level metadata fields of a pipeline. Each field named in `columns` must be a top-level field declared in the pipeline's metadata schema with a scalar type. The index uses the same expression as a metadata filter on the field, so filters on the field use the index. The response body is the new index.
Get a metadata index
Returns a metadata index of a pipeline by identifier. The response body holds the index identifier and the fields the index covers.
Delete a metadata index
Deletes a metadata index of a pipeline. The response has no body.