Skip to main content

Filter operators

This page defines the metadata filter language: the structure of a filter, every operator with the value types that the operator accepts, taxonomy filters, the treatment of missing values and the errors that a filter returns. Integrators use this page as the reference while writing filters. Metadata, filters and taxonomies explains metadata schemas and taxonomies.

Where filters apply​

OperationFieldForm
POST /pipelines/{id}/searchfilters in the bodyOne filter
POST /agents/{id}/search and POST /agents/{id}/executefilters in the bodyAn object that maps each retrieval placeholder name to one filter. A placeholder without an entry is not filtered, and names that match no placeholder are ignored.
GET /pipelines/{id}/documentsmetadata query parameterOne filter as URL-encoded JSON

Every filter is checked against the metadata schema of the pipeline. The search and agent tools of the MCP server accept filters in the same language.

Filter structure​

A filter is a JSON object in one of two forms:

  • Field conditions. An object that maps each metadata field name to a comparison. All conditions in the object must be satisfied. An empty object, {}, applies no restriction.
  • Logical operator. An object with exactly one key, $and, $or or $not, that combines other filters.

The following grammar summarizes the structure:

filter = field-map | logical
field-map = { field: comparison, ... }
logical = { "$and": [filter, ...] } | { "$or": [filter, ...] } | { "$not": filter }
comparison = { operator: value, ... }

The following rules apply to every filter:

  • Several operators per field. A comparison object holds one or more operators, and the field must satisfy all of them. A range on one field therefore takes one comparison object, such as {"timestamp": {"$ge": "2026-09-01T00:00:00.000Z", "$lt": "2026-09-16T00:00:00.000Z"}}, and two conditions combined with $and give the same result. A comparison object without an operator, {}, is invalid.
  • Explicit operators. A field always takes a comparison object. The shorthand {"room_id": "GENERAL"} is invalid; the valid form is {"room_id": {"$eq": "GENERAL"}}.
  • Separate logical objects. A logical operator and field conditions cannot share one object. A filter that needs both places the field conditions inside the $and array.
  • Field names. A field name consists of letters, digits and underscores and does not start with a digit. A field inside a field of type object is named with a period between the names, such as author.name, at any depth, and the schema declares the field under the properties of the object.
  • Nesting. $and, $or and $not nest to any depth. The code sets no limit on depth or on the number of conditions, and the request body limit bounds the size of a filter.

Comparison operators​

OperatorMeaningField typesValue
$eqEqual toString, integer, number, booleanOne value
$neqNot equal toString, integer, number, booleanOne value
$gtGreater thanString, integer, number, booleanOne value
$geGreater than or equal toString, integer, number, booleanOne value
$ltLess thanString, integer, number, booleanOne value
$leLess than or equal toString, integer, number, booleanOne value
$likeMatches a pattern, case-sensitiveStringA pattern string
$ilikeMatches a pattern, case-insensitiveStringA pattern string
$inEqual to one of the valuesString, integer, number, booleanAn array of values of one type
$not-inEqual to none of the valuesString, integer, number, booleanAn array of values of one type
$teqEqual to a value or to any value beneath that value in a taxonomyStringAn array of two strings: the value and the taxonomy identifier

The operator names are exact. $ne, $gte, $lte, $nin, $exists, $contains and $regex are not operators of the filter language.

The following filters show each comparison operator:

OperatorExample
$eq{"room_id": {"$eq": "GENERAL"}}
$neq{"username": {"$neq": "system"}}
$gt{"priority": {"$gt": 2}}
$ge{"timestamp": {"$ge": "2026-09-01T00:00:00.000Z"}}
$lt{"rating": {"$lt": 0.5}}
$le{"timestamp": {"$le": "2026-09-15T00:00:00.000Z"}}
$like{"username": {"$like": "ali%"}}
$ilike{"title": {"$ilike": "%vpn%"}}
$in{"room_id": {"$in": ["GENERAL", "6f1c2a"]}}
$not-in{"username": {"$not-in": ["system", "rocket.cat"]}}
$teq{"region": {"$teq": ["Europe", "0b8d6f3e-5c1a-4e2b-9a7d-3f6e1c2b4a90"]}}

Value types​

The metadata schema of the pipeline declares the type of each field, and the value in a comparison must have that type. PostgreSQL converts the stored metadata value to the declared type and compares the converted values.

Schema typeAccepted JSON valuesComparison
stringStringsText comparison under the collation of the database, as described in Strings and dates
integerIntegers64-bit integer comparison
numberIntegers and decimalsDouble-precision comparison
booleantrue and falseBoolean comparison, where false is less than true
objectNoneFields of type object cannot be compared

null, arrays and objects are not valid comparison values. Metadata fields of type array cannot be declared in a schema, so no operator tests whether an array contains a value.

The conversion applies to the stored values as well. An integer field accepts a whole number written with a decimal point, such as 5.0, when the document is submitted, and Foundation4 stores the value as the integer 5. While a document stores a value with a decimal point in a field that a filter compares as integer, every such filter returns HTTP 500 Internal server error, for every document of the pipeline, and the API server log names the database error invalid input syntax for type bigint. Such a value remains in the following cases:

  • Large values. A whole number of 9e15 or more in magnitude, written with a decimal point, is stored as submitted.
  • Documents stored earlier. A document stored by an earlier release, or stored before a schema update declared the field as integer, has the value rewritten for the document list, but searches read a separate copy of the metadata. Search filters on the field then return HTTP 500 until the document is submitted again. Creating a metadata index on the field returns HTTP 500 as well, and submitting the document again does not clear that error.

A client application therefore sends whole numbers without a decimal point. An integer comparison value outside the 64-bit range returns HTTP 400 with Schema error.

Missing values​

A comparison is never satisfied by a document whose metadata lacks the field or holds null in the field. The rule applies to every comparison operator, including $neq and $not-in, and to $not: a $not is satisfied only by documents that hold every field named inside the $not. The filter {"$not": {"username": {"$eq": "system"}}} therefore excludes the documents of the system user and every document without a username field, and the filter {"$not": {"$and": [{"username": {"$eq": "system"}}, {"room_id": {"$eq": "GENERAL"}}]}} excludes every document that lacks username or room_id. The filter language has no operator that tests whether a field is present.

One case includes documents that lack the field: an empty $not-in list matches every document, as described in Lists.

A pipeline that filters on a field therefore sets the field in the metadata of every document, with a default value where the source has none.

Strings and dates​

The collation of the database decides the order of strings that differ in case or in punctuation. Under a linguistic collation, such as en_US.UTF-8, letters compare without regard to case first, so Beta is greater than b. Under the C collation, every uppercase letter sorts before every lowercase letter, so Beta is less than b. A client application that compares strings with $gt, $ge, $lt or $le stores the values in one case.

The filter language has no date type. Dates are stored as strings and compared as text, so date comparisons are chronological only when every stored value and every filter value use the same format. The format that Rocket.Chat uses serves as an example: Coordinated Universal Time (UTC) with milliseconds and the suffix Z, such as 2026-09-12T10:00:00.000Z. A schema that declares a format for the field, such as "format": "date-time" or "format": "date", refuses other values with HTTP 400 Invalid metadata, such as "11/11/2026" is not a "date" in details, as described in Metadata, filters and taxonomies. Without a format, the client application that writes metadata applies the format.

Patterns​

$like and $ilike use the pattern syntax of the SQL LIKE operator:

  • %. Matches any sequence of characters, including none.
  • _. Matches exactly one character.

A pattern without % or _ matches the whole value only. A backslash before % or _ matches the character itself, so the pattern 100\% matches the value 100% and not 100x. In JSON the backslash is doubled: {"label": {"$like": "100\\%"}}.

Lists​

$in and $not-in take an array whose values all have one JSON type, matching the field type. An array that mixes types, such as ["a", 1], is invalid.

An empty array is valid on string, integer, number and boolean fields alike. An empty $in list matches no document, and an empty $not-in list matches every document, including documents that lack the field.

Taxonomy filters​

$teq matches a value and every value beneath that value in a taxonomy. The operator takes a two-element array: the value, as a string, and the identifier of the taxonomy.

{"region": {"$teq": ["Europe", "0b8d6f3e-5c1a-4e2b-9a7d-3f6e1c2b4a90"]}}

The following rules apply to taxonomy filters:

  • Expansion. Foundation4 follows the taxonomy entries from the starting field and value to every descendant, at any depth and across fields, and matches documents that carry any of the values reached. Ancestors and siblings of the starting value do not match. Field names and values match exactly, including case.
  • Schema. The starting field and every field reached by the expansion must be declared in the pipeline schema as strings.
  • Current entries. The entries are read at search time, so changes to taxonomy entries apply to the next search without reprocessing any document.
  • Links per taxonomy. Each taxonomy holds its own links from a parent field and value to a child field and value. Two taxonomies can hold the same link, and $teq with either taxonomy expands through the link. A link that a taxonomy already holds is not stored twice. A link that an earlier release did not store, because another taxonomy held the link, stays missing until the link is added again.
  • Permissions. The API key requires read permission on the taxonomy. A taxonomy that the key cannot read returns HTTP 403 with Unauthorized: read access to Taxonomy required, and details.error holds Filtering with a taxonomy requires read access to it. An identifier that names no taxonomy returns HTTP 404 with Taxonomy not found and the identifier in details.id.
  • Value without entries. A value without entries in the taxonomy returns no error. The filter then matches the starting value only, as $eq does.

Logical operators​

OperatorOperandSatisfied when
$andAn array of filtersEvery filter in the array is satisfied. An empty array is always satisfied.
$orAn array of filtersAt least one filter in the array is satisfied. An empty array is never satisfied.
$notOne filter objectThe filter is not satisfied, subject to Missing values

Inside $and, an empty object, {}, applies no restriction. Inside $or, an empty object is ignored in any position, so {"$or": [{}, X]} matches the same fragments as X, and an $or that holds only empty objects matches nothing.

The following filter restricts results to two rooms and to messages sent from 1 September to 15 September 2026:

{
"$and": [
{"room_id": {"$in": ["GENERAL", "6f1c2a"]}},
{"timestamp": {"$ge": "2026-09-01T00:00:00.000Z"}},
{"timestamp": {"$le": "2026-09-15T23:59:59.999Z"}}
]
}

The same filter in the form of field conditions, with two operators on timestamp:

{
"room_id": {"$in": ["GENERAL", "6f1c2a"]},
"timestamp": {"$ge": "2026-09-01T00:00:00.000Z", "$le": "2026-09-15T23:59:59.999Z"}
}

The following filter matches messages in the room GENERAL, or messages from the user alice in any room:

{
"$or": [
{"room_id": {"$eq": "GENERAL"}},
{"username": {"$eq": "alice"}}
]
}

PostgreSQL applies the filter in the same query that selects the fragments, together with the classifications and the current-version condition, before the k limit. Full-text search therefore returns up to k fragments that satisfy the filter. Similarity and MMR search use an approximate vector index, and a search with a selective filter can return fewer than k fragments. A metadata index, created with POST /pipelines/{id}/indexes, is built on the same expression as the filters on its fields, and PostgreSQL decides for each query whether to use the index.

Errors​

StatusBodyCauses
400Schema errorThe filter names a field that the schema does not declare, compares a field of type object, compares a value of another type than the field, compares an integer field with a value outside the 64-bit range, uses $like or $ilike on a field that is not a string, or reaches a field through $teq that is not declared as a string
422Failed to deserialize the JSON body: data did not match any variant of untagged enum ExprOrValueExprThe filter of a pipeline search does not follow the grammar: an unknown operator, a comparison object without an operator, the shorthand form, null as a value, a mixed-type array, an invalid field name or a logical operator combined with field conditions. In agent requests, the message starts with Failed to deserialize the JSON body into the target type: and ends with the line and column of the error.
400Failed to deserialize query string: data did not match any variant of untagged enum ExprOrValueExprThe metadata query parameter of the document list does not follow the grammar
500Internal server errorThe filter reaches a stored value that cannot be converted to the type that the schema declares, such as 5.0 in an integer field, as described in Value types. The API server log names the database error.

Every response in the table carries the JSON error body. Schema error carries the reason in details.error and the field name, where one applies, in details.field, such as {"error": "Unknown or invalid field name", "field": "usrname"}. For an integer comparison value outside the 64-bit range, details.field holds the value instead of the field name, as in {"error": "Value must be a valid integer.", "field": "9223372036854775808"}. In agent requests, a filter error in a similarity or MMR placeholder returns the same status as in a pipeline search. Errors describes the error format.

Query parameter filters​

List operations such as GET /pipelines accept a separate filter form in the query string, field$op=value, which compares the columns of an object rather than metadata. That form has a separate set of operators, $contains, $startswith, $gt, $ge, $lt and $le, described in API conventions. The one exception is the metadata query parameter of the document list, which takes a filter in the language on this page:

curl -G "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/documents" \
--data-urlencode 'metadata={"room_id": {"$eq": "GENERAL"}}' \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET"

Expected result: status 200 and a list of the documents whose metadata matches the filter. The pipeline's schema declares room_id as a string, as in the schema of Metadata, filters and taxonomies; on a pipeline without that field, the request returns HTTP 400 with Schema error.