diff --git a/docs.json b/docs.json index 3124636b2..989f8fba0 100644 --- a/docs.json +++ b/docs.json @@ -785,6 +785,7 @@ "redis/search/document-updates", "redis/search/schema-definition", "redis/search/querying", + "redis/search/hybrid-search", "redis/search/streams", "redis/search/aggregations", "redis/search/counting", @@ -827,6 +828,15 @@ } ] }, + { + "group": "Retrievers", + "pages": [ + "redis/search/retrievers/overview", + "redis/search/retrievers/rrf", + "redis/search/retrievers/ann", + "redis/search/retrievers/standard" + ] + }, { "group": "Aggregation Operators", "pages": [ diff --git a/llms-full.txt b/llms-full.txt index a210426ee..ec0d2bb06 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -55066,6 +55066,8 @@ SEARCH.AGGREGATE '' '' `` is an [Upstash JSON filter](/docs/redis/search/querying). The command accepts an index name or alias. See [Aggregations](/docs/redis/search/aggregations) for the aggregation object and supported operators. +`` can be an `$rrf` retriever when it has no `$ann` retriever. The command then aggregates every document that matches any of the queries. The `window` and `weight` parameters of `$rrf` do not apply. `$ann` is not supported. + ## Response In `redis-cli --json`, the response is an object keyed by aggregation alias: @@ -55658,6 +55660,8 @@ SEARCH.COUNT '' `` is an [Upstash JSON filter](/docs/redis/search/querying). The command accepts an index name or alias. +`` can be an `$rrf` retriever when it has no `$ann` retriever. The command then counts every document that matches any of the queries. The `window` and `weight` parameters of `$rrf` do not apply. `$ann` is not supported. + ## Response Returns the number of matching documents as an integer. Returns `-1` if the index does not exist. @@ -55835,11 +55839,11 @@ Source: https://upstash.com/docs/redis/commands/search/search-create Use `SEARCH.CREATE` to create a search index over JSON, hash, or string values, or over a stream. -`ON` names the type of key to index and `PREFIX` the key prefixes to watch, so the index covers exactly the keys that match both, including keys written after it was created. The `SCHEMA` then declares which fields are searchable and how: `TEXT` fields are analyzed for full-text search, with `NOSTEM` and `NOTOKENIZE` to turn parts of that off, numeric, boolean, and date fields are matched exactly and can be marked `FAST` to make them usable for sorting and scoring, and `KEYWORD` and `FACET` fields are kept whole for exact matching and faceting. `FROM` maps a schema field to a differently named field in the document. +`ON` names the type of key to index and `PREFIX` the key prefixes to watch, so the index covers exactly the keys that match both, including keys written after it was created. The `SCHEMA` then declares which fields are searchable and how: `TEXT` fields are analyzed for full-text search, with `NOSTEM` and `NOTOKENIZE` to turn parts of that off, numeric, boolean, and date fields are matched exactly and can be marked `FAST` to make them usable for sorting and scoring, and `KEYWORD` and `FACET` fields are kept whole for exact matching and faceting. `HYBRID` fields are `TEXT` fields that also store a vector embedding, so [`$ann`](/docs/redis/search/retrievers/ann) can search them by meaning. `FROM` maps a schema field to a differently named field in the document. `ON STREAM` indexes a single stream instead of a set of keys, so it takes the stream key in place of `PREFIX`. Each entry added with [`XADD`](/docs/redis/commands/streams/xadd) becomes a document whose fields are the entry's fields and whose ID is the entry ID, which is what makes a stream searchable by content rather than only by ID range. Entries removed from the stream leave the index as well. -`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields can index multiple values. Store the field as a JSON array; for hash and stream indexes, store the complete JSON-encoded array in the field string. Each valid element is indexed separately, and a document matches when any element matches. `BOOL` and `FACET` fields remain single-valued. +`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields can index multiple values. Store the field as a JSON array; for hash and stream indexes, store the complete JSON-encoded array in the field string. Each valid element is indexed separately, and a document matches when any element matches. `BOOL` and `FACET` fields remain single-valued. Creating an index starts an initial scan of the matching keys, which `SKIPINITIALSCAN` skips when you only want to index data written from now on. `EXISTSOK` makes the command succeed instead of failing when the index already exists. @@ -55860,6 +55864,7 @@ SEARCH.CREATE [EXISTSOK] SCHEMA TEXT [NOSTEM] [NOTOKENIZE] [FROM ] + | HYBRID [NOSTEM] [FROM ] | [FAST] [FROM ] | [FROM ] [...] @@ -55872,7 +55877,7 @@ SEARCH.CREATE | `ON` | Type of Redis value to index: `JSON`, `HASH`, `STRING`, or `STREAM`. A `STRING` value must contain a JSON object. | | `PREFIX` | One or more key prefixes. Prefixes in the same index cannot be duplicates or overlap one another. Not allowed with `ON STREAM`. | | `` | Key of the stream to index. Required with `ON STREAM`, and given in place of `PREFIX`. | -| `LANGUAGE` | Stemming language for `TEXT` fields. Defaults to `english`. Supported values are `arabic`, `danish`, `dutch`, `english`, `finnish`, `french`, `german`, `greek`, `hungarian`, `italian`, `norwegian`, `portuguese`, `romanian`, `russian`, `spanish`, `swedish`, `tamil`, and `turkish`. | +| `LANGUAGE` | Stemming language for `TEXT` and `HYBRID` fields. Defaults to `english`. Supported values are `arabic`, `danish`, `dutch`, `english`, `finnish`, `french`, `german`, `greek`, `hungarian`, `italian`, `norwegian`, `portuguese`, `romanian`, `russian`, `spanish`, `swedish`, `tamil`, and `turkish`. | | `SKIPINITIALSCAN` | Create the index without scanning existing keys. Later writes are still indexed. | | `EXISTSOK` | Return `0` when an index with the same data type, prefixes, and schema already exists. A configuration mismatch returns an error. | | `SCHEMA` | One or more field definitions. `SCHEMA` must be the final top-level clause. | @@ -55882,11 +55887,11 @@ SEARCH.CREATE | Option | Valid field types | Description | | --- | --- | --- | | `FAST` | `U64`, `I64`, `F64`, `BOOL`, `DATE` | Store the field for operations such as sorting and aggregations. Score functions accept `FAST` fields of type `U64`, `I64`, or `F64`. | -| `NOSTEM` | `TEXT` | Index text without stemming words to their roots. | +| `NOSTEM` | `TEXT`, `HYBRID` | Index text without stemming words to their roots. | | `NOTOKENIZE` | `TEXT` | Index the entire value as one token. | | `FROM ` | All field types | Read the value from a different document field or nested dot path while exposing it under `` in the index. | -`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields accept multiple values. A field whose value is a JSON array is indexed element by element, and a document matches when any valid element matches. See [Multi-value fields](/docs/redis/search/schema-definition#multi-value-fields) for data-type-specific input formats and query behavior. +`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields accept multiple values. A field whose value is a JSON array is indexed element by element, and a document matches when any valid element matches. See [Multi-value fields](/docs/redis/search/schema-definition#multi-value-fields) for data-type-specific input formats and query behavior. ## Response @@ -55903,6 +55908,9 @@ SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT price F64 FAST # Index a stream instead of a set of keys SEARCH.CREATE events ON STREAM stream:events SCHEMA message TEXT level KEYWORD + +# Index a field for hybrid search +SEARCH.CREATE articles ON JSON PREFIX 1 article: SCHEMA title TEXT body HYBRID ``` @@ -56172,7 +56180,7 @@ Returns index metadata as alternating field names and values. The response conta | `name` | The index name. | | `type` | The indexed Redis value type: `HASH`, `JSON`, `STRING`, or `STREAM`. | | `prefixes` | Key prefixes included in the index. For a `STREAM` index, the single stream key it covers. | -| `language` | Stemming language used by `TEXT` fields. | +| `language` | Stemming language used by `TEXT` and `HYBRID` fields. | | `indexSize` | Current index size in bytes. | | `pendingUpdates` | Number of updates waiting to be indexed. | | `schema` | Field definitions, including `FROM`, `NOSTEM`, `NOTOKENIZE`, and `FAST` options where present. | @@ -56948,7 +56956,9 @@ The filter is a JSON object naming index fields and the values to match, so `'{" Results come back ordered by relevance score by default. `ORDERBY` sorts by a `FAST` field instead, `LIMIT` and `OFFSET` page through the matches, `SELECT` and `NOCONTENT` cut the payload down to the fields you need, `HIGHLIGHT` wraps the matched terms in tags for display, and `SCOREFUNC` blends numeric fields such as popularity or recency into the relevance score. -For multi-value `TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields, a document matches when any indexed array element satisfies the condition. Phrase queries on `TEXT` stay within one array element. `HIGHLIGHT` preserves the stored array shape in the response, marking matching string elements while leaving other elements unchanged. See [Multi-value queries](/docs/redis/search/querying#multi-value-fields) for examples. +For multi-value `TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields, a document matches when any indexed array element satisfies the condition. Phrase queries on `TEXT` stay within one array element. `HIGHLIGHT` preserves the stored array shape in the response, marking matching string elements while leaving other elements unchanged. See [Multi-value queries](/docs/redis/search/querying#multi-value-fields) for examples. + +The query can also be a retriever. `$ann` searches `HYBRID` fields by meaning, and `$rrf` merges the results of several retrievers into one list. With `$ann`, results are ordered by similarity. With `$rrf`, they are ordered by the merged score. See [Retrievers](/docs/redis/search/retrievers/overview) for details. See [Querying and filtering](/docs/redis/search/querying) for the full filter syntax and worked examples, and [`SEARCH.COUNT`](/docs/redis/commands/search/search-count) when you only need the number of matches. @@ -56985,6 +56995,8 @@ SEARCH.QUERY '' | `SCOREFUNC` | Adjust relevance scores using one or more `FAST` numeric fields. `FACTOR` defaults to `1`, `MISSING` to `0`, `MODIFIER` to `NONE`, `SCOREMODE` to `SUM`, and `COMBINEMODE` to `SUM`. | Disabled | `NOCONTENT` cannot be combined with `SELECT` or `HIGHLIGHT`. `SCOREFUNC` cannot be combined with `ORDERBY`. Inside `MULTI` or `EVAL`, the command requires `NOCONTENT`. + + `ORDERBY` cannot be used with `$rrf` or `$ann`. `SCOREFUNC` and `HIGHLIGHT` cannot be used with `$ann` at the root of the query. With `$rrf`, `LIMIT` cannot be larger than the `$rrf` `window`, and `OFFSET` requires an explicit `window`. See [Query Options](/docs/redis/search/retrievers/overview#query-options) and [`$rrf` paging](/docs/redis/search/retrievers/rrf#paging). See [Querying and filtering](/docs/redis/search/querying) for the JSON filter operators and detailed query examples. @@ -56994,7 +57006,7 @@ See [Querying and filtering](/docs/redis/search/querying) for the JSON filter op Returns an array of `[key, score, content]` results, or `null` if the index does not exist: * `key` is the Redis key of the matching document. -* `score` is the floating-point relevance score. +* `score` is the floating-point relevance score. With `$ann`, it is the similarity score. With `$rrf`, it is the merged score. * `content` is an array of field-value pairs. JSON indexes return `[["$", ""]]`; hash indexes return `[["field", "value"], ...]`. When `NOCONTENT` is used, each result is `[key, score]`. When `SELECT` is used, only fields that exist in the document appear in the content. For an index created with `ON STREAM`, `key` is the entry ID of the matching stream entry. @@ -57016,6 +57028,9 @@ A highlighted multi-valued field keeps its array form in the reply. Highlighting ```bash SEARCH.QUERY products '{"name": "wireless"}' LIMIT 10 OFFSET 0 + +# Merge a full-text query and a semantic search on a HYBRID field +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "wireless"}, {"$ann": {"field": "description", "query": "wireless"}}]}}' LIMIT 10 ``` @@ -79242,7 +79257,7 @@ Source: https://upstash.com/docs/redis/commands/vector/overview A vector index stores embeddings under IDs and answers nearest-neighbour queries over them. It lives at an ordinary Redis key, so [`EXISTS`](/docs/redis/commands/generic/exists), [`EXPIRE`](/docs/redis/commands/generic/expire), and [`DEL`](/docs/redis/commands/generic/del) apply to it, and a key already holding another type cannot be turned into one. -[`VECTOR.CREATE`](/docs/redis/commands/vector/vector-create) fixes two properties for the life of the index: `DIM`, the number of elements every vector must have, and `METRIC`, how similarity is measured. Use `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when magnitude carries meaning, and `EUCLIDEAN` when straight-line distance does. Neither can be changed afterwards; to change one, drop the index and create it again. +[`VECTOR.CREATE`](/docs/redis/commands/vector/vector-create) fixes two properties for the life of the index: `DIM`, the number of elements every vector must have, and `METRIC`, how similarity is measured. Use `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when your embeddings are already normalized to unit vectors, and `EUCLIDEAN` when straight-line distance does. Neither can be changed afterwards; to change one, drop the index and create it again. [`VECTOR.ADD`](/docs/redis/commands/vector/vector-add) upserts a vector under an ID and [`VECTOR.QUERY`](/docs/redis/commands/vector/vector-query) returns the nearest IDs with a score. Scores are normalized to `0` through `1` for every metric, so a higher score always means a closer match. Search is approximate: `PROFILE` moves a query along the recall and latency trade-off without changing anything stored. @@ -79335,6 +79350,7 @@ VECTOR.ADD ## Important points * The three vector forms are mutually exclusive and exactly one must be given. `VALUES` is the readable form; `FP32` avoids decimal formatting on a binary-safe connection; `BASE64-FP32` carries the same bytes through JSON, which is what the [REST API](/docs/redis/features/restapi) needs. +* On a `DOT` index, the vector must be a unit vector. Other vectors are rejected. * A vector whose length differs from the index's `DIM` returns `ERR vector dimension mismatch: expected , got `. * An index that does not exist returns `ERR Index is not found`. Create it with [`VECTOR.CREATE`](/docs/redis/commands/vector/vector-create) first. @@ -79690,7 +79706,7 @@ Source: https://upstash.com/docs/redis/commands/vector/vector-create Use `VECTOR.CREATE` to create a vector index at a key. -`DIM` fixes the number of elements every vector in the index must have, and `METRIC` fixes how similarity between two vectors is measured. Both are permanent: they decide how the index is built, so they cannot be changed afterwards, and a vector whose length does not match `DIM` is rejected at write time. Pick `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when magnitude carries meaning, and `EUCLIDEAN` when straight-line distance in the embedding space does. +`DIM` fixes the number of elements every vector in the index must have, and `METRIC` fixes how similarity between two vectors is measured. Both are permanent: they decide how the index is built, so they cannot be changed afterwards, and a vector whose length does not match `DIM` is rejected at write time. Pick `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when your embeddings are already normalized to unit vectors, and `EUCLIDEAN` when straight-line distance in the embedding space does. The index occupies its key like any other Redis value, so [`EXISTS`](/docs/redis/commands/generic/exists), [`EXPIRE`](/docs/redis/commands/generic/expire), and [`DEL`](/docs/redis/commands/generic/del) work on it, and creating one at a key already used by another type fails. `EXISTSOK` makes the command succeed instead of failing when the index is already there, which keeps a start-up path idempotent; a mismatch in dimension or metric is still an error. @@ -79714,6 +79730,7 @@ VECTOR.CREATE DIM METRIC [EXISTSO ## Important points * `DIM` and `METRIC` are both required and cannot be changed after the index is created. Use [`VECTOR.DROP`](/docs/redis/commands/vector/vector-drop) and create it again to change either. +* A `DOT` index accepts only unit vectors, which are vectors with a magnitude of `1`. [`VECTOR.ADD`](/docs/redis/commands/vector/vector-add) and [`VECTOR.QUERY`](/docs/redis/commands/vector/vector-query) reject other vectors. Normalize your vectors first, or use `COSINE`. * Without `EXISTSOK`, creating an index that already exists returns `ERR Index already exists`. * With `EXISTSOK`, an existing index with a different dimension or metric returns `ERR Index already exists with a different configuration`. @@ -80658,6 +80675,7 @@ VECTOR.QUERY TOPK * `TOPK` and the query vector are both required; the clauses may be given in any order. * The three vector forms are mutually exclusive and exactly one must be given. `VALUES` is the readable form; `FP32` avoids decimal formatting on a binary-safe connection; `BASE64-FP32` carries the same bytes through JSON, which is what the [REST API](/docs/redis/features/restapi) needs. * Results are ordered by score, highest first. Scores are normalized to `0` through `1` for every metric. +* On a `DOT` index, the query vector must be a unit vector. Other vectors are rejected. * A query vector whose length differs from the index's `DIM` returns `ERR vector dimension mismatch: expected , got `. ## Response @@ -85116,6 +85134,9 @@ Source: https://upstash.com/docs/redis/overall/changelog * Added the [Insights](/docs/redis/howto/insights) tab to the console for databases with Prod Pack: request routing between regions, client IP ranges and SDKs, latency and throughput charts, usage against plan limits, Lua script statistics and a keyspace analysis. +* Added [hybrid search](/docs/redis/search/hybrid-search) to Redis Search. `HYBRID` fields store a vector embedding, so you can search them by meaning. +* Added [retrievers](/docs/redis/search/retrievers/overview) to Redis Search queries. [`$rrf`](/docs/redis/search/retrievers/rrf) merges the results of several retrievers into one list, [`$ann`](/docs/redis/search/retrievers/ann) searches `HYBRID` fields by meaning, and [`$standard`](/docs/redis/search/retrievers/standard) sets the weight of a query inside `$rrf`. +* A [`$mustNot`](/docs/redis/search/query-operators/boolean-operators/must-not) query with no other conditions now matches every document it does not exclude, with a score of `0`. Inside `$should`, `$mustNot` now matches the documents it does not exclude. Before, `$mustNot` matched no documents in both cases. @@ -94188,7 +94209,7 @@ SEARCH.AGGREGATE products '{}' '{"avg_price": {"$avg": {"field": "price"}}, "by_ Use `filter` to restrict which documents participate in the aggregation. -Filtering uses the same query syntax as [queries](./querying). +Filtering uses the same query syntax as [queries](./querying). `$ann` is not supported. @@ -95052,6 +95073,116 @@ SEARCH.COUNT products '{"price": {"$lt": 150}}' +# Hybrid Search +Source: https://upstash.com/docs/redis/search/hybrid-search + +Full-text search finds documents that contain the words of your query. Semantic search finds documents +with a similar meaning, even when they use different words. Hybrid search runs both and merges the +results into one list. + +Hybrid search has two parts: + +* `HYBRID` fields are indexed for both full-text and semantic search. +* [Retrievers](/docs/redis/search/retrievers/overview) run the searches. [`$ann`](/docs/redis/search/retrievers/ann) + runs a semantic search, and [`$rrf`](/docs/redis/search/retrievers/rrf) merges the results of several + retrievers into one list. + +## Hybrid fields + +A `HYBRID` field is a `TEXT` field that also stores a vector embedding. Full-text queries work on it +the same way as on a `TEXT` field, and `$ann` searches its embeddings by meaning. See +[Hybrid Fields](/docs/redis/search/schema-definition#hybrid-fields) for the field options. + +* An index can have more than one `HYBRID` field. +* If the text of a document cannot be embedded, the document is still searchable by its text. +* `HYBRID` fields work with JSON, hash, string, and stream indexes. + +## Create an index + +Declare the fields you want to search by meaning as `HYBRID`: + + + + +```bash +SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT description HYBRID category KEYWORD price F64 FAST +``` + + + + +## Add documents + +Write documents as usual. Search indexes them and creates their embeddings: + + + + +```bash +JSON.SET product:1 $ '{"name": "Studio Headphones", "description": "Over-ear headphones with active noise cancelling", "category": "audio", "price": 199}' +JSON.SET product:2 $ '{"name": "Travel Earplugs", "description": "Soft foam earplugs that keep airplane cabin sound out", "category": "travel", "price": 9}' +JSON.SET product:3 $ '{"name": "Sport Earbuds", "description": "Sweat-proof wireless earbuds for running", "category": "sports", "price": 79}' + +# Wait until the documents are indexed +SEARCH.WAITINDEXING products +``` + + + + +## Search by meaning + +Use [`$ann`](/docs/redis/search/retrievers/ann) to find the documents that are closest in meaning to a +text. This query can find the earplugs, even though their description does not contain the word +"noise": + + + + +```bash +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "block noise on a flight"}}' + +# Search by meaning within one category +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "headphones for running", "filter": {"category": "sports"}}}' +``` + + + + +## Combine full-text and semantic search + +Use [`$rrf`](/docs/redis/search/retrievers/rrf) to run a full-text query and a semantic search, and merge +their results into one list. Documents that rank high in both come first: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "noise cancelling"}, {"$ann": {"field": "description", "query": "noise cancelling"}}]}}' +``` + + + + +To give one side more influence, set its weight. A full-text query takes a weight when you wrap it in +[`$standard`](/docs/redis/search/retrievers/standard): + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"$standard": {"query": {"description": "noise cancelling"}, "weight": 2}}, {"$ann": {"field": "description", "query": "noise cancelling"}}]}}' +``` + + + + +## Next steps + +* See [Retrievers](/docs/redis/search/retrievers/overview) for all retriever parameters and rules. +* See [Schema Definition](/docs/redis/search/schema-definition#hybrid-fields) for field types and options. +* See [`SEARCH.QUERY`](/docs/redis/commands/search/search-query) for all query options. + # Indices Source: https://upstash.com/docs/redis/search/index-management @@ -95648,6 +95779,7 @@ Upstash Redis Search is our first extension beyond the official Redis spec. Usin * **Auto-Synchronization**: Once you created an index, all write operations are automatically tracked * **Intuitive Query Language**: A type-safe JSON-based query syntax with boolean operators, fuzzy matching, phrase queries, regex support, and more. +* **Hybrid Search**: Search `HYBRID` fields by meaning, and merge full-text and semantic search results into one list * **Production-Ready Performance**: Built on Tantivy, a fast full-text search engine library written in Rust ## Next Steps @@ -95674,6 +95806,13 @@ Upstash Redis Search is our first extension beyond the official Redis spec. Usin > Learn the JSON-based query language with filters and operators + + Combine full-text and semantic search in one query + + + +```bash +# Search by meaning +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "something to block noise on a flight"}}' + +# Search by meaning within cheap sports products +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "headphones for running", "filter": {"category": "sports", "price": {"$lt": 100}}}}' + +# Use the most accurate profile, and return keys and scores only +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "headphones for running", "profile": "precise"}}' NOCONTENT +``` + + + + +See [Query Options](./overview#query-options) for the `SEARCH.QUERY` options that `$ann` does not +support. + +# Overview +Source: https://upstash.com/docs/redis/search/retrievers/overview + +A retriever runs one or more queries and returns the top documents. Queries describe which documents +match. Retrievers decide how those documents are found and ranked. + +### Available Retrievers + +| Retriever | Description | +|---|---| +| [`$rrf`](./rrf) | Merges the results of 2 to 10 retrievers with Reciprocal Rank Fusion. | +| [`$ann`](./ann) | Finds the documents whose `HYBRID` field is closest in meaning to a text. | +| [`$standard`](./standard) | Runs one query. Use it inside `$rrf` to set the weight of a query. | + + + + +```bash +# Search a HYBRID field by meaning +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "something to block noise on a flight"}}' + +# Merge a full-text query and a semantic search +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "noise cancelling headphones"}, {"$ann": {"field": "description", "query": "noise cancelling headphones"}}]}}' +``` + + + + +### Where to Use Retrievers + +A retriever must be the only key of its object. You can use it in two places: + +* At the root of the query in [`SEARCH.QUERY`](/docs/redis/commands/search/search-query), + [`SEARCH.COUNT`](/docs/redis/commands/search/search-count), and + [`SEARCH.AGGREGATE`](/docs/redis/commands/search/search-aggregate). +* As an entry of `$rrf` `retrievers`. + +A retriever cannot be used inside a query, such as inside `$must`, `$should`, `$mustNot`, or an `$ann` +filter. `$rrf` cannot be used inside another `$rrf`. + +### Query Options + +Some `SEARCH.QUERY` options do not work with retrievers: + +| Option | With `$rrf` | With `$ann` at the root | +|---|---|---| +| `ORDERBY` | Not supported | Not supported | +| `SCOREFUNC` | Supported. It changes the scores of the full-text queries before their results are merged. It does not change `$ann` results. | Not supported | +| `HIGHLIGHT` | Supported | Not supported | + +`$standard` at the root supports all options, the same as a plain query. + +### Counting and Aggregations + +`SEARCH.COUNT` and `SEARCH.AGGREGATE` accept `$rrf` when it has no `$ann` retriever. +They use every document that matches any of the queries. `window` and `weight` do not apply, so the +count can be larger than the number of results `SEARCH.QUERY` returns. + +`SEARCH.COUNT` and `SEARCH.AGGREGATE` do not support `$ann`. + +# $rrf +Source: https://upstash.com/docs/redis/search/retrievers/rrf + +The `$rrf` retriever runs several retrievers and merges their results with Reciprocal Rank Fusion +(RRF). Each retriever ranks its own results. A document gets points from every retriever that returns +it, so documents that rank high in several retrievers come first. + +Use `$rrf` for hybrid search, to merge a full-text query with an [`$ann`](./ann) semantic search. It +also merges full-text queries only, or `$ann` retrievers only. + +```json +{ + "$rrf": { + "retrievers": [ + { "description": "noise cancelling headphones" }, + { "$ann": { "field": "description", "query": "noise cancelling headphones" } } + ] + } +} +``` + +### Parameters + +| Parameter | Description | Default | +|---|---|---| +| `retrievers` | Required. 2 to 10 retrievers to merge. Each entry is a query object, a [`$standard`](./standard), or an [`$ann`](./ann). | | +| `rankConstant` | An integer between `1` and `2147483647`. Higher values give lower-ranked results more influence. | `60` | +| `window` | An integer between `1` and `11000`. The number of results each retriever contributes, and the maximum number of merged results. See [Paging](#paging). | `LIMIT` | + +### Scoring + +Each retriever that returns a document adds `weight / (rankConstant + rank)` to its score. `rank` is the +position of the document in that retriever's results, starting at `1`. + +These scores are small numbers. Compare them only with other results of the same query. + +### Weights + +A weight scales how much a retriever adds to the score. All retrievers have a weight of `1` by default. + +* To set the weight of a query object, wrap it in [`$standard`](./standard). +* An `$ann` retriever takes `weight` directly. +* A weight must be `0` or greater. With a weight of `0`, the documents of that retriever stay in the + results, but the retriever adds nothing to their score. + +### Paging + +`window` sets how deep `$rrf` looks: + +* Each retriever contributes up to `window` results, and the merged list is cut at `window`. +* When `window` is not set, it is equal to `LIMIT`, which is `10` by default. +* `LIMIT` cannot be larger than `window`. +* To use `OFFSET`, set `window`. A page that starts at or after `window` is empty. + +Keep the same `window` on every page so that pages do not overlap. + +### Examples + +#### Hybrid Search + +Merge a full-text query and a semantic search on the same field: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "noise cancelling headphones"}, {"$ann": {"field": "description", "query": "noise cancelling headphones"}}]}}' +``` + + + + +#### Weighted Hybrid Search + +Give the full-text query twice the weight of the semantic search: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"$standard": {"query": {"description": "noise cancelling headphones"}, "weight": 2}}, {"$ann": {"field": "description", "query": "noise cancelling headphones", "weight": 1}}]}}' +``` + + + + +#### Full-Text Queries Only + +`$rrf` does not need `HYBRID` fields: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"name": "wireless headphones"}, {"description": "bluetooth"}]}}' +``` + + + + +#### Paging Through Results + +Set `window` and keep it the same on every page: + + + + +```bash +# Page 1 +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "headphones"}, {"$ann": {"field": "description", "query": "headphones"}}], "window": 100}}' LIMIT 10 + +# Page 2 +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "headphones"}, {"$ann": {"field": "description", "query": "headphones"}}], "window": 100}}' LIMIT 10 OFFSET 10 +``` + + + + +See [Query Options](./overview#query-options) for the `SEARCH.QUERY` options that work with `$rrf`, and +[Counting and Aggregations](./overview#counting-and-aggregations) for `SEARCH.COUNT` and +`SEARCH.AGGREGATE`. + +# $standard +Source: https://upstash.com/docs/redis/search/retrievers/standard + +The `$standard` retriever runs one query. Use it inside [`$rrf`](./rrf) to set the weight of a query. + +```json +{ "$standard": { "query": { "name": "headphones" }, "weight": 2 } } +``` + +### Parameters + +| Parameter | Description | Default | +|---|---|---| +| `query` | Required. A query object. See [Queries](/docs/redis/search/querying). | | +| `weight` | A number, `0` or greater. It scales how much the query adds to the `$rrf` score. Valid only inside `$rrf`. | `1` | + +### Plain Query Objects + +A plain query object works the same as `$standard` without a weight. Use `$standard` only when you need +to set a weight. + +These two `$rrf` retrievers give the same results: + +```json +{ "$rrf": { "retrievers": [{ "name": "headphones" }, { "description": "wireless" }] } } + +{ "$rrf": { "retrievers": [{ "$standard": { "query": { "name": "headphones" } } }, { "$standard": { "query": { "description": "wireless" } } }] } } +``` + +### Examples + + + + +```bash +# Give the name match twice the weight of the description match +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"$standard": {"query": {"name": "headphones"}, "weight": 2}}, {"description": "headphones"}]}}' + +# Same as SEARCH.QUERY products '{"name": "headphones"}' +SEARCH.QUERY products '{"$standard": {"query": {"name": "headphones"}}}' +``` + + + + # Schemas Source: https://upstash.com/docs/redis/search/schema-definition @@ -97801,6 +98251,7 @@ import { Redis, s } from "@upstash/redis" | `s.date()` | `DATE` | Date/time field | Yes | Yes | Yes | No | | `s.boolean()` | `BOOL` | Boolean field | Yes | No | No | No | | `s.facet()` | `FACET` | Hierarchical path-based field | No | No | No | No (only `$eq`, `$in`) | +| - | `HYBRID` | Full-text field that also supports semantic search | No | Yes | No | `$smart`, `$phrase`, `$fuzzy`, `$regex` | In the TypeScript SDK, `s.number()` defaults to `F64`. You can specify `s.number("U64")` or @@ -97873,8 +98324,9 @@ const schema = s.object({ ## Multi-value fields A document field can contain several values without a separate array type in the schema. Declare the -field as `TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, or `DATE`, then store its value as a JSON array. -Search indexes every valid array element under the same field. +field as `TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, or `DATE`, then store its value as a JSON array. +Search indexes every valid array element under the same field. For a `HYBRID` field, `$ann` uses one +embedding of all elements together. For example, this schema accepts several tags, descriptions, prices, and restock dates per product: @@ -98047,6 +98499,29 @@ SEARCH.CREATE idx ON JSON PREFIX 1 prefix: SCHEMA tag KEYWORD *** +## Hybrid Fields + +The `HYBRID` field type is a `TEXT` field that also supports semantic search. Search creates a +vector embedding of the field value, and the [`$ann`](/docs/redis/search/retrievers/ann) +retriever searches these embeddings by meaning. Full-text queries work on the field the same way as +on a `TEXT` field. + +`HYBRID` fields accept the `NOSTEM` and `FROM` options. + + + + +```bash +SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT description HYBRID +``` + + + + +See [Hybrid Search](/docs/redis/search/hybrid-search) for how to query `HYBRID` fields. + +*** + ## Facet Fields The `FACET` field type is for hierarchical path-based faceted search. Values must be `/`-delimited paths starting with `/`. @@ -98523,12 +98998,12 @@ Redis stores stream field values as strings. Search converts each value accordin * `U64`, `I64`, and `F64` fields require values in the corresponding numeric format. * `BOOL` fields accept `true` or `false`. * `DATE` fields require an RFC 3339 timestamp, such as `2026-08-24T09:30:00Z`. -* `TEXT`, `KEYWORD`, and `FACET` fields use string values. +* `TEXT`, `HYBRID`, `KEYWORD`, and `FACET` fields use string values. Missing fields, empty values, and values that cannot be converted are omitted from the indexed document. If an entry has no valid schema fields, it is not added to the index. -`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields also accept JSON-encoded arrays. Each valid +`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields also accept JSON-encoded arrays. Each valid element is indexed as a separate value of the same field. For example, the `service` field can contain more than one service: diff --git a/llms.txt b/llms.txt index 1e23ba1b7..b6c159e7a 100644 --- a/llms.txt +++ b/llms.txt @@ -1198,6 +1198,7 @@ Search these docs: GET https://upstash.com/docs/search?q= returns JSON { - [Counting](https://upstash.com/docs/redis/search/counting.md) - [Document Updates](https://upstash.com/docs/redis/search/document-updates.md) - [Quickstart](https://upstash.com/docs/redis/search/getting-started.md) +- [Hybrid Search](https://upstash.com/docs/redis/search/hybrid-search.md): Combine full-text and semantic search with HYBRID fields and retrievers. - [Indices](https://upstash.com/docs/redis/search/index-management.md) - [Introduction](https://upstash.com/docs/redis/search/introduction.md) - [$boost](https://upstash.com/docs/redis/search/query-operators/boolean-operators/boost.md) @@ -1219,6 +1220,10 @@ Search these docs: GET https://upstash.com/docs/search?q= returns JSON { - [E-commerce Search](https://upstash.com/docs/redis/search/recipes/e-commerce-search.md) - [Overview](https://upstash.com/docs/redis/search/recipes/overview.md) - [User Directory](https://upstash.com/docs/redis/search/recipes/user-directory.md) +- [$ann](https://upstash.com/docs/redis/search/retrievers/ann.md): Find the documents whose HYBRID field is closest in meaning to a text. +- [Overview](https://upstash.com/docs/redis/search/retrievers/overview.md): Retrievers run one or more queries and return the top documents. +- [$rrf](https://upstash.com/docs/redis/search/retrievers/rrf.md): Merge the results of several retrievers with Reciprocal Rank Fusion. +- [$standard](https://upstash.com/docs/redis/search/retrievers/standard.md): Run one query, and set its weight inside $rrf. - [Schemas](https://upstash.com/docs/redis/search/schema-definition.md) - [Searching Streams](https://upstash.com/docs/redis/search/streams.md): Index and search individual Redis stream entries. - [Troubleshooting](https://upstash.com/docs/redis/search/troubleshooting.md) diff --git a/redis/commands/search/search-aggregate.mdx b/redis/commands/search/search-aggregate.mdx index c6b50304b..af5569ba1 100644 --- a/redis/commands/search/search-aggregate.mdx +++ b/redis/commands/search/search-aggregate.mdx @@ -23,6 +23,8 @@ SEARCH.AGGREGATE '' '' `` is an [Upstash JSON filter](/redis/search/querying). The command accepts an index name or alias. See [Aggregations](/redis/search/aggregations) for the aggregation object and supported operators. +`` can be an `$rrf` retriever when it has no `$ann` retriever. The command then aggregates every document that matches any of the queries. The `window` and `weight` parameters of `$rrf` do not apply. `$ann` is not supported. + ## Response In `redis-cli --json`, the response is an object keyed by aggregation alias: diff --git a/redis/commands/search/search-count.mdx b/redis/commands/search/search-count.mdx index 2a5a3a328..3f3cf1c1e 100644 --- a/redis/commands/search/search-count.mdx +++ b/redis/commands/search/search-count.mdx @@ -17,6 +17,8 @@ SEARCH.COUNT '' `` is an [Upstash JSON filter](/redis/search/querying). The command accepts an index name or alias. +`` can be an `$rrf` retriever when it has no `$ann` retriever. The command then counts every document that matches any of the queries. The `window` and `weight` parameters of `$rrf` do not apply. `$ann` is not supported. + ## Response Returns the number of matching documents as an integer. Returns `-1` if the index does not exist. diff --git a/redis/commands/search/search-create.mdx b/redis/commands/search/search-create.mdx index 132314efe..a7b8a18ec 100644 --- a/redis/commands/search/search-create.mdx +++ b/redis/commands/search/search-create.mdx @@ -5,11 +5,11 @@ description: "Create a search index." Use `SEARCH.CREATE` to create a search index over JSON, hash, or string values, or over a stream. -`ON` names the type of key to index and `PREFIX` the key prefixes to watch, so the index covers exactly the keys that match both, including keys written after it was created. The `SCHEMA` then declares which fields are searchable and how: `TEXT` fields are analyzed for full-text search, with `NOSTEM` and `NOTOKENIZE` to turn parts of that off, numeric, boolean, and date fields are matched exactly and can be marked `FAST` to make them usable for sorting and scoring, and `KEYWORD` and `FACET` fields are kept whole for exact matching and faceting. `FROM` maps a schema field to a differently named field in the document. +`ON` names the type of key to index and `PREFIX` the key prefixes to watch, so the index covers exactly the keys that match both, including keys written after it was created. The `SCHEMA` then declares which fields are searchable and how: `TEXT` fields are analyzed for full-text search, with `NOSTEM` and `NOTOKENIZE` to turn parts of that off, numeric, boolean, and date fields are matched exactly and can be marked `FAST` to make them usable for sorting and scoring, and `KEYWORD` and `FACET` fields are kept whole for exact matching and faceting. `HYBRID` fields are `TEXT` fields that also store a vector embedding, so [`$ann`](/redis/search/retrievers/ann) can search them by meaning. `FROM` maps a schema field to a differently named field in the document. `ON STREAM` indexes a single stream instead of a set of keys, so it takes the stream key in place of `PREFIX`. Each entry added with [`XADD`](/redis/commands/streams/xadd) becomes a document whose fields are the entry's fields and whose ID is the entry ID, which is what makes a stream searchable by content rather than only by ID range. Entries removed from the stream leave the index as well. -`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields can index multiple values. Store the field as a JSON array; for hash and stream indexes, store the complete JSON-encoded array in the field string. Each valid element is indexed separately, and a document matches when any element matches. `BOOL` and `FACET` fields remain single-valued. +`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields can index multiple values. Store the field as a JSON array; for hash and stream indexes, store the complete JSON-encoded array in the field string. Each valid element is indexed separately, and a document matches when any element matches. `BOOL` and `FACET` fields remain single-valued. Creating an index starts an initial scan of the matching keys, which `SKIPINITIALSCAN` skips when you only want to index data written from now on. `EXISTSOK` makes the command succeed instead of failing when the index already exists. @@ -30,6 +30,7 @@ SEARCH.CREATE [EXISTSOK] SCHEMA TEXT [NOSTEM] [NOTOKENIZE] [FROM ] + | HYBRID [NOSTEM] [FROM ] | [FAST] [FROM ] | [FROM ] [...] @@ -42,7 +43,7 @@ SEARCH.CREATE | `ON` | Type of Redis value to index: `JSON`, `HASH`, `STRING`, or `STREAM`. A `STRING` value must contain a JSON object. | | `PREFIX` | One or more key prefixes. Prefixes in the same index cannot be duplicates or overlap one another. Not allowed with `ON STREAM`. | | `` | Key of the stream to index. Required with `ON STREAM`, and given in place of `PREFIX`. | -| `LANGUAGE` | Stemming language for `TEXT` fields. Defaults to `english`. Supported values are `arabic`, `danish`, `dutch`, `english`, `finnish`, `french`, `german`, `greek`, `hungarian`, `italian`, `norwegian`, `portuguese`, `romanian`, `russian`, `spanish`, `swedish`, `tamil`, and `turkish`. | +| `LANGUAGE` | Stemming language for `TEXT` and `HYBRID` fields. Defaults to `english`. Supported values are `arabic`, `danish`, `dutch`, `english`, `finnish`, `french`, `german`, `greek`, `hungarian`, `italian`, `norwegian`, `portuguese`, `romanian`, `russian`, `spanish`, `swedish`, `tamil`, and `turkish`. | | `SKIPINITIALSCAN` | Create the index without scanning existing keys. Later writes are still indexed. | | `EXISTSOK` | Return `0` when an index with the same data type, prefixes, and schema already exists. A configuration mismatch returns an error. | | `SCHEMA` | One or more field definitions. `SCHEMA` must be the final top-level clause. | @@ -52,11 +53,11 @@ SEARCH.CREATE | Option | Valid field types | Description | | --- | --- | --- | | `FAST` | `U64`, `I64`, `F64`, `BOOL`, `DATE` | Store the field for operations such as sorting and aggregations. Score functions accept `FAST` fields of type `U64`, `I64`, or `F64`. | -| `NOSTEM` | `TEXT` | Index text without stemming words to their roots. | +| `NOSTEM` | `TEXT`, `HYBRID` | Index text without stemming words to their roots. | | `NOTOKENIZE` | `TEXT` | Index the entire value as one token. | | `FROM ` | All field types | Read the value from a different document field or nested dot path while exposing it under `` in the index. | -`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields accept multiple values. A field whose value is a JSON array is indexed element by element, and a document matches when any valid element matches. See [Multi-value fields](/redis/search/schema-definition#multi-value-fields) for data-type-specific input formats and query behavior. +`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields accept multiple values. A field whose value is a JSON array is indexed element by element, and a document matches when any valid element matches. See [Multi-value fields](/redis/search/schema-definition#multi-value-fields) for data-type-specific input formats and query behavior. ## Response @@ -73,6 +74,9 @@ SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT price F64 FAST # Index a stream instead of a set of keys SEARCH.CREATE events ON STREAM stream:events SCHEMA message TEXT level KEYWORD + +# Index a field for hybrid search +SEARCH.CREATE articles ON JSON PREFIX 1 article: SCHEMA title TEXT body HYBRID ``` diff --git a/redis/commands/search/search-describe.mdx b/redis/commands/search/search-describe.mdx index 60cc1279a..7e0a19cbd 100644 --- a/redis/commands/search/search-describe.mdx +++ b/redis/commands/search/search-describe.mdx @@ -26,7 +26,7 @@ Returns index metadata as alternating field names and values. The response conta | `name` | The index name. | | `type` | The indexed Redis value type: `HASH`, `JSON`, `STRING`, or `STREAM`. | | `prefixes` | Key prefixes included in the index. For a `STREAM` index, the single stream key it covers. | -| `language` | Stemming language used by `TEXT` fields. | +| `language` | Stemming language used by `TEXT` and `HYBRID` fields. | | `indexSize` | Current index size in bytes. | | `pendingUpdates` | Number of updates waiting to be indexed. | | `schema` | Field definitions, including `FROM`, `NOSTEM`, `NOTOKENIZE`, and `FAST` options where present. | diff --git a/redis/commands/search/search-query.mdx b/redis/commands/search/search-query.mdx index 62fab49c7..284b37da0 100644 --- a/redis/commands/search/search-query.mdx +++ b/redis/commands/search/search-query.mdx @@ -9,7 +9,9 @@ The filter is a JSON object naming index fields and the values to match, so `'{" Results come back ordered by relevance score by default. `ORDERBY` sorts by a `FAST` field instead, `LIMIT` and `OFFSET` page through the matches, `SELECT` and `NOCONTENT` cut the payload down to the fields you need, `HIGHLIGHT` wraps the matched terms in tags for display, and `SCOREFUNC` blends numeric fields such as popularity or recency into the relevance score. -For multi-value `TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields, a document matches when any indexed array element satisfies the condition. Phrase queries on `TEXT` stay within one array element. `HIGHLIGHT` preserves the stored array shape in the response, marking matching string elements while leaving other elements unchanged. See [Multi-value queries](/redis/search/querying#multi-value-fields) for examples. +For multi-value `TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields, a document matches when any indexed array element satisfies the condition. Phrase queries on `TEXT` stay within one array element. `HIGHLIGHT` preserves the stored array shape in the response, marking matching string elements while leaving other elements unchanged. See [Multi-value queries](/redis/search/querying#multi-value-fields) for examples. + +The query can also be a retriever. `$ann` searches `HYBRID` fields by meaning, and `$rrf` merges the results of several retrievers into one list. With `$ann`, results are ordered by similarity. With `$rrf`, they are ordered by the merged score. See [Retrievers](/redis/search/retrievers/overview) for details. See [Querying and filtering](/redis/search/querying) for the full filter syntax and worked examples, and [`SEARCH.COUNT`](/redis/commands/search/search-count) when you only need the number of matches. @@ -46,6 +48,8 @@ SEARCH.QUERY '' | `SCOREFUNC` | Adjust relevance scores using one or more `FAST` numeric fields. `FACTOR` defaults to `1`, `MISSING` to `0`, `MODIFIER` to `NONE`, `SCOREMODE` to `SUM`, and `COMBINEMODE` to `SUM`. | Disabled | `NOCONTENT` cannot be combined with `SELECT` or `HIGHLIGHT`. `SCOREFUNC` cannot be combined with `ORDERBY`. Inside `MULTI` or `EVAL`, the command requires `NOCONTENT`. + + `ORDERBY` cannot be used with `$rrf` or `$ann`. `SCOREFUNC` and `HIGHLIGHT` cannot be used with `$ann` at the root of the query. With `$rrf`, `LIMIT` cannot be larger than the `$rrf` `window`, and `OFFSET` requires an explicit `window`. See [Query Options](/redis/search/retrievers/overview#query-options) and [`$rrf` paging](/redis/search/retrievers/rrf#paging). See [Querying and filtering](/redis/search/querying) for the JSON filter operators and detailed query examples. @@ -55,7 +59,7 @@ See [Querying and filtering](/redis/search/querying) for the JSON filter operato Returns an array of `[key, score, content]` results, or `null` if the index does not exist: - `key` is the Redis key of the matching document. -- `score` is the floating-point relevance score. +- `score` is the floating-point relevance score. With `$ann`, it is the similarity score. With `$rrf`, it is the merged score. - `content` is an array of field-value pairs. JSON indexes return `[["$", ""]]`; hash indexes return `[["field", "value"], ...]`. When `NOCONTENT` is used, each result is `[key, score]`. When `SELECT` is used, only fields that exist in the document appear in the content. For an index created with `ON STREAM`, `key` is the entry ID of the matching stream entry. @@ -77,6 +81,9 @@ A highlighted multi-valued field keeps its array form in the reply. Highlighting ```bash SEARCH.QUERY products '{"name": "wireless"}' LIMIT 10 OFFSET 0 + +# Merge a full-text query and a semantic search on a HYBRID field +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "wireless"}, {"$ann": {"field": "description", "query": "wireless"}}]}}' LIMIT 10 ``` diff --git a/redis/commands/vector/overview.mdx b/redis/commands/vector/overview.mdx index a891bd09f..46d57fa15 100644 --- a/redis/commands/vector/overview.mdx +++ b/redis/commands/vector/overview.mdx @@ -10,7 +10,7 @@ description: "Commands for storing embeddings and running nearest-neighbour sear A vector index stores embeddings under IDs and answers nearest-neighbour queries over them. It lives at an ordinary Redis key, so [`EXISTS`](/redis/commands/generic/exists), [`EXPIRE`](/redis/commands/generic/expire), and [`DEL`](/redis/commands/generic/del) apply to it, and a key already holding another type cannot be turned into one. -[`VECTOR.CREATE`](/redis/commands/vector/vector-create) fixes two properties for the life of the index: `DIM`, the number of elements every vector must have, and `METRIC`, how similarity is measured. Use `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when magnitude carries meaning, and `EUCLIDEAN` when straight-line distance does. Neither can be changed afterwards; to change one, drop the index and create it again. +[`VECTOR.CREATE`](/redis/commands/vector/vector-create) fixes two properties for the life of the index: `DIM`, the number of elements every vector must have, and `METRIC`, how similarity is measured. Use `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when your embeddings are already normalized to unit vectors, and `EUCLIDEAN` when straight-line distance does. Neither can be changed afterwards; to change one, drop the index and create it again. [`VECTOR.ADD`](/redis/commands/vector/vector-add) upserts a vector under an ID and [`VECTOR.QUERY`](/redis/commands/vector/vector-query) returns the nearest IDs with a score. Scores are normalized to `0` through `1` for every metric, so a higher score always means a closer match. Search is approximate: `PROFILE` moves a query along the recall and latency trade-off without changing anything stored. diff --git a/redis/commands/vector/vector-add.mdx b/redis/commands/vector/vector-add.mdx index 71bfdbc1b..c93be8d4b 100644 --- a/redis/commands/vector/vector-add.mdx +++ b/redis/commands/vector/vector-add.mdx @@ -31,6 +31,7 @@ VECTOR.ADD ## Important points - The three vector forms are mutually exclusive and exactly one must be given. `VALUES` is the readable form; `FP32` avoids decimal formatting on a binary-safe connection; `BASE64-FP32` carries the same bytes through JSON, which is what the [REST API](/redis/features/restapi) needs. +- On a `DOT` index, the vector must be a unit vector. Other vectors are rejected. - A vector whose length differs from the index's `DIM` returns `ERR vector dimension mismatch: expected , got `. - An index that does not exist returns `ERR Index is not found`. Create it with [`VECTOR.CREATE`](/redis/commands/vector/vector-create) first. diff --git a/redis/commands/vector/vector-create.mdx b/redis/commands/vector/vector-create.mdx index e6dce0eeb..98b5db8d1 100644 --- a/redis/commands/vector/vector-create.mdx +++ b/redis/commands/vector/vector-create.mdx @@ -5,7 +5,7 @@ description: "Create a vector index." Use `VECTOR.CREATE` to create a vector index at a key. -`DIM` fixes the number of elements every vector in the index must have, and `METRIC` fixes how similarity between two vectors is measured. Both are permanent: they decide how the index is built, so they cannot be changed afterwards, and a vector whose length does not match `DIM` is rejected at write time. Pick `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when magnitude carries meaning, and `EUCLIDEAN` when straight-line distance in the embedding space does. +`DIM` fixes the number of elements every vector in the index must have, and `METRIC` fixes how similarity between two vectors is measured. Both are permanent: they decide how the index is built, so they cannot be changed afterwards, and a vector whose length does not match `DIM` is rejected at write time. Pick `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when your embeddings are already normalized to unit vectors, and `EUCLIDEAN` when straight-line distance in the embedding space does. The index occupies its key like any other Redis value, so [`EXISTS`](/redis/commands/generic/exists), [`EXPIRE`](/redis/commands/generic/expire), and [`DEL`](/redis/commands/generic/del) work on it, and creating one at a key already used by another type fails. `EXISTSOK` makes the command succeed instead of failing when the index is already there, which keeps a start-up path idempotent; a mismatch in dimension or metric is still an error. @@ -29,6 +29,7 @@ VECTOR.CREATE DIM METRIC [EXISTSO ## Important points - `DIM` and `METRIC` are both required and cannot be changed after the index is created. Use [`VECTOR.DROP`](/redis/commands/vector/vector-drop) and create it again to change either. +- A `DOT` index accepts only unit vectors, which are vectors with a magnitude of `1`. [`VECTOR.ADD`](/redis/commands/vector/vector-add) and [`VECTOR.QUERY`](/redis/commands/vector/vector-query) reject other vectors. Normalize your vectors first, or use `COSINE`. - Without `EXISTSOK`, creating an index that already exists returns `ERR Index already exists`. - With `EXISTSOK`, an existing index with a different dimension or metric returns `ERR Index already exists with a different configuration`. diff --git a/redis/commands/vector/vector-query.mdx b/redis/commands/vector/vector-query.mdx index f2ec8f85d..a34e9b01d 100644 --- a/redis/commands/vector/vector-query.mdx +++ b/redis/commands/vector/vector-query.mdx @@ -37,6 +37,7 @@ VECTOR.QUERY TOPK - `TOPK` and the query vector are both required; the clauses may be given in any order. - The three vector forms are mutually exclusive and exactly one must be given. `VALUES` is the readable form; `FP32` avoids decimal formatting on a binary-safe connection; `BASE64-FP32` carries the same bytes through JSON, which is what the [REST API](/redis/features/restapi) needs. - Results are ordered by score, highest first. Scores are normalized to `0` through `1` for every metric. +- On a `DOT` index, the query vector must be a unit vector. Other vectors are rejected. - A query vector whose length differs from the index's `DIM` returns `ERR vector dimension mismatch: expected , got `. ## Response diff --git a/redis/overall/changelog.mdx b/redis/overall/changelog.mdx index 5a9c2145d..3613c56e5 100644 --- a/redis/overall/changelog.mdx +++ b/redis/overall/changelog.mdx @@ -4,6 +4,9 @@ title: Changelog - Added the [Insights](/redis/howto/insights) tab to the console for databases with Prod Pack: request routing between regions, client IP ranges and SDKs, latency and throughput charts, usage against plan limits, Lua script statistics and a keyspace analysis. +- Added [hybrid search](/redis/search/hybrid-search) to Redis Search. `HYBRID` fields store a vector embedding, so you can search them by meaning. +- Added [retrievers](/redis/search/retrievers/overview) to Redis Search queries. [`$rrf`](/redis/search/retrievers/rrf) merges the results of several retrievers into one list, [`$ann`](/redis/search/retrievers/ann) searches `HYBRID` fields by meaning, and [`$standard`](/redis/search/retrievers/standard) sets the weight of a query inside `$rrf`. +- A [`$mustNot`](/redis/search/query-operators/boolean-operators/must-not) query with no other conditions now matches every document it does not exclude, with a score of `0`. Inside `$should`, `$mustNot` now matches the documents it does not exclude. Before, `$mustNot` matched no documents in both cases. diff --git a/redis/search/aggregations.mdx b/redis/search/aggregations.mdx index ed8fe8412..a72696a8f 100644 --- a/redis/search/aggregations.mdx +++ b/redis/search/aggregations.mdx @@ -113,7 +113,7 @@ SEARCH.AGGREGATE products '{}' '{"avg_price": {"$avg": {"field": "price"}}, "by_ Use `filter` to restrict which documents participate in the aggregation. -Filtering uses the same query syntax as [queries](./querying). +Filtering uses the same query syntax as [queries](./querying). `$ann` is not supported. diff --git a/redis/search/hybrid-search.mdx b/redis/search/hybrid-search.mdx new file mode 100644 index 000000000..3c0c2f4f0 --- /dev/null +++ b/redis/search/hybrid-search.mdx @@ -0,0 +1,111 @@ +--- +title: Hybrid Search +description: Combine full-text and semantic search with HYBRID fields and retrievers. +--- + +Full-text search finds documents that contain the words of your query. Semantic search finds documents +with a similar meaning, even when they use different words. Hybrid search runs both and merges the +results into one list. + +Hybrid search has two parts: + +- `HYBRID` fields are indexed for both full-text and semantic search. +- [Retrievers](/redis/search/retrievers/overview) run the searches. [`$ann`](/redis/search/retrievers/ann) + runs a semantic search, and [`$rrf`](/redis/search/retrievers/rrf) merges the results of several + retrievers into one list. + +## Hybrid fields + +A `HYBRID` field is a `TEXT` field that also stores a vector embedding. Full-text queries work on it +the same way as on a `TEXT` field, and `$ann` searches its embeddings by meaning. See +[Hybrid Fields](/redis/search/schema-definition#hybrid-fields) for the field options. + +- An index can have more than one `HYBRID` field. +- If the text of a document cannot be embedded, the document is still searchable by its text. +- `HYBRID` fields work with JSON, hash, string, and stream indexes. + +## Create an index + +Declare the fields you want to search by meaning as `HYBRID`: + + + + +```bash +SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT description HYBRID category KEYWORD price F64 FAST +``` + + + + +## Add documents + +Write documents as usual. Search indexes them and creates their embeddings: + + + + +```bash +JSON.SET product:1 $ '{"name": "Studio Headphones", "description": "Over-ear headphones with active noise cancelling", "category": "audio", "price": 199}' +JSON.SET product:2 $ '{"name": "Travel Earplugs", "description": "Soft foam earplugs that keep airplane cabin sound out", "category": "travel", "price": 9}' +JSON.SET product:3 $ '{"name": "Sport Earbuds", "description": "Sweat-proof wireless earbuds for running", "category": "sports", "price": 79}' + +# Wait until the documents are indexed +SEARCH.WAITINDEXING products +``` + + + + +## Search by meaning + +Use [`$ann`](/redis/search/retrievers/ann) to find the documents that are closest in meaning to a +text. This query can find the earplugs, even though their description does not contain the word +"noise": + + + + +```bash +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "block noise on a flight"}}' + +# Search by meaning within one category +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "headphones for running", "filter": {"category": "sports"}}}' +``` + + + + +## Combine full-text and semantic search + +Use [`$rrf`](/redis/search/retrievers/rrf) to run a full-text query and a semantic search, and merge +their results into one list. Documents that rank high in both come first: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "noise cancelling"}, {"$ann": {"field": "description", "query": "noise cancelling"}}]}}' +``` + + + + +To give one side more influence, set its weight. A full-text query takes a weight when you wrap it in +[`$standard`](/redis/search/retrievers/standard): + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"$standard": {"query": {"description": "noise cancelling"}, "weight": 2}}, {"$ann": {"field": "description", "query": "noise cancelling"}}]}}' +``` + + + + +## Next steps + +- See [Retrievers](/redis/search/retrievers/overview) for all retriever parameters and rules. +- See [Schema Definition](/redis/search/schema-definition#hybrid-fields) for field types and options. +- See [`SEARCH.QUERY`](/redis/commands/search/search-query) for all query options. diff --git a/redis/search/introduction.mdx b/redis/search/introduction.mdx index f889757b3..0317aff80 100644 --- a/redis/search/introduction.mdx +++ b/redis/search/introduction.mdx @@ -12,6 +12,7 @@ Upstash Redis Search is our first extension beyond the official Redis spec. Usin - **Auto-Synchronization**: Once you created an index, all write operations are automatically tracked - **Intuitive Query Language**: A type-safe JSON-based query syntax with boolean operators, fuzzy matching, phrase queries, regex support, and more. +- **Hybrid Search**: Search `HYBRID` fields by meaning, and merge full-text and semantic search results into one list - **Production-Ready Performance**: Built on Tantivy, a fast full-text search engine library written in Rust ## Next Steps @@ -38,6 +39,13 @@ Upstash Redis Search is our first extension beyond the official Redis spec. Usin > Learn the JSON-based query language with filters and operators + + Combine full-text and semantic search in one query + + + +```bash +# Search by meaning +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "something to block noise on a flight"}}' + +# Search by meaning within cheap sports products +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "headphones for running", "filter": {"category": "sports", "price": {"$lt": 100}}}}' + +# Use the most accurate profile, and return keys and scores only +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "headphones for running", "profile": "precise"}}' NOCONTENT +``` + + + + +See [Query Options](./overview#query-options) for the `SEARCH.QUERY` options that `$ann` does not +support. diff --git a/redis/search/retrievers/overview.mdx b/redis/search/retrievers/overview.mdx new file mode 100644 index 000000000..3320b3d3d --- /dev/null +++ b/redis/search/retrievers/overview.mdx @@ -0,0 +1,61 @@ +--- +title: Overview +description: Retrievers run one or more queries and return the top documents. +--- + +A retriever runs one or more queries and returns the top documents. Queries describe which documents +match. Retrievers decide how those documents are found and ranked. + +### Available Retrievers + +| Retriever | Description | +|---|---| +| [`$rrf`](./rrf) | Merges the results of 2 to 10 retrievers with Reciprocal Rank Fusion. | +| [`$ann`](./ann) | Finds the documents whose `HYBRID` field is closest in meaning to a text. | +| [`$standard`](./standard) | Runs one query. Use it inside `$rrf` to set the weight of a query. | + + + + +```bash +# Search a HYBRID field by meaning +SEARCH.QUERY products '{"$ann": {"field": "description", "query": "something to block noise on a flight"}}' + +# Merge a full-text query and a semantic search +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "noise cancelling headphones"}, {"$ann": {"field": "description", "query": "noise cancelling headphones"}}]}}' +``` + + + + +### Where to Use Retrievers + +A retriever must be the only key of its object. You can use it in two places: + +- At the root of the query in [`SEARCH.QUERY`](/redis/commands/search/search-query), + [`SEARCH.COUNT`](/redis/commands/search/search-count), and + [`SEARCH.AGGREGATE`](/redis/commands/search/search-aggregate). +- As an entry of `$rrf` `retrievers`. + +A retriever cannot be used inside a query, such as inside `$must`, `$should`, `$mustNot`, or an `$ann` +filter. `$rrf` cannot be used inside another `$rrf`. + +### Query Options + +Some `SEARCH.QUERY` options do not work with retrievers: + +| Option | With `$rrf` | With `$ann` at the root | +|---|---|---| +| `ORDERBY` | Not supported | Not supported | +| `SCOREFUNC` | Supported. It changes the scores of the full-text queries before their results are merged. It does not change `$ann` results. | Not supported | +| `HIGHLIGHT` | Supported | Not supported | + +`$standard` at the root supports all options, the same as a plain query. + +### Counting and Aggregations + +`SEARCH.COUNT` and `SEARCH.AGGREGATE` accept `$rrf` when it has no `$ann` retriever. +They use every document that matches any of the queries. `window` and `weight` do not apply, so the +count can be larger than the number of results `SEARCH.QUERY` returns. + +`SEARCH.COUNT` and `SEARCH.AGGREGATE` do not support `$ann`. diff --git a/redis/search/retrievers/rrf.mdx b/redis/search/retrievers/rrf.mdx new file mode 100644 index 000000000..afaeabe99 --- /dev/null +++ b/redis/search/retrievers/rrf.mdx @@ -0,0 +1,123 @@ +--- +title: $rrf +description: Merge the results of several retrievers with Reciprocal Rank Fusion. +--- + +The `$rrf` retriever runs several retrievers and merges their results with Reciprocal Rank Fusion +(RRF). Each retriever ranks its own results. A document gets points from every retriever that returns +it, so documents that rank high in several retrievers come first. + +Use `$rrf` for hybrid search, to merge a full-text query with an [`$ann`](./ann) semantic search. It +also merges full-text queries only, or `$ann` retrievers only. + +```json +{ + "$rrf": { + "retrievers": [ + { "description": "noise cancelling headphones" }, + { "$ann": { "field": "description", "query": "noise cancelling headphones" } } + ] + } +} +``` + +### Parameters + +| Parameter | Description | Default | +|---|---|---| +| `retrievers` | Required. 2 to 10 retrievers to merge. Each entry is a query object, a [`$standard`](./standard), or an [`$ann`](./ann). | | +| `rankConstant` | An integer between `1` and `2147483647`. Higher values give lower-ranked results more influence. | `60` | +| `window` | An integer between `1` and `11000`. The number of results each retriever contributes, and the maximum number of merged results. See [Paging](#paging). | `LIMIT` | + +### Scoring + +Each retriever that returns a document adds `weight / (rankConstant + rank)` to its score. `rank` is the +position of the document in that retriever's results, starting at `1`. + +These scores are small numbers. Compare them only with other results of the same query. + +### Weights + +A weight scales how much a retriever adds to the score. All retrievers have a weight of `1` by default. + +- To set the weight of a query object, wrap it in [`$standard`](./standard). +- An `$ann` retriever takes `weight` directly. +- A weight must be `0` or greater. With a weight of `0`, the documents of that retriever stay in the + results, but the retriever adds nothing to their score. + +### Paging + +`window` sets how deep `$rrf` looks: + +- Each retriever contributes up to `window` results, and the merged list is cut at `window`. +- When `window` is not set, it is equal to `LIMIT`, which is `10` by default. +- `LIMIT` cannot be larger than `window`. +- To use `OFFSET`, set `window`. A page that starts at or after `window` is empty. + +Keep the same `window` on every page so that pages do not overlap. + +### Examples + +#### Hybrid Search + +Merge a full-text query and a semantic search on the same field: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "noise cancelling headphones"}, {"$ann": {"field": "description", "query": "noise cancelling headphones"}}]}}' +``` + + + + +#### Weighted Hybrid Search + +Give the full-text query twice the weight of the semantic search: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"$standard": {"query": {"description": "noise cancelling headphones"}, "weight": 2}}, {"$ann": {"field": "description", "query": "noise cancelling headphones", "weight": 1}}]}}' +``` + + + + +#### Full-Text Queries Only + +`$rrf` does not need `HYBRID` fields: + + + + +```bash +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"name": "wireless headphones"}, {"description": "bluetooth"}]}}' +``` + + + + +#### Paging Through Results + +Set `window` and keep it the same on every page: + + + + +```bash +# Page 1 +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "headphones"}, {"$ann": {"field": "description", "query": "headphones"}}], "window": 100}}' LIMIT 10 + +# Page 2 +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "headphones"}, {"$ann": {"field": "description", "query": "headphones"}}], "window": 100}}' LIMIT 10 OFFSET 10 +``` + + + + +See [Query Options](./overview#query-options) for the `SEARCH.QUERY` options that work with `$rrf`, and +[Counting and Aggregations](./overview#counting-and-aggregations) for `SEARCH.COUNT` and +`SEARCH.AGGREGATE`. diff --git a/redis/search/retrievers/standard.mdx b/redis/search/retrievers/standard.mdx new file mode 100644 index 000000000..4a49165c2 --- /dev/null +++ b/redis/search/retrievers/standard.mdx @@ -0,0 +1,46 @@ +--- +title: $standard +description: Run one query, and set its weight inside $rrf. +--- + +The `$standard` retriever runs one query. Use it inside [`$rrf`](./rrf) to set the weight of a query. + +```json +{ "$standard": { "query": { "name": "headphones" }, "weight": 2 } } +``` + +### Parameters + +| Parameter | Description | Default | +|---|---|---| +| `query` | Required. A query object. See [Queries](/redis/search/querying). | | +| `weight` | A number, `0` or greater. It scales how much the query adds to the `$rrf` score. Valid only inside `$rrf`. | `1` | + +### Plain Query Objects + +A plain query object works the same as `$standard` without a weight. Use `$standard` only when you need +to set a weight. + +These two `$rrf` retrievers give the same results: + +```json +{ "$rrf": { "retrievers": [{ "name": "headphones" }, { "description": "wireless" }] } } + +{ "$rrf": { "retrievers": [{ "$standard": { "query": { "name": "headphones" } } }, { "$standard": { "query": { "description": "wireless" } } }] } } +``` + +### Examples + + + + +```bash +# Give the name match twice the weight of the description match +SEARCH.QUERY products '{"$rrf": {"retrievers": [{"$standard": {"query": {"name": "headphones"}, "weight": 2}}, {"description": "headphones"}]}}' + +# Same as SEARCH.QUERY products '{"name": "headphones"}' +SEARCH.QUERY products '{"$standard": {"query": {"name": "headphones"}}}' +``` + + + diff --git a/redis/search/schema-definition.mdx b/redis/search/schema-definition.mdx index d3ea73c0d..16f423f6a 100644 --- a/redis/search/schema-definition.mdx +++ b/redis/search/schema-definition.mdx @@ -20,6 +20,7 @@ import { Redis, s } from "@upstash/redis" | `s.date()` | `DATE` | Date/time field | Yes | Yes | Yes | No | | `s.boolean()` | `BOOL` | Boolean field | Yes | No | No | No | | `s.facet()` | `FACET` | Hierarchical path-based field | No | No | No | No (only `$eq`, `$in`) | +| - | `HYBRID` | Full-text field that also supports semantic search | No | Yes | No | `$smart`, `$phrase`, `$fuzzy`, `$regex` | In the TypeScript SDK, `s.number()` defaults to `F64`. You can specify `s.number("U64")` or @@ -92,8 +93,9 @@ const schema = s.object({ ## Multi-value fields A document field can contain several values without a separate array type in the schema. Declare the -field as `TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, or `DATE`, then store its value as a JSON array. -Search indexes every valid array element under the same field. +field as `TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, or `DATE`, then store its value as a JSON array. +Search indexes every valid array element under the same field. For a `HYBRID` field, `$ann` uses one +embedding of all elements together. For example, this schema accepts several tags, descriptions, prices, and restock dates per product: @@ -266,6 +268,29 @@ SEARCH.CREATE idx ON JSON PREFIX 1 prefix: SCHEMA tag KEYWORD --- +## Hybrid Fields + +The `HYBRID` field type is a `TEXT` field that also supports semantic search. Search creates a +vector embedding of the field value, and the [`$ann`](/redis/search/retrievers/ann) +retriever searches these embeddings by meaning. Full-text queries work on the field the same way as +on a `TEXT` field. + +`HYBRID` fields accept the `NOSTEM` and `FROM` options. + + + + +```bash +SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT description HYBRID +``` + + + + +See [Hybrid Search](/redis/search/hybrid-search) for how to query `HYBRID` fields. + +--- + ## Facet Fields The `FACET` field type is for hierarchical path-based faceted search. Values must be `/`-delimited paths starting with `/`. diff --git a/redis/search/streams.mdx b/redis/search/streams.mdx index 9a39e0e97..3b46acc16 100644 --- a/redis/search/streams.mdx +++ b/redis/search/streams.mdx @@ -195,12 +195,12 @@ Redis stores stream field values as strings. Search converts each value accordin - `U64`, `I64`, and `F64` fields require values in the corresponding numeric format. - `BOOL` fields accept `true` or `false`. - `DATE` fields require an RFC 3339 timestamp, such as `2026-08-24T09:30:00Z`. -- `TEXT`, `KEYWORD`, and `FACET` fields use string values. +- `TEXT`, `HYBRID`, `KEYWORD`, and `FACET` fields use string values. Missing fields, empty values, and values that cannot be converted are omitted from the indexed document. If an entry has no valid schema fields, it is not added to the index. -`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields also accept JSON-encoded arrays. Each valid +`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields also accept JSON-encoded arrays. Each valid element is indexed as a separate value of the same field. For example, the `service` field can contain more than one service: