Update search and vector documentation for the 1.19 release

docs.json

@@ -785,6 +785,7 @@
                           "redis/search/document-updates",
                           "redis/search/schema-definition",
                           "redis/search/querying",
+                          "redis/search/hybrid-search",
                           "redis/search/streams",
                           "redis/search/aggregations",
                           "redis/search/counting",
@@ -827,6 +828,15 @@
                               }
                             ]
                           },
+                          {
+                            "group": "Retrievers",
+                            "pages": [
+                              "redis/search/retrievers/overview",
+                              "redis/search/retrievers/rrf",
+                              "redis/search/retrievers/ann",
+                              "redis/search/retrievers/standard"
+                            ]
+                          },
                           {
                             "group": "Aggregation Operators",
                             "pages": [

llms.txt

@@ -1198,6 +1198,7 @@ Search these docs: GET https://upstash.com/docs/search?q=<query> returns JSON {
 - [Counting](https://upstash.com/docs/redis/search/counting.md)
 - [Document Updates](https://upstash.com/docs/redis/search/document-updates.md)
 - [Quickstart](https://upstash.com/docs/redis/search/getting-started.md)
+- [Hybrid Search](https://upstash.com/docs/redis/search/hybrid-search.md): Combine full-text and semantic search with HYBRID fields and retrievers.
 - [Indices](https://upstash.com/docs/redis/search/index-management.md)
 - [Introduction](https://upstash.com/docs/redis/search/introduction.md)
 - [$boost](https://upstash.com/docs/redis/search/query-operators/boolean-operators/boost.md)
@@ -1219,6 +1220,10 @@ Search these docs: GET https://upstash.com/docs/search?q=<query> returns JSON {
 - [E-commerce Search](https://upstash.com/docs/redis/search/recipes/e-commerce-search.md)
 - [Overview](https://upstash.com/docs/redis/search/recipes/overview.md)
 - [User Directory](https://upstash.com/docs/redis/search/recipes/user-directory.md)
+- [$ann](https://upstash.com/docs/redis/search/retrievers/ann.md): Find the documents whose HYBRID field is closest in meaning to a text.
+- [Overview](https://upstash.com/docs/redis/search/retrievers/overview.md): Retrievers run one or more queries and return the top documents.
+- [$rrf](https://upstash.com/docs/redis/search/retrievers/rrf.md): Merge the results of several retrievers with Reciprocal Rank Fusion.
+- [$standard](https://upstash.com/docs/redis/search/retrievers/standard.md): Run one query, and set its weight inside $rrf.
 - [Schemas](https://upstash.com/docs/redis/search/schema-definition.md)
 - [Searching Streams](https://upstash.com/docs/redis/search/streams.md): Index and search individual Redis stream entries.
 - [Troubleshooting](https://upstash.com/docs/redis/search/troubleshooting.md)

redis/commands/search/search-aggregate.mdx

@@ -23,6 +23,8 @@ SEARCH.AGGREGATE <name> '<json_filter>' '<json_aggregations>'
 
 `<json_filter>` is an [Upstash JSON filter](/redis/search/querying). The command accepts an index name or alias. See [Aggregations](/redis/search/aggregations) for the aggregation object and supported operators.
 
+`<json_filter>` can be an `$rrf` retriever when it has no `$ann` retriever. The command then aggregates every document that matches any of the queries. The `window` and `weight` parameters of `$rrf` do not apply. `$ann` is not supported.
+
 ## Response
 
 In `redis-cli --json`, the response is an object keyed by aggregation alias:

redis/commands/search/search-count.mdx

@@ -17,6 +17,8 @@ SEARCH.COUNT <name> '<json_filter>'
 
 `<json_filter>` is an [Upstash JSON filter](/redis/search/querying). The command accepts an index name or alias.
 
+`<json_filter>` can be an `$rrf` retriever when it has no `$ann` retriever. The command then counts every document that matches any of the queries. The `window` and `weight` parameters of `$rrf` do not apply. `$ann` is not supported.
+
 ## Response
 
 Returns the number of matching documents as an integer. Returns `-1` if the index does not exist.

redis/commands/search/search-create.mdx

@@ -5,11 +5,11 @@ description: "Create a search index."
 
 Use `SEARCH.CREATE` to create a search index over JSON, hash, or string values, or over a stream.
 
-`ON` names the type of key to index and `PREFIX` the key prefixes to watch, so the index covers exactly the keys that match both, including keys written after it was created. The `SCHEMA` then declares which fields are searchable and how: `TEXT` fields are analyzed for full-text search, with `NOSTEM` and `NOTOKENIZE` to turn parts of that off, numeric, boolean, and date fields are matched exactly and can be marked `FAST` to make them usable for sorting and scoring, and `KEYWORD` and `FACET` fields are kept whole for exact matching and faceting. `FROM` maps a schema field to a differently named field in the document.
+`ON` names the type of key to index and `PREFIX` the key prefixes to watch, so the index covers exactly the keys that match both, including keys written after it was created. The `SCHEMA` then declares which fields are searchable and how: `TEXT` fields are analyzed for full-text search, with `NOSTEM` and `NOTOKENIZE` to turn parts of that off, numeric, boolean, and date fields are matched exactly and can be marked `FAST` to make them usable for sorting and scoring, and `KEYWORD` and `FACET` fields are kept whole for exact matching and faceting. `HYBRID` fields are `TEXT` fields that also store a vector embedding, so [`$ann`](/redis/search/retrievers/ann) can search them by meaning. `FROM` maps a schema field to a differently named field in the document.
 
 `ON STREAM` indexes a single stream instead of a set of keys, so it takes the stream key in place of `PREFIX`. Each entry added with [`XADD`](/redis/commands/streams/xadd) becomes a document whose fields are the entry's fields and whose ID is the entry ID, which is what makes a stream searchable by content rather than only by ID range. Entries removed from the stream leave the index as well.
 
-`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields can index multiple values. Store the field as a JSON array; for hash and stream indexes, store the complete JSON-encoded array in the field string. Each valid element is indexed separately, and a document matches when any element matches. `BOOL` and `FACET` fields remain single-valued.
+`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields can index multiple values. Store the field as a JSON array; for hash and stream indexes, store the complete JSON-encoded array in the field string. Each valid element is indexed separately, and a document matches when any element matches. `BOOL` and `FACET` fields remain single-valued.
 
 Creating an index starts an initial scan of the matching keys, which `SKIPINITIALSCAN` skips when you only want to index data written from now on. `EXISTSOK` makes the command succeed instead of failing when the index already exists.
 
@@ -30,6 +30,7 @@ SEARCH.CREATE <name>
   [EXISTSOK]
   SCHEMA
     <field> TEXT [NOSTEM] [NOTOKENIZE] [FROM <source_field>]
+    | <field> HYBRID [NOSTEM] [FROM <source_field>]
     | <field> <U64|I64|F64|BOOL|DATE> [FAST] [FROM <source_field>]
     | <field> <KEYWORD|FACET> [FROM <source_field>]
     [...]
@@ -42,7 +43,7 @@ SEARCH.CREATE <name>
 | `ON` | Type of Redis value to index: `JSON`, `HASH`, `STRING`, or `STREAM`. A `STRING` value must contain a JSON object. |
 | `PREFIX` | One or more key prefixes. Prefixes in the same index cannot be duplicates or overlap one another. Not allowed with `ON STREAM`. |
 | `<stream-key>` | Key of the stream to index. Required with `ON STREAM`, and given in place of `PREFIX`. |
-| `LANGUAGE` | Stemming language for `TEXT` fields. Defaults to `english`. Supported values are `arabic`, `danish`, `dutch`, `english`, `finnish`, `french`, `german`, `greek`, `hungarian`, `italian`, `norwegian`, `portuguese`, `romanian`, `russian`, `spanish`, `swedish`, `tamil`, and `turkish`. |
+| `LANGUAGE` | Stemming language for `TEXT` and `HYBRID` fields. Defaults to `english`. Supported values are `arabic`, `danish`, `dutch`, `english`, `finnish`, `french`, `german`, `greek`, `hungarian`, `italian`, `norwegian`, `portuguese`, `romanian`, `russian`, `spanish`, `swedish`, `tamil`, and `turkish`. |
 | `SKIPINITIALSCAN` | Create the index without scanning existing keys. Later writes are still indexed. |
 | `EXISTSOK` | Return `0` when an index with the same data type, prefixes, and schema already exists. A configuration mismatch returns an error. |
 | `SCHEMA` | One or more field definitions. `SCHEMA` must be the final top-level clause. |
@@ -52,11 +53,11 @@ SEARCH.CREATE <name>
 | Option | Valid field types | Description |
 | --- | --- | --- |
 | `FAST` | `U64`, `I64`, `F64`, `BOOL`, `DATE` | Store the field for operations such as sorting and aggregations. Score functions accept `FAST` fields of type `U64`, `I64`, or `F64`. |
-| `NOSTEM` | `TEXT` | Index text without stemming words to their roots. |
+| `NOSTEM` | `TEXT`, `HYBRID` | Index text without stemming words to their roots. |
 | `NOTOKENIZE` | `TEXT` | Index the entire value as one token. |
 | `FROM <source_field>` | All field types | Read the value from a different document field or nested dot path while exposing it under `<field>` in the index. |
 
-`TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields accept multiple values. A field whose value is a JSON array is indexed element by element, and a document matches when any valid element matches. See [Multi-value fields](/redis/search/schema-definition#multi-value-fields) for data-type-specific input formats and query behavior.
+`TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields accept multiple values. A field whose value is a JSON array is indexed element by element, and a document matches when any valid element matches. See [Multi-value fields](/redis/search/schema-definition#multi-value-fields) for data-type-specific input formats and query behavior.
 
 ## Response
 
@@ -73,6 +74,9 @@ SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT price F64 FAST
 
 # Index a stream instead of a set of keys
 SEARCH.CREATE events ON STREAM stream:events SCHEMA message TEXT level KEYWORD
+
+# Index a field for hybrid search
+SEARCH.CREATE articles ON JSON PREFIX 1 article: SCHEMA title TEXT body HYBRID
 ```
 
 </Accordion>

redis/commands/search/search-describe.mdx

@@ -26,7 +26,7 @@ Returns index metadata as alternating field names and values. The response conta
 | `name` | The index name. |
 | `type` | The indexed Redis value type: `HASH`, `JSON`, `STRING`, or `STREAM`. |
 | `prefixes` | Key prefixes included in the index. For a `STREAM` index, the single stream key it covers. |
-| `language` | Stemming language used by `TEXT` fields. |
+| `language` | Stemming language used by `TEXT` and `HYBRID` fields. |
 | `indexSize` | Current index size in bytes. |
 | `pendingUpdates` | Number of updates waiting to be indexed. |
 | `schema` | Field definitions, including `FROM`, `NOSTEM`, `NOTOKENIZE`, and `FAST` options where present. |

redis/commands/search/search-query.mdx

@@ -9,7 +9,9 @@ The filter is a JSON object naming index fields and the values to match, so `'{"
 
 Results come back ordered by relevance score by default. `ORDERBY` sorts by a `FAST` field instead, `LIMIT` and `OFFSET` page through the matches, `SELECT` and `NOCONTENT` cut the payload down to the fields you need, `HIGHLIGHT` wraps the matched terms in tags for display, and `SCOREFUNC` blends numeric fields such as popularity or recency into the relevance score.
 
-For multi-value `TEXT`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields, a document matches when any indexed array element satisfies the condition. Phrase queries on `TEXT` stay within one array element. `HIGHLIGHT` preserves the stored array shape in the response, marking matching string elements while leaving other elements unchanged. See [Multi-value queries](/redis/search/querying#multi-value-fields) for examples.
+For multi-value `TEXT`, `HYBRID`, `KEYWORD`, `U64`, `I64`, `F64`, and `DATE` fields, a document matches when any indexed array element satisfies the condition. Phrase queries on `TEXT` stay within one array element. `HIGHLIGHT` preserves the stored array shape in the response, marking matching string elements while leaving other elements unchanged. See [Multi-value queries](/redis/search/querying#multi-value-fields) for examples.
+
+The query can also be a retriever. `$ann` searches `HYBRID` fields by meaning, and `$rrf` merges the results of several retrievers into one list. With `$ann`, results are ordered by similarity. With `$rrf`, they are ordered by the merged score. See [Retrievers](/redis/search/retrievers/overview) for details.
 
 See [Querying and filtering](/redis/search/querying) for the full filter syntax and worked examples, and [`SEARCH.COUNT`](/redis/commands/search/search-count) when you only need the number of matches.
 
@@ -46,6 +48,8 @@ SEARCH.QUERY <name> '<query>'
 | `SCOREFUNC` | Adjust relevance scores using one or more `FAST` numeric fields. `FACTOR` defaults to `1`, `MISSING` to `0`, `MODIFIER` to `NONE`, `SCOREMODE` to `SUM`, and `COMBINEMODE` to `SUM`. | Disabled |
 <Warning>
   `NOCONTENT` cannot be combined with `SELECT` or `HIGHLIGHT`. `SCOREFUNC` cannot be combined with `ORDERBY`. Inside `MULTI` or `EVAL`, the command requires `NOCONTENT`.
+
+  `ORDERBY` cannot be used with `$rrf` or `$ann`. `SCOREFUNC` and `HIGHLIGHT` cannot be used with `$ann` at the root of the query. With `$rrf`, `LIMIT` cannot be larger than the `$rrf` `window`, and `OFFSET` requires an explicit `window`. See [Query Options](/redis/search/retrievers/overview#query-options) and [`$rrf` paging](/redis/search/retrievers/rrf#paging).
 </Warning>
 
 See [Querying and filtering](/redis/search/querying) for the JSON filter operators and detailed query examples.
@@ -55,7 +59,7 @@ See [Querying and filtering](/redis/search/querying) for the JSON filter operato
 Returns an array of `[key, score, content]` results, or `null` if the index does not exist:
 
 - `key` is the Redis key of the matching document.
-- `score` is the floating-point relevance score.
+- `score` is the floating-point relevance score. With `$ann`, it is the similarity score. With `$rrf`, it is the merged score.
 - `content` is an array of field-value pairs. JSON indexes return `[["$", "<json_string>"]]`; hash indexes return `[["field", "value"], ...]`.
 
 When `NOCONTENT` is used, each result is `[key, score]`. When `SELECT` is used, only fields that exist in the document appear in the content. For an index created with `ON STREAM`, `key` is the entry ID of the matching stream entry.
@@ -77,6 +81,9 @@ A highlighted multi-valued field keeps its array form in the reply. Highlighting
 
 ```bash
 SEARCH.QUERY products '{"name": "wireless"}' LIMIT 10 OFFSET 0
+
+# Merge a full-text query and a semantic search on a HYBRID field
+SEARCH.QUERY products '{"$rrf": {"retrievers": [{"description": "wireless"}, {"$ann": {"field": "description", "query": "wireless"}}]}}' LIMIT 10
 ```
 
 </Accordion>

redis/commands/vector/overview.mdx

@@ -10,7 +10,7 @@ description: "Commands for storing embeddings and running nearest-neighbour sear
 
 A vector index stores embeddings under IDs and answers nearest-neighbour queries over them. It lives at an ordinary Redis key, so [`EXISTS`](/redis/commands/generic/exists), [`EXPIRE`](/redis/commands/generic/expire), and [`DEL`](/redis/commands/generic/del) apply to it, and a key already holding another type cannot be turned into one.
 
-[`VECTOR.CREATE`](/redis/commands/vector/vector-create) fixes two properties for the life of the index: `DIM`, the number of elements every vector must have, and `METRIC`, how similarity is measured. Use `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when magnitude carries meaning, and `EUCLIDEAN` when straight-line distance does. Neither can be changed afterwards; to change one, drop the index and create it again.
+[`VECTOR.CREATE`](/redis/commands/vector/vector-create) fixes two properties for the life of the index: `DIM`, the number of elements every vector must have, and `METRIC`, how similarity is measured. Use `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when your embeddings are already normalized to unit vectors, and `EUCLIDEAN` when straight-line distance does. Neither can be changed afterwards; to change one, drop the index and create it again.
 
 [`VECTOR.ADD`](/redis/commands/vector/vector-add) upserts a vector under an ID and [`VECTOR.QUERY`](/redis/commands/vector/vector-query) returns the nearest IDs with a score. Scores are normalized to `0` through `1` for every metric, so a higher score always means a closer match. Search is approximate: `PROFILE` moves a query along the recall and latency trade-off without changing anything stored.
 

redis/commands/vector/vector-add.mdx

@@ -31,6 +31,7 @@ VECTOR.ADD <index> <id>
 ## Important points
 
 - The three vector forms are mutually exclusive and exactly one must be given. `VALUES` is the readable form; `FP32` avoids decimal formatting on a binary-safe connection; `BASE64-FP32` carries the same bytes through JSON, which is what the [REST API](/redis/features/restapi) needs.
+- On a `DOT` index, the vector must be a unit vector. Other vectors are rejected.
 - A vector whose length differs from the index's `DIM` returns `ERR vector dimension mismatch: expected <dim>, got <n>`.
 - An index that does not exist returns `ERR Index <index> is not found`. Create it with [`VECTOR.CREATE`](/redis/commands/vector/vector-create) first.
 

redis/commands/vector/vector-create.mdx

@@ -5,7 +5,7 @@ description: "Create a vector index."
 
 Use `VECTOR.CREATE` to create a vector index at a key.
 
-`DIM` fixes the number of elements every vector in the index must have, and `METRIC` fixes how similarity between two vectors is measured. Both are permanent: they decide how the index is built, so they cannot be changed afterwards, and a vector whose length does not match `DIM` is rejected at write time. Pick `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when magnitude carries meaning, and `EUCLIDEAN` when straight-line distance in the embedding space does.
+`DIM` fixes the number of elements every vector in the index must have, and `METRIC` fixes how similarity between two vectors is measured. Both are permanent: they decide how the index is built, so they cannot be changed afterwards, and a vector whose length does not match `DIM` is rejected at write time. Pick `COSINE` when only the direction of the embedding matters, which is the usual choice for text embeddings, `DOT` when your embeddings are already normalized to unit vectors, and `EUCLIDEAN` when straight-line distance in the embedding space does.
 
 The index occupies its key like any other Redis value, so [`EXISTS`](/redis/commands/generic/exists), [`EXPIRE`](/redis/commands/generic/expire), and [`DEL`](/redis/commands/generic/del) work on it, and creating one at a key already used by another type fails. `EXISTSOK` makes the command succeed instead of failing when the index is already there, which keeps a start-up path idempotent; a mismatch in dimension or metric is still an error.
 
@@ -29,6 +29,7 @@ VECTOR.CREATE <index> DIM <dimension> METRIC <COSINE | EUCLIDEAN | DOT> [EXISTSO
 ## Important points
 
 - `DIM` and `METRIC` are both required and cannot be changed after the index is created. Use [`VECTOR.DROP`](/redis/commands/vector/vector-drop) and create it again to change either.
+- A `DOT` index accepts only unit vectors, which are vectors with a magnitude of `1`. [`VECTOR.ADD`](/redis/commands/vector/vector-add) and [`VECTOR.QUERY`](/redis/commands/vector/vector-query) reject other vectors. Normalize your vectors first, or use `COSINE`.
 - Without `EXISTSOK`, creating an index that already exists returns `ERR Index <index> already exists`.
 - With `EXISTSOK`, an existing index with a different dimension or metric returns `ERR Index <index> already exists with a different configuration`.