Benchmark target, README that matches it, and release 1.1.0

API.md

@@ -22,7 +22,7 @@ Returns the current version of the SQLite Vector Extension.
 
 ```sql
 SELECT vector_version();
--- e.g., '1.0.0'
+-- e.g., '1.1.0'
 ```
 
 ---

CHANGELOG.md

@@ -6,11 +6,52 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## [Unreleased]
 
+## [1.1.0] - 2026-08-24
+
+A source audit closed thirteen defects, including two crashes and four memory-safety
+issues, and found that every x86 release had been shipping scalar code. Search is
+substantially faster as a result, and the SIMD kernels are now actually tested.
+
+### Added
+
+- **`normalized=1` option for `vector_init()`**: declares that every stored vector is unit length. With `type=FLOAT32` and `distance=COSINE` a full-precision scan then computes `1 - dot` instead of the full cosine, dropping two thirds of the arithmetic from the inner loop. The query is normalized once per scan, so reported distances are unchanged. It is an assertion about your data, not a request: if the stored vectors are not unit length the distances will be wrong. Quantized scans ignore it.
+- **`make benchmark`**: brute-force k-NN benchmark across every storage and quantization mode, with recall scored against the exact scan. Defaults to k=20 over 1,000,000 vectors of dimension 768; see the Benchmark section of the README.
+- **`make unittest-simd`**: runs the test suite against the SIMD kernels. The existing `unittest` target builds every source in one invocation, which left `__AVX2__` and `__AVX512F__` undefined and silently exercised the scalar fallback instead.
+- **CI coverage for the AVX-512 kernels**, on hardware where the runner has it and under Intel SDE where it does not, with an assertion that the expected backend was the one that ran.
+
+### Changed
+
+- **x86 builds now contain the AVX2 and AVX-512 kernels.** They were guarded by `__AVX2__` / `__AVX512F__`, which the build never defined, so both compiled to nothing — and because the runtime dispatch was an `if / else if` ladder, a CPU reporting AVX2 called an empty stub and never fell back to the SSE2 kernels that were compiled in. Every x86 release so far ran the plain C fallback.
+- **Faster distance kernels.** The FLOAT32 kernels now use four independent accumulators and true FMA; the UINT8/INT8 kernels were rewritten around the instructions built for them (`vabd`/`vmull`/`vpadal` on NEON, `PSADBW`/`PMADDWD` on x86). AVX2 and AVX-512 cosine no longer makes three separate passes over the data. Measured on Apple M5 Pro at dimension 768, a preloaded quantized scan went from 17.4 to 62.6 Mvec/s. Accuracy improved as well: reductions widen to 64 bits before folding lanes, and integer cosine sums exact integers rather than accumulating in `float`.
+- **Faster top-k.** The candidate set is a binary max-heap instead of a linear rescan plus an exchange sort. At k=1000 over 20,000 rows a 1-bit scan went from 3.17 ms to 0.16 ms per query; at k=4000, from 33.5 ms to 0.55 ms. Small k is unchanged.
+- **TurboQuant lookup scans return the same distance on every CPU.** The per-backend implementations accumulated in `float` while the scalar one used `double`, so results differed by up to 1.5e-4 relative depending on which backend ran — enough to reorder near-ties. There is now one implementation.
+- **`vector_turboquant_backend()`** still returns the same strings, but the value identifies the SIMD tier selected at load time rather than a TurboQuant-specific code path, since there is only one.
+- **`qtype=AUTO` on a `BIT` column now means `1BIT`.** Previously it failed on a populated table with an unrelated message, and on an empty one silently recorded `UINT8`, which then applied to rows inserted later. An explicit 8-bit request on a `BIT` column is now refused with a message that says so.
+- **`distance=HAMMING` is rejected for any type other than `BIT`** at `vector_init()` time.
+
 ### Fixed
 
+- **Crash on `distance=HAMMING` with a non-`BIT` vector type.** The dispatch table only implements Hamming for `BIT`, and the combination was accepted, so scanning called a NULL function pointer.
+- **Out-of-bounds read from an undersized query vector.** A query passed as a `BLOB` was never length-checked, so the distance kernels read `dimension` elements from whatever the caller supplied. The JSON form already validated this.
+- **SQL injection through table and column names.** Identifiers were interpolated with `%q`, which escapes string literals and does nothing for `;` or `"`. A table whose name contains a semicolon could inject statements through `vector_quantize_cleanup()`. This also fixes ordinary names: tables or columns containing spaces or dashes previously failed with a syntax error and now work.
+- **`ORDER BY` was silently ignored** on `vector_full_scan()` and `vector_quantize_scan()`. The planner was told the cursor already satisfied any ordering, so SQLite dropped the sorter — including for `ORDER BY distance DESC`.
+- **Out-of-bounds reads on a malformed quantized index.** The `UINT8`/`INT8`/`1BIT` scan paths decoded rows without checking the blob against the row count it claimed. The shadow table is an ordinary writable table.
+- **Heap buffer overflow in `vector_quantize_preload()`.** The buffer was sized from `SUM(LENGTH(data))` and filled with no per-row bound. No concurrency was needed to trigger it: `LENGTH()` counts characters on a TEXT value while the byte length can be larger.
+- **Use-after-free when `vector_quantize_cleanup()` or `vector_quantize_preload()` ran while a streaming cursor was open.** The in-memory index is now reference counted, so a scan keeps reading a consistent snapshot.
+- **Double free on the extension-init error path**, which SQLite could reach immediately because it invokes the destructor when `sqlite3_create_function_v2()` itself fails.
+- **`k = 0`** returned an error code from `xFilter` instead of an empty result.
+- **Undefined float-to-int conversion** in the unrolled 8-bit quantizers for NaN and out-of-range values.
+- **Primary-key detection on `WITHOUT ROWID` tables** bound a parameter to a statement that has none and read columns from an arbitrary row of a grouped query.
+- **`vector_quantize()` reported "not an error"** whenever the failure came from the extension rather than from SQLite.
+- **Uninitialised bytes** in the index when a `BIT` column was quantized to 8 bits.
 - **GCC 13 build failure on AVX2 targets**: a static `__m256i` initializer is now built from a plain byte array, so the extension compiles with GCC 13's stricter constant-expression rules.
 - **Swift Package**: removed the deprecated `.iOS(.v11)` platform declaration that produced a warning (and, on recent toolchains, an error) when resolving the package.
 
+### Notes
+
+- **For cosine, prefer `qtype=INT8` over `UINT8`.** They are the same size and the same speed, but unsigned quantization subtracts the dataset minimum before scaling, and cosine measures angle, which that shift destroys. Omitting `qtype` selects `UINT8` for non-negative data, which is correct for L2 and wrong for cosine. See the Benchmark section of the README.
+- **Tie-breaking among equal distances changed** with the new top-k. Neither ordering was stable and none was guaranteed, but it is observable.
+
 ## [1.0.0] - 2026-05-25
 
 ### Added

Makefile

@@ -181,6 +181,33 @@ unittest-simd: $(BUILD_DIR)/backend $(BUILD_DIR)/test_vector_simd
 	$(RUNNER) ./$(BUILD_DIR)/backend $(EXPECT_BACKEND)
 	$(RUNNER) ./$(BUILD_DIR)/test_vector_simd
 
+# Brute-force k-NN benchmark across every storage and quantization mode, with recall
+# measured against the exact scan. Built at -O3 with the same per-unit ISA flags as the
+# shipped extension, so it measures the kernels that actually ship.
+#
+#   make benchmark                                    k=20 over 1M vectors of dim 768
+#   make benchmark NVECS=100000 DIM=384 K=10          smaller, for a quick look
+#   make benchmark DISTANCE=l2                        a different metric
+NVECS ?= 1000000
+DIM ?= 768
+K ?= 20
+NQUERIES ?= 20
+DISTANCE ?= cosine
+
+BENCH_OBJ = $(patsubst %.c, $(BUILD_DIR)/bm-%.o, $(notdir $(SRC_FILES))) $(BUILD_DIR)/bm-sqlite3.o
+
+$(BUILD_DIR)/bm-distance-avx2.o: ISA_CFLAGS := $(AVX2_CFLAGS)
+$(BUILD_DIR)/bm-distance-avx512.o: ISA_CFLAGS := $(AVX512_CFLAGS)
+
+$(BUILD_DIR)/bm-%.o: %.c
+	$(CC) $(CFLAGS) $(ISA_CFLAGS) -DSQLITE_CORE -O3 -c $< -o $@
+
+$(BUILD_DIR)/benchmark: test/benchmark.c $(BENCH_OBJ)
+	$(CC) $(CFLAGS) -DSQLITE_CORE -O3 -DNVECS=$(NVECS) -DDIM=$(DIM) -DK=$(K) -DNQUERIES=$(NQUERIES) -DDISTANCE='"$(DISTANCE)"' $< $(BENCH_OBJ) -o $@ -lm -lpthread
+
+benchmark: $(BUILD_DIR)/benchmark
+	./$(BUILD_DIR)/benchmark
+
 # Clean up generated files
 clean:
 	rm -rf $(BUILD_DIR)/* $(DIST_DIR)/* *.gcda *.gcno *.gcov *.sqlite

README.md

@@ -145,6 +145,70 @@ SELECT e.id, v.distance FROM images AS e
    LIMIT 10;
 ```
 
+## Benchmark
+
+Every number below comes from one command, so you can reproduce it and compare machines:
+
+```bash
+make benchmark
+```
+
+That builds `test/benchmark.c` at `-O3` with the same per-translation-unit SIMD flags the
+shipped extension uses, then searches **k=20 over 1,000,000 vectors of dimension 768**
+with cosine distance, 20 queries, reporting the best. Recall is the overlap with the exact
+full-precision top-20. Override any of it:
+
+```bash
+make benchmark NVECS=100000 DIM=384 K=10 DISTANCE=l2
+```
+
+### Apple M5 Pro (6P+12E, 64 GB, macOS 26.6.2) — NEON backend
+
+| Mode | Index | ms/query | Mvec/s | Recall@20 |
+| --- | ---: | ---: | ---: | ---: |
+| `FLOAT32` exact | 2930 MB | 147.8 | 6.8 | 100.0% |
+| `UINT8` | 740 MB | 55.4 | 18.0 | 33.8% |
+| `UINT8` preloaded | 740 MB | 37.2 | 26.9 | 33.8% |
+| `INT8` | 740 MB | 56.4 | 17.7 | 99.5% |
+| **`INT8` preloaded** | **740 MB** | **37.7** | **26.5** | **99.5%** |
+| `1BIT` | 99 MB | 5.3 | 187.5 | 10.0% |
+| `1BIT` preloaded | 99 MB | 2.7 | 377.6 | 10.0% |
+| `TURBO2` | 195 MB | 53.0 | 18.9 | 45.2% |
+| `TURBO2` preloaded | 195 MB | 48.2 | 20.7 | 45.2% |
+| `TURBO4` | 378 MB | 160.4 | 6.2 | 81.8% |
+| `TURBO4` preloaded | 378 MB | 151.8 | 6.6 | 81.8% |
+
+*Contributions from other CPUs welcome — run the command above and open a PR adding a
+section.*
+
+### Reading the table
+
+**The data is uniform random**, which is the worst case for every quantizer: real
+embeddings have structure that quantization exploits, so recall on your own vectors will
+be higher, often much higher. Treat the recall column as a floor and a way to rank the
+modes against each other, not as a prediction for your dataset.
+
+Three things are worth knowing before you pick a mode.
+
+**For cosine, use `INT8`, not `UINT8`.** They cost exactly the same and store exactly the
+same number of bytes, but `UINT8` recall collapses to 33.8% while `INT8` holds 99.5%.
+Unsigned quantization subtracts the dataset minimum before scaling, and cosine measures
+angle, which that shift destroys. `UINT8` is the right choice for L2, where a common
+translation cancels out. If you do not set `qtype`, the extension picks `UINT8` for
+non-negative data and `INT8` otherwise — which is the correct call for L2 and the wrong
+one for cosine, so set it explicitly when you use cosine.
+
+**`1BIT` is a filter, not an answer.** 377 Mvec/s and 30x less memory, at 10% recall on
+this data. It earns its place as a first pass whose survivors you re-rank at full
+precision, not as the final ranking.
+
+**TurboQuant trades speed for size, not for speed.** `TURBO4` here is *slower* than the
+exact scan (160 ms against 148 ms) while using 8x less memory and returning 81.8% recall.
+The lookup-table scan is one table gather per row, and at dimension 768 that is 384
+gathers into a 393 KB table for every vector — already about one lookup per cycle, so
+there is no headroom left in the current storage layout. Choose TurboQuant when the
+memory budget is what binds; choose `INT8` when throughput is.
+
 ## TurboQuant Benchmark and Recall
 
 TurboQuant can be selected with `qtype=TURBO,qbits=N`, where `N` is `2`, `3`, or `4`. Shorthand aliases are also available: `TURBO2`, `TURBO3`, and `TURBO4`.
@@ -157,13 +221,15 @@ SELECT vector_quantize('images', 'embedding', 'qtype=TURBO,qbits=4');
 SELECT vector_quantize('images', 'embedding', 'qtype=TURBO2');
 ```
 
-The following benchmark compares `vector_full_scan()` brute force against `vector_quantize_scan()` using TurboQuant on a synthetic dataset of **1,000,000 vectors**, **768 dimensions**, **DOT** distance, **k=10**, and **5 queries**. The database was file-backed, with raw vectors stored in SQLite and quantized data preloaded for the scan. Recall is measured as overlap with exact brute-force top-10 results. These numbers were measured on macOS ARM64 using the NEON backend; timings vary by CPU, storage, cache settings, and allocator behavior.
-
-| Mode | Quantized storage | Max RSS | Peak memory footprint | Full scan / query | TurboQuant / query | Speedup | Recall@10 |
-| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
-| TurboQuant 4-bit | 396 MB | ~488 MB | ~487 MB | 3248 ms | 218 ms | 14.92x | 0.84 |
-| TurboQuant 3-bit | 300 MB | ~394 MB | ~393 MB | 1727 ms | 188 ms | 9.19x | 0.74 |
-| TurboQuant 2-bit | 204 MB | ~310 MB | ~310 MB | 3265 ms | 85 ms | 38.27x | 0.48 |
+An earlier synthetic benchmark reported speedups of 15x for 4-bit and 38x for 2-bit
+against `vector_full_scan()`. Those numbers were measured with a **file-backed** database,
+where the full scan reads 3 GB of raw vectors off disk and the comparison is dominated by
+I/O rather than by arithmetic — and before the distance kernels were rewritten, which made
+the full-precision scan itself substantially faster. Against an in-memory baseline on
+current code the picture is different: see [Benchmark](#benchmark) below, where `TURBO4`
+is marginally slower than the exact scan and its argument is memory, not speed. Both
+measurements are real; they answer different questions. If your working set does not fit
+in RAM, the file-backed comparison is the one that describes your deployment.
 
 For comparison, the raw `FLOAT32` vectors alone are about **3.07 GB** for 1M x 768 before SQLite row/page overhead. TurboQuant 4-bit reduces the scan representation to about **13%** of that raw vector payload, TurboQuant 3-bit to about **10%**, and TurboQuant 2-bit to about **7%**. Actual resident memory depends on whether the database is in-memory or file-backed, SQLite cache settings, preloading, page cache behavior, and the host allocator.
 
@@ -296,6 +362,19 @@ You can store your vectors as `BLOB` columns in ordinary tables. Supported forma
 
 Simply insert a vector as a binary blob into your table. No special table types or schemas are required.
 
+A stored column is quantized separately with `vector_quantize(table, column, 'qtype=...')`,
+which builds a compact index the scan reads instead of the raw vectors:
+
+| `qtype` | Bytes per dimension | Notes |
+| --- | ---: | --- |
+| `UINT8` | 1 | Asymmetric. Correct for L2; see the [benchmark](#benchmark) before using it with cosine |
+| `INT8` | 1 | Symmetric. The default choice for cosine |
+| `1BIT` | 1/8 | Hamming only. A pre-filter to re-rank, not a final ranking |
+| `TURBO2` / `TURBO3` / `TURBO4` | 1/4, 3/8, 1/2 | Lookup-table scan; smallest indexes, see [TurboQuant](#turboquant-benchmark-and-recall) |
+
+Omitting `qtype` picks `UINT8` for non-negative data and `INT8` otherwise. `BIT` columns
+are already binary, so `1BIT` is the only quantization they accept.
+
 
 ### Supported Distance Metrics
 
@@ -306,10 +385,15 @@ Optimized implementations available:
 * **L1 Distance (Manhattan)**
 * **Cosine Distance**
 * **Dot Product**
-* **Hamming Distance** (available only with 1bit vectors)
+* **Hamming Distance** (available only with 1bit vectors — `vector_init()` rejects it for any other type)
 
 These are implemented in pure C and optimized for SIMD when available, ensuring maximum performance on modern CPUs and mobile devices.
 
+If your embeddings are already unit length, say so with `normalized=1`: cosine on a
+`FLOAT32` column then reduces to `1 - dot`, dropping two thirds of the arithmetic from the
+inner loop for the same results. It is an assertion about your data, not a request — see
+[API.md](API.md#vector_inittable-column-options).
+
 ---
 
 # What Is Vector Search?

src/sqlite-vector.h

@@ -24,7 +24,7 @@
 extern "C" {
 #endif
 
-#define SQLITE_VECTOR_VERSION "1.0.0"
+#define SQLITE_VECTOR_VERSION "1.1.0"
 
 SQLITE_VECTOR_API int sqlite3_vector_init (sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi);