[lucidprogrammer]

Adds vector columns and exact search to the workspace under v2/. Nothing that builds from crates/ changes; the root Cargo.toml and Cargo.lock are untouched. One commit on dev 6b44713.

What is in it

• Vector columns and exact search, the Postgres and pgvector way. VECTOR(n) columns; SELECT id, embedding <=> $q AS distance FROM notes ORDER BY distance LIMIT 10 returns the nearest rows with their cosine distance, filtered by the statement's WHERE. The answer equals a full sort by distance; rows at the same distance come in primary-key order. There is no hidden score column.
• How vectors are stored. Each vector once at full precision, plus a one-byte-per-number copy packed into blocks of about a quarter of a megabyte per column and partition. A search screens the blocks and reads the full vector only of the rows that can reach the answer. PARTITION_KEY (column, ...) lays vectors out so that a filtered search reads only its part; MAX_PARTITIONS (256 unless declared) counts the partitions that hold a vector.
• Sealing is a call. New vectors wait on a list and are searched directly; Database::run_maintenance_cycle packs them into blocks, at most 4,096 a call, in one commit. Nothing in this version calls it on its own.
• Large values share pages. Text, JSON and vectors over about 1 KiB are kept beside their row in small slices that share pages, where the first version gave each value a page. Text and JSON up to 1 MiB are stored compressed (lz4_flex) when that saves an eighth or more.
• Storage library. Two more changes, listed in v2/redb/README.md: a page that fills because records were added at its end stays full, and past 1 MiB the file grows in small steps where upstream doubles it (34 lines added, 15 removed in v2/redb/src).
• Memory. A session's page cache is a quarter of the memory limit in force when it opens, at least 8 MiB, and 8 MiB with no limit. SET MEMORY_LIMIT is refused, with the action to take, only below what the session's cache and the statement in flight already hold.
• Fixes to the first version. Closing a store releases its locks explicitly, so a store closed and reopened at once is never refused because a child process inherited a copy of its lock. A number written with a minus sign is a negative literal, so WHERE k = -1 uses an index as k = 1 does. Stored bytes of a large value, a vector record or a vector declaration that disagree with themselves are reported as damage, never a panic.
• The check. .github/workflows/check.yml now states the size of the shipped code rounded up to 500 lines and the number of tests rounded up to 10, and fails when a change crosses either without editing the number; it also refuses a shipped file over 1,500 lines, a test-only name in shipped code, an ignored test and an allow attribute beyond the four v2/AGENTS.md names. v2/AGENTS.md gains "Making a change".

Refused by name until a later version: SEARCH_MODE INDEXED and USE VECTOR INDEXED (the index), a search without LIMIT, a second sort key after a distance, a distance in WHERE, ORDER BY ... DESC on a distance, OFFSET with a search. <-> and <#> are not in the grammar.

Measured at this commit

By v2/numbers.sh: a release build on a 16-core AMD Ryzen 9 9955HX with 64 GB of memory and an NVMe drive, with nothing else running.

	First version	This change
20,000 rows of 1.6 KB, one commit per row	17.2 s, 60.3 MiB	11.0 s, 2.4 MiB (the script's rows repeat their text, which compresses far better than real text)
COUNT(*) of those rows; open after a clean close	0.026 ms; 0.6 ms	0.025 ms; 0.6 ms
One million rows under a 64 MiB limit	11.9 s to load, 246 ms to scan; 13.4 MiB allocated	9.7 s, 239 ms; 16.4 MiB allocated
Ten nearest of 20,000 vectors, sealed, through SQL (95th percentile)		1.5 ms; the store is 44.3 MiB, 1.50 times the bytes it holds
First build of a program that embeds the library	5 s, 46 packages, 2.7 MiB	6 s, 47 packages, 3.0 MiB
Fresh clone to a green cargo test	2.1 min	2.7 min
Shipped lines; tests	7,369; 98	10,809; 171 (155 tests and 16 SQL script files)

Against the current engine, like for like

The same rows loaded clean into both engines, 500 to a commit, into the same table: text passages with one 768-number embedding each, in 8 partitions (a real workload that is not in the repository). The same 100 questions, a read session at 1 GiB, one query at a time, one core each, 95th percentile, same machine. Both engines return the same ten rows in the same order for every question.

	Current engine	This change
11,588 passages: find in one / several / all partitions	49.5 / 173.7 / 391.8 ms	0.3 / 0.8 / 1.5 ms
11,588 passages: store on disk; time to load	200.5 MiB; 4.3 s	48.8 MiB; 0.4 s
100,008 passages: find in one / several / all partitions	522 / 1,454 / 4,031 ms	1.6 / 4.9 / 10.9 ms
100,008 passages: store on disk; time to load	2,465 MiB; 58 s	418 MiB; 3.5 s

The figures of this change for one and several partitions are with no memory limit declared, which is its slower case (an 8 MiB page cache); under 64 MiB the all-partition find at 100,008 passages is 14.4 ms. Before the vectors are sealed the all-partition find is 42 ms at 11,588 passages and 382 ms at 100,008. 173 files of 343.9 MiB of session text, stored as rows, make a 227.9 MiB store.

Verification

• Every command of check.yml as written, on the commit, plus a cross-compiled lint for aarch64-apple-darwin: all pass. 155 of 155 tests and the documentation tests; the storage library's 202 (upstream's 196 and 6 for the changes); the build on Rust 1.89; 10,809 shipped lines against the stated 11,000 and 171 tests against the stated 180.
• GitHub Actions on 5d6b3a66e: https://github.com/context-graph-ai/contextdb/actions/runs/42974637240 — green, 4 jobs (lint, tests on Linux, tests on macOS, Rust 1.89), 8 min (06:45–06:53 UTC). The existing workflow does not run for a change under v2/.
• Reviewed by a second model family: SHIP, no defect. The changes made after that review were reviewed again: SHIP.

Known limits of this version

• Nothing seals on its own in this version; a host calls run_maintenance_cycle after writing vectors. A search before that returns the same rows, slower.
• A slot left by an updated or deleted vector stays in its block until its partition empties; rewriting blocks comes with the index.
• An UPDATE writes a row's large values again even when it does not assign them.
• A key has no limit on its length; a limit is set with the rest of the data rules.
• Four calls of Database that return a plain value panic on a handle that was closed.