Add bounded host storage transport for Tantivy - #22
Merged
Conversation
kylebernhardy
force-pushed
the
codex/issue-7-harper-storage-proof
branch
from
September 9, 2026 04:16
625260c to
6f324a3
Compare
This was referenced Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a bounded, versioned request/response bridge so Tantivy directory operations can run against caller-owned Harper storage. This is the transport and durability-policy foundation for Phase 0: prove the Harper-backed Tantivy storage and derived-index path; the real Harper/RocksDB adapter remains the next slice, so this PR does not close the issue.
For the human reviewer
HostStorage.write()applies an atomic batch through Harper’s existing WAL-backed transaction primitive. Because current rocksdb-js does not expose per-write WAL sync, a TantivyWAL_SYNCrequest is deliberately executed asatomic WAL write → explicit durability barrier. If the write succeeds and the barrier fails, the bytes are known-applied but not proven durable; the native commit fails and poisons that generation. The adapter must not claim rollback or continue writing.Verification
The Node proof constructs a host-backed
KvDirectory, runs a real Tantivy create/write/commit/reopen lifecycle through the native transport, and verifies atomic WAL writes followed by explicit durability barriers. A fault test proves a failed barrier does not misreport the preceding write as rolled back./bin/zsh -licwith the pinned Rust toolchain passed formatting, TypeScript, the native build, and all 13 focused host-storage tests. Earlier full validation passed 38 Rust tests, 33 active Node tests, packed-package verification, and every CI job across Linux, macOS, and Windows; CI is rerunning for the corrected head. Three legacy rocksdb-js lease experiments remain intentionally skipped because that provider will not ship.Local Claude and Gemini CLI review completed for the durability correction. Gemini reported no findings; Claude’s applicable concerns are captured above.
Comment generated by kAIle (GPT-5)
Complexity: complicated
Review-Coverage: authored=codex; ran=claude,gemini; declined=cursor-grok,cursor-composer,domain; rounds=6 @ 6f324a3
Human-Review-Need: 4 @ 6f324a3