Repository navigation
NU-009 Persist to Neo4j: alias matching, delta apply, refcounted provenance #23
Description
Part of #30.
Persist compiled triples into Neo4j with correct matching, delta application and refcounted
provenance.
Build
Neo4jPropertyGraphStorewithembed_kg_nodes=True. Embeddings live on the graph nodes, not
pgvector: the HNSW hit is a graph node, so entry and expansion happen in one store on one
identifier. Splitting them would mean every persist writes twice, and a half-landed pair leaves
a vector pointing at a missing node — silent, and directly on the only retrieval path.- Match by
resolve_key(name, label, aliases)— natural key extended by confirmed aliases. - Apply
plan_delta: create adds, update changed in place, drop refs, leave unchanged alone and
do not re-embed them — that is where the embedding spend is saved. - Provenance as a set per node. Re-approve removes only the old ref; a node disappears only
when its last ref goes. - Embedding model id stored on each node so a partial migration is detectable.
The refcounting trap
An entity can come from several documents — the fixture guarantees it, since session store
appears in both A and B. With a single-valued provenance field, "remove everything from
A-v1" deletes a node B still depends on. Silently. Detectable only by a query that later fails
to traverse. Hence a set, and refcounted deletion.
This is FR-7's problem, solved cheaply
Scoping a re-approve to the touched slice is exactly FR-7's touched-subtree curation — but at
write time, where provenance refs hand you the touched set directly. No Merkle tree, no
hash-of-subtree hypothesis, no determinism gamble. FR-7's Merkle question remains open only for
scheduled re-curation.
One-way door
The embedding dimension (1024) is baked into the Neo4j vector index. Changing model or
dimensions later means re-embedding every node and rebuilding the index — a migration, not a
config edit.
Closes
- A6 — persist-apply half. NU-003 (deterministic core) closes the delta-classification half:
that story provesplan_deltacorrectly classifies add/update/unchanged; this story proves that
classification is applied to Neo4j so node count stays N after a re-approve, not 2N. - A7 — refcount-apply half. NU-003 closes the refcount-logic half: that story proves
release_refcomputes the correct remaining count; this story proves the conditional delete
actually happens (or doesn't) in the graph. - A9 — fully.
Activity
[-]Persist to Neo4j: alias matching, delta apply, refcounted provenance[/-][+]NU-008 Persist to Neo4j: alias matching, delta apply, refcounted provenance[/+][-]NU-008 Persist to Neo4j: alias matching, delta apply, refcounted provenance[/-][+]NU-009 Persist to Neo4j: alias matching, delta apply, refcounted provenance[/+]