Repository navigation
NU-003 Deterministic pipeline core: identity, header parse, naming, delta #17
Description
Part of #30.
The pure, LLM-free core: content identity, Content Header parsing, entity naming and aliases,
the persistence delta, and provenance refcounting. Everything reproducible byte-for-byte.
Build
services/nuron-ai/src/nuron_ai/core.py:
content_hash(bytes)/object_key(digest)— sha256 identity, two-char-prefixed object key.parse_header(text, filename, source_owner)→ContentHeader(subject, author, author_source, timestamp, tags). Precedence: frontmatter → in-prose signature → filename date. File mtime
is never consulted — that is the file's date, not the decision's.normalize(name)/natural_key(name, label)/resolve_key(...)/merge_alias(...).plan_delta(prior, current)→add | update | unchanged | drop_ref, withneeds_embedding.release_ref(refs, ref)→(remaining, delete_node?).
The parser and the fixture are designed together
A2 asserts author_source == "extracted". Fixture A (now authored in #31) carries its attribution
as an in-prose signature, not frontmatter. If the parser only reads frontmatter, the fallback
fires, author_source == "default", and A2 goes red for a reason that has nothing to do with what
A2 tests. The signature regex is therefore load-bearing, and it must survive markdown conversion
so the same rule works on LlamaParse output (A11).
Known ceiling
normalize under-merges on purpose. "session store" and "sessions table" stay distinct;
only a human merge (A10) unifies them. It deliberately does not singularize — naive stemming
would merge referents that genuinely differ. Upgrade path is embedding-similarity matching
(PRD FR-3), which needs corpus volume to calibrate a threshold.
Closes
- A2 — fully. Parser-determined; nothing downstream can change whether attribution was
extracted or defaulted. - A5 — hash/identity half only. NU-005 (watcher) closes the scan-skip half: this story proves
content_hash/plan_deltacorrectly classify "unchanged"; NU-005 proves a re-scan actually
consults that classification and skips work. - A6 — delta-classification half only. NU-009 (persist) closes the apply half: this story
provesplan_deltacorrectly classifies add/update/unchanged; NU-009 proves that classification
is applied to Neo4j without doubling node count. - A7 — refcount-logic half only. NU-009 closes the apply half: this story proves
release_ref
computes the correct remaining count; NU-009 proves the conditional delete actually happens (or
doesn't) in the graph. - A10 — matcher half only. NU-010 (merge gate 2) closes the confirmation half: this story
proves the alias/natural-key matching logic; NU-010 proves the human-confirm flow that invokes
it and makes re-ingest of D a no-op. - A12 — identity half only. See NU-004 (storage half), NU-005 (directory half), NU-012 (upload
half) for the rest.
Status
Not started. An earlier draft was written outside the tracker (21 tests green) and has been deleted, since it preceded the story. The design above is what it validated, so the second attempt should reach green quickly.
Activity
[-]Deterministic pipeline core: identity, header parse, naming, delta[/-][+]NU-002 Deterministic pipeline core: identity, header parse, naming, delta[/+][-]NU-002 Deterministic pipeline core: identity, header parse, naming, delta[/-][+]NU-003 Deterministic pipeline core: identity, header parse, naming, delta[/+]