Skip to content

Tracer Bullet 01 — document to cited answer #15

Closed

Description

@BasuruK

Retired — consolidated into #30. This map and #30 (the /to-spec output) restated the same architecture, assertions and decisions in two places. #30 is now the single tracker issue for Tracer Bullet 01: all 14 child stories (NU-001…NU-014, #16–#28, #31) have been reparented from here to #30, and #30 has absorbed this issue's Frontier/blocking table. docs/tracer-bullet-01.md remains the long-form rationale document, unaffected by this consolidation.

The fixture-corpus source text preserved in the comment thread below remains the source of truth for NU-002/#31 — nothing there has moved; it's just reached via #30 now instead of here.

Everything below this line is the original map body, kept for history.


Part of #14.

Wayfinding map for the first vertical slice of Nuron. Design is in docs/tracer-bullet-01.md; this map carries the scope, the assertions, and the frontier.

Superseded by #30, which now carries the full Frontier/blocking table.

Scope

A document lands (watched directory or upload) → original preserved in object storage → format extraction → deterministic parse → human review and approval → LLM compile → human merge confirmation → property graph → a query returns a grounded answer citing a Decision node reached by graph traversal.

Single user. No auth, no users, no roles, no config UI, no audit log, no rate limiting, no tenancy, no supersession, no connectors, no write-back.

What this proves: given human-verified input, does graph persistence plus vector-entry/graph-expand retrieval produce a cited answer that a chunk store could not have produced — and does human merge confirmation make the entity matcher improve over time?

What it does not prove: whether an LLM could extract Decisions accurately without a human (the gate answers that by fiat); whether the pipeline scales (a human gate on every file is the bottleneck by design); which edge-type vocabulary reviewers actually need.

Architecture

Six containers. RabbitMQ is deliberately not among them.

Container Role Exposure
nuron-web SvelteKit. Upload, review queue, diff view, body editor, merge confirm, query box. host
nuron-api Laravel. REST facade. Owns schema nuron_api. host
nuron-ai Python/LlamaIndex. Watcher, extractors, parser, review state, Compiler, persistence, query agent. Owns schema nuron_ai. internal
Postgres One instance, two schemas, separate roles, no cross-schema grants. internal
Neo4j Property graph and HNSW vector index. internal
RustFS S3-compatible object storage. Original document bytes, content-addressed. internal

Arrows point one way: web → api → ai → {postgres, neo4j, rustfs}. nuron-ai never calls upward. Schema isolation is enforced by DB roles, not convention — if nuron-api could read nuron_ai tables it eventually would, and the one-way layering would become a comment rather than a constraint.

Pipeline states: landed → extracted → parsed → awaiting_review → content_approved → compiled → awaiting_merge_confirm → persisted. One Postgres table. The review queue is the work queue — claimed with FOR UPDATE SKIP LOCKED, which is why there is no broker. Both blocking states are real persisted states, not UI steps: a reviewer may close the browser between them and restart-safety requires resuming where it paused.

Two agents

Compiler and query agent. The parser and format extractors are code, not agents. The Curator is out of scope. "Default Agent" is retired as a name — the ingest pipeline is a state machine.

Fixture

Five files in fixtures/watched/. A one-file graph sits entirely within vector reach and would let the bullet pass without traversing anything. Authoring and committing these files is NU-002/#31 — until it closes, the corpus exists only as text in the comment below.

  • A auth-token-decision.md — the decision, attributed in prose (— Basuru, 2026-05-14).
  • B rate-limiter.md — "the limiter keys off the session store". Never mentions JWT, sessions being dropped, or the decision.
  • C cdn-token-caching.md — noise. Superficially about tokens/auth/performance, semantically unrelated.
  • D sessions-table-replication.md — the alias case (sessions table).
  • E a one-page PDF of the same decision as A. Used only by A11.

Query under test: "what did the auth change affect?" The answer is the rate limiter, in B, reachable only via Decision(A) → Entity(session store) → Entity(rate limiter).

Assertions

# Assertion
A1 The query returns the rate limiter, citing a node contributed by B, reached by traversal.
A2 Cited Decision node has label=Decision, author="Basuru", author_source="extracted", timestamp=2026-05-14, and an EVIDENCE edge resolving to Reviewed Source A-v1.
A3 Control — same query with graph expansion disabled (path_depth=0) must degrade or fail. If it passes, the graph contributed nothing and this slice's finding is negative.
A4 A file cannot reach compiled without content approval, nor persisted without merge confirmation.
A5 Re-scan with unchanged hash is a no-op: no re-extract, no re-parse, no re-review, no LLM call.
A6 Change A, re-review, approve v2 → node count stays N, not 2N; content reflects v2; nodes contributed only by B, C, D are untouched.
A7 The session store entity, cited by both A and B, survives removal of A-v1's provenance ref.
A8 A query with no matching Decision returns an explicit "no matching decision" citing nearest entities. Never a fabricated answer.
A9 Ingest A then B. Entity(session store) is a single node with provenance refs to both Reviewed Sources, and A1's traversal path exists without any human link step.
A10 Ingest D. Reviewer is offered sessions table ≈ session store; on confirm one node survives carrying sessions table as an alias, refs to A, B and D. Re-ingesting D is then a no-op — the alias matches without asking again.
A11 Non-blocking. PDF E extracts via LlamaParse to markdown and the deterministic header parse finds the same Subject, author and date it finds in A.
A12 An uploaded document and the same document dropped in the watched directory produce one Landing Zone record and one object.

Why these carry weight:

  • A2's author_source="extracted" stops A2 passing by copying a config value into a node property.
  • A3 converts "it worked" into "it worked because of the graph".
  • A7 is the reference-counting bug a one-file fixture would never catch.
  • A9 proves the cross-document join: the shared entity name is the join, produced by the natural key with no human involvement.
  • A10 proves the compounding property. Every confirmed merge extends normalize(name) → node to many-names-per-node, so the matcher improves monotonically from human input — no threshold, no tuning, no regression. This is the only part of the system that gets better as it is used.
  • A11 is deliberately non-blocking so a LlamaCloud timeout can never invalidate a graph finding.
  • A12 proves two entry points converge on one identity. Without it, uploading an already-ingested file silently duplicates the whole chain.

Decisions

Full rationale in docs/tracer-bullet-01.md §3. Load-bearing ones:

  1. Citation is a Decision node with author, timestamp and resolving Evidence edge — not a filename. A filename citation passes with a plain chunk store and proves nothing.
  2. Compiler emits structured output with an explicit decisions[], not a Markdown document.
  3. author is extracted from content with a configured fallback, plus author_source: extracted | default | unknown. The config value is named source_owner, never author.
  4. Pass 1 is deterministic code: format extractor → common markdown form → header parse.
  5. Every file blocks for human review. Reviewers may edit body content, not just metadata.
  6. Reviewed Source is the Evidence root, not the raw file. Human-vouched content is better evidence than raw.
  7. nuron-ai owns the pipeline and its state; nuron-api owns the user-facing surface.
  8. RabbitMQ deleted — Postgres is the queue. The review queue is the work queue, so a broker would be a second copy of the same state.
  9. "Reply" means v1.1 write-back, not query answering. Query answering is synchronous end to end.
  10. Retrieval is HNSW entry → graph expand (VectorContextRetriever(path_depth=1)). No BM25, RRF or LLMRerank yet.
  11. Embeddings live in Neo4j on the graph nodes, not pgvector — no dual-write between stores.
  12. Content hash is identity, and the hash is the object key. Not a UUID, which would break dedupe.
  13. Re-approve replaces the document's provenance slice. Provenance is a set, refcounted.
  14. Node identity across compiles is a natural key normalize(name) + label, extended by human-confirmed aliases. Under-merges by design; a human resolves rather than a tuned threshold.
  15. Human confirms merges, not links. Auto-joins are shown as information, never asked. Link creation deferred.
  16. Originals in RustFS, content-addressed, read back and re-hashed before acking a write.
  17. v1 formats .md .txt .docx .pdf. No .doc, no OCR — near-empty extraction fails loudly rather than reaching a reviewer as an empty item.
  18. LlamaParse is configurable and off by default. It is third-party cloud; PRD §5.1 promises no data leaves the customer network.

Known gaps, deliberately deferred

  • Local PDF extraction is on the critical path to shipping. LlamaParse is dev/test only under §5.1, so no customer deployment can parse PDFs until a local path exists.
  • Reviewer's edits are lost on re-ingest — mitigated by showing a diff, not a three-way merge. This is the workflow that decides whether reviewers engage or start rubber-stamping.
  • Automatic near-duplicate merge, OCR, .doc, multipart upload, page-anchored citations, typed link creation, gate-on-low-confidence, supersession, Curator re-curation, BM25/RRF/LLMRerank, write-back.
  • Uploaded documents have no second copy — RustFS is Beta and holds the only one. The read-back-and-re-hash check is the only thing between a silent write failure and permanent loss of an original.

Frontier

Children are listed as sub-issues and ordered by dependency. A child is workable when every blocker is closed.

Order and blockers

Story Issue Blocked by Closes
NU-001 Scaffold: compose, schema, env #16 — —
NU-002 Fixture corpus #31 — — (enables A1 A2 A3 A5 A6 A7 A9 A10 A11 A12; see footnote)
NU-003 Deterministic core #17 NU-002 A2, A5 (hash half), A6 (delta half), A7 (refcount-logic half), A10 (matcher half), A12 (identity half)
NU-004 RustFS object storage #18 NU-001 A12 (storage half)
NU-005 Watcher and Landing Zone #19 NU-001 NU-003 NU-004 A5 (scan-skip half), A12 (directory half)
NU-006 Format extraction seam #20 NU-003 NU-005 A11
NU-007 Review gate 1 #21 NU-005 A4 (content-approval half)
NU-008 Compiler #22 NU-006 NU-007 —
NU-009 Persist to Neo4j #23 NU-008 A6 (persist-apply half), A7 (refcount-apply half), A9
NU-010 Merge gate 2 #24 NU-009 A4 (merge-confirmation half), A10 (confirmation half)
NU-011 Query agent #25 NU-009 A1 A3 A8
NU-012 nuron-api #26 NU-007 NU-010 NU-011 A12 (upload half)
NU-013 nuron-web #27 NU-012 —
NU-014 Delete orphan vitest files #28 — —

Every "half" split is intentional, not an ownership gap: each half is independently real risk
— a correct primitive in one story cannot make the observable behavior correct if the story that
exercises it end-to-end is wrong, and vice versa. A5/A6/A7/A10 previously read as claimed in full
by two stories with no qualifier; they're relabeled above to match the pattern A4/A12 already used
correctly.

Open frontier: NU-001, NU-002, NU-014. Everything else has an open blocker.

Nothing is implemented. A first pass at NU-001 and (old) NU-002 was written before these stories
existed and has been deleted — code ahead of the tracker was the wrong way to start. The repo
currently holds only docs/tracer-bullet-01.md and the pre-existing planning artifacts.

The fixture corpus text is preserved in a comment on this issue as the source of truth for exact
wording; NU-002/#31 is the story that turns it into committed files. The constraints on it (B must
never mention JWT; C must be plausible noise; A must carry attribution in prose, not frontmatter)
are what make A1 and A2 test anything.

Activity

  1. 7 remaining items

  2. BasuruK commented on Aug 18, 2026

    @BasuruK
    OwnerAuthor

    Retired in favor of #30 — see updated issue body. All 14 child stories reparented, Frontier table moved there. This avoids maintaining the same architecture/assertions/decisions in two places.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    wayfinder:mapWayfinding map: scope, decisions, frontier

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions